AI, Actually – Episode 23: Loops vs. Graphs: What These AI Agent Design Patterns Mean for Your Business

So fast, I thought,

you know, this would be an interesting topic, right? Six months ago where it was all about, you know, loops. Remember the Ralph Wiggum loop, right? And you

know, all these, all these sorts of things. And so all of a sudden, you we talked about context

engineering and loop engineering and now, you know, graph engineering. And I just thought it’d be a really interesting topic to cover with this group. I know it was knee deep in all the above. So but so before we build out some AI org chart, I figured, you know, maybe we we’ll we’ll

talk about, you know, how these things work. So maybe the the first thing to answer is well, what first of all, what I didn’t even know what a loop was, right? So what so what’s a loop? How do I use it? How do I think about it?

who wants to who wants to take that first one?

Shanti Greene (02:16)

I’ll I’ll throw something out there, which is that

a loop is a type of graph. It’s a proper subset, right? We talk about like DAGs a lot, acyclic graphs, but a loop is just a directed cyclic graph. It’s nothing that is so different or special. It’s a different way of thinking about in this context, where are we going to let the model and the non-determinism of that model live? Are we going to say the model controls that and that’s pretty much what the looping piece was? Or are we going to

put some more structure around it that controls it, and that’s what people are referring to as graphs now.

Pete (02:52)

And for the for the for the business person in the audience, help me help me come back to all right, what what’s a loop?

Mike (02:58)

the way I would think of it is you know, we we’ve been talking about, you know, tires and engines and pickup trucks, right? That’s the various parts of an AI system, from a kind of abstract level. And now somebody’s said, no, it’s not just pickup trucks. It can be semi tractor trailers, it can be sports cars, right? So so the graphs is as you know, as Shanti is saying, they’re they are

what we’ve been doing is a graph. It’s just a specific kind of graph, right? the

Pete (03:23)

What why would

people use loops? Like what what was a loop for and why was it different and w like how would I use it and

Mike (03:28)

Yeah. Yeah.

It’s a very utilitarian approach that basically is like the Ralph Wiggum embodied. You just said if the AI was didn’t finish, then keep trying. You just tell it

Pete (03:37)

Keep going.

Mike (03:38)

to try again and try again and try again. Right. That was the the classic loop. And then the question is, how do you break out of that loop? Because it can’t go on forever. you know, do you break out on time or budget? Do you break out when the job’s done? How do you know the job’s done? Right. So so these these loops got a a sort of a life of their own. And that’s what we called an agent, right? We called it an agent when

When you could run a loop and it would intelligently stop at a good point and it would finish a job and it would record its history and all that, right? Now, stepping back a a point from there

Shanti Greene (04:05)

Stepping back a point.

Mike (04:07)

though, what we realize is, well, the reason we liked agents is because they could do things that weren’t well defined upfront. You could say, Go figure out the problem and it would stay in a loop, trying again until it got it. But then the thing is, some things that we want AI to do aren’t go figure out problems, right? We want it to

use specific

sources. We want it to record what it did. We want it to verify the results it got. So that’s more like a graph, right? That’s more like a a specific way of doing things that might contain some loops, but it’s an overall structure

that would be a graph.

Pete (04:38)

I keep trying to bring it back to like what’s a b business example of

a loop? Like if I’m in finance, like what would I use a loop for? Like what I don’t even

Stew (04:47)

Yeah.

I I think it’s it’s probably best I’ll try to explain a loop and a graph, maybe going back to more human

terms, if you think of it that way. Right. So imagine a loop is like one of your people in finance, right? hey, go create this spreadsheet for me that does A,

B, and C, right? And and there’s some

What the difference between the the loop and the tradition and the pre-loop approach, which is where you know you you put in a prompt is and it runs for a period of time and it tries to achieve that effort and stops, is it’ll just keep iterating until whatever exit criteria you define are done,

Pete (05:28)

Right, here’s your spreadsheet. Yeah.

Stew (05:37)

right? and and we’ll keep working on that spreadsheet.

So let’s say I’m an analyst, I’m financial

analyst. My goal was to create that spreadsheet. I’m in a loop working on the spreadsheet until you know I whatever criteria I had in my head is or standard operating. I I accomplished the goal. I

Pete (05:51)

Yeah, I finally I finally got to the thing that produces the output I wanted.

Stew (05:55)

have something good enough now to share with others. Think of now what a graph starts to enable is a little more of almost like an org chart kind of view.

Right. So think about this now as you’re putting much more you’re putting much more discrete boundaries around certain types of activities. And you’re saying, inside of that activity, I’m gonna have an agent do this work. But there’s a handoff mechanism and approach where I can hand off to another agent that can do other work.

sorts of things in in that regard. But the but it’s th that’s kind of the graph versus the org, right? And so now a graph might be like, my whole process for putting together this spreadsheet is not just me the analyst creating the spreadsheet. It’s then it gets sent to, you know, someone else who’s a designer and makes the spreadsheet pretty and then gets sent to someone else who’s you know a really meticulous fact checker and make

Mike (06:50)

To certify the facts, right?

Stew (06:51)

sure everything is fine and then it gets

sent to my boss who looks at this from a certain angle. And then it gets sent to someone who’s looking at it from a client. So you can build a kind of a whole structure around what it takes to really get that deliverable put together at a at a you know at a holistic level and with clear boundaries between who’s responsible who’s responsible for what. It’s just in this case instead of the who being a human it’s an agent.

although you can have graphs that include human in the loop checkpoints, which is we’ll get back to I’m sure, but is one of

Pete (07:23)

Mm-hmm. Okay.

Stew (07:25)

the advantages of a of of of a graph is it can give you checkpoints.

Mike (07:28)

Yeah, like I like I like all

Pete (07:29)

Okay.

Mike (07:30)

these analogies a lot. The the accounting analogy is really good. And let me just lay layer on another piece to this, which is the generative element, right? We all love models able to generate content, not just answer choices. Generation comes into these graphs through the notion that at the end of generating the spreadsheet and going to the boss and all that, the the agent the sorry, the graph would actually have an agent that looks at what it just did and and improves itself, right?

That says, I I’m missing a step here. That would be better to have another review step in here. So it changes what that graph is for the next time. Or maybe it saves what we just did as an example that feeds into the next time because it was really good. It got the thumbs up, right? That’s the generative component that’s in fact in these graphs. I’m gonna I’m gonna share a screenshot here that

Pete (08:15)

Yeah, yeah, good.

Mike (08:16)

I think is is maybe gonna add add some value here. Let’s see. yeah, you guys.

So so this guy basically over here on the left side is kind of where we came from, right? We’re all sort of used to the user prompt to an LLM and we you know, we all use GPT, right? And then there’s the the

Pete (08:30)

Yeah. Right.

Mike (08:31)

the the idea of the loop. and then there’s you know the idea with tools and of loops with tools and reasoning. This is like the Roth Wigam loop, right? And then you get all the way over here into the graph area where not only are we involving some of these loops like we did before, but we also have planning happening. We have verification. As Stu said, there are humans that are in the loop.

You know, there’s some subagent that’s being done, right? So one thing that’s important in this context to to talk about is is the I feel like some of these words that we brought in, context and and tool and skill, right? They they they have real world me world meanings and they come into the AI world with a kind of a very different sort of a meaning. I don’t think that’s true for graphs. I think graphs in the AI world mean exactly what they mean in the real world. They they are they’re as Shanti was saying, they’re a layout for how things chain together.

how they connect, how they flow backwards, how they flow forward. and and I and that’s one reason why I think this is a terminal point, meaning meaning we’re we graphs are not going to go away in the AI world. It’s not like we we’re we’re we’ve we’ve abandoned some layers of things that we used to worry a lot about. I think graphs are here to stay because ultimately they let us scale up and down in terms of what we’re asking our AIs to do.

Shanti Greene (09:40)

Yeah, I say when you think about something like Langchain, which started out as a direct linear chain, what’d they create? They created Lang graph because they realized, wait, a single linear piece doesn’t actually work for this. We need a workflow that could support going backwards, fanning out, spanning parallel agents, or just having multiple processes happening simultaneously, maybe reviewing those processes. And that’s like now a S DK for them.

Mike (10:05)

Yeah. Well and and the the sort of the tacit implication here that’s really interesting is that AI is acting more like people, right? We’re we’re having to adopt our AI methodologies, adapt our AI methodology to be more like what people do because the AI is acting more like what people do. And so now we need these bigger tools to put this stuff to work, right? To to make it make it yield what we need from it.

Stew (10:29)

I I think go back to the

people analogy as well. I I think it in a lot of things by the way, loop engineering and graph engineering are not competing frameworks. You’re very often gonna have

loops running inside of this broader framework which is put together with inside of a graph. There’s all sorts of things like that. But I think applying it back to that human idea of I’ve got a team of people working on a problem. I’ve got some

division known division of responsibilities who’s responsible for what dividing point standard operating procedures you have that mode versus let’s go back to the spreadsheet example right versus the single person putting together the spreadsheet once you get that team together there’s all sorts of other th

Pete (11:12)

Mm-hmm.

Stew (11:21)

other things have to be thought about now right so and there’s reasons and and there’s good and bad that comes from this

But if I’m just working on the spreadsheet, I’m putting everything together, I’m

do I’m I’m going through all of these bits.

one, you lose a little bit of of

auditabil auditability and governability around those activities, right? Like like I can get in I can get in a state of mind, right, around it should work this way. And but maybe someone more clear

headed would think it’s it would be in another way. So division responsibilities helps there.

But now when I hand it off, I also have to, you’re kind of handing off like

here’s a version, and I’m handing it off to someone else. Right. And so there’s shared state and context that has

to be shared

Pete (12:08)

Mm-hmm.

Stew (12:09)

between

all of these different people participating in this project. And I think when you start thinking about like,

Let’s say I can accomplish the same goal through a loop, one agent running

a long time in a loop, versus a series of agents tied together through this graph. one of it can be an advantage or disadvantage depending

on your use case. But one of the things that this as you move to the graph, what you’ve got to do is you’ve got to have some sort of shared state, some sort of being able to

to do this.

An advantage of that is

it does create an audit trail as well, right? Like I’m having to produce things about my work at each step to hand it off. And it creates a little the way agents work, it creates a little more predictability

as well. As I can design on and say, like, hey, I’m always gonna go through, you know, there’s always gonna be these handoffs to these agents at roughly

these points.

to make sure certain things are done versus if you put it in a loop, it may skip some of those points as long as it’s able to kind of check off that it that it created whatever the exit criteria we

Pete (13:16)

Mm.

Stew (13:23)

created was. So there’s just a few things along those lines that I think tie back to Mike’s point around in some ways we’re kind of applying

the idea of an

of an org chart and standard operating procedures from the human work

Pete (13:37)

Yeah.

Stew (13:39)

into this into this agent design framework.

Shanti Greene (13:42)

You could also think about it as who’s writing the edges of your graph. If the agent is writing them themselves, you’ve got a loop. If you’re writing the human is in there creating the edges ahead of time, then you’re creating a quote graph instead. If you want if you really want to differentiate

those.

Pete (13:58)

can you explain what you mean by edge, Shanti?

Shanti Greene (14:01)

Sure. So graphs have connected points. We call those nodes. And they’re connected by edges, the things that you traverse to get from one place to another. And that’s usually in this metaphor an activity that the agent might do, some information that it might be transferring. So it’s doing something at the node, taking that and then reaching out to the next state that it needs to go to, and that state is going to be that edge.

Stew (14:23)

Just think of if you were drawing a workflow, the node would be the circle

and or two circles, right? Would be two nodes. And the edge would be the line that connects them,

right? think about that simple way.

Pete (14:34)

Right, right. And the line represents the data that’s be moving

from one to the other.

Shanti Greene (14:39)

Yeah, the the other

nice thing when you think about the how you might want to separate these is if you want to separate the responsibilities of your agent, it’s a lot easier to do that in this quote graph framework because you can have more specialized agents rather than one agent that has to keep the all of the context for what is it doing, checking is it done, is it correct, and then just looping within itself versus I’ve got a graph that might have an orchestrator, I might have ten agents that have different capabilities.

That are each looping within themselves, but I’ve got one that can have the responsibility of only checking the results of my subagents. Each one of those can have can also then take minimal context. It only needs the context required to do its specialized job, which tends to lead to better success because you don’t have to try to fill its context window with everything that’s happened. Here’s exactly what you need to do your immediate job. We’re not going to give you anything extra.

no fluff around it, which should help it be successful.

Pete (15:38)

Yeah.

Yeah, you’re saying one one advantage of this call in in the world of graphs, this division of responsibilities, as you’re sort of describing it, is that each one then has a fresher context, it has a a clearer mind about an an understanding of the problem and how to solve the problem is less likely to get confused, more likely to stay on task and to complete the deliverable. Is that what you’re saying?

Shanti Greene (16:00)

Yeah.

I think there’s a lot of real practical implications for why you want to think about things that way when it comes to management of multiple agents or agent swarms,

Pete (16:09)

Mm-hmm.

Shanti Greene (16:10)

being able to understand that they’re different, they’re doing different things. It’s not just one agent with a hundred tools. And then you’re trying to get into your observability

Pete (16:16)

Yeah.

Shanti Greene (16:17)

stack and be like, well, which tool did it choose? And why did it make that choice? What

Pete (16:20)

Yeah.

Shanti Greene (16:21)

was it my tool definition that got it confused between which of two things it could choose? Or I actually just separate them out and you’re like, it made a bad decision.

Like called the wrong sub agent.

Pete (16:30)

Okay.

Shanti Greene (16:30)

That was easy.

Pete (16:31)

Mm-hmm.

Mike (16:32)

Yeah, so that it’s an interesting one that we that we something we’ve done lately in a couple of our implementations is add what we call an investigator agent. And the investigator’s only job is when something goes bad, it has to look at what happened. and what’s fascinating is the the the number of things that it can find that went badly, right? Obviously maybe the API provider was down or something simple like that, but also it could be that, you know

last Tuesday somebody added an MCP tool and and then on Thursday whoever it was changed the definition of the elements in that thing and now it broke the agent, you know, a thousand miles away, right? So so the just the ability to have that investigator in the mix is interesting. So now

suppose I I previously had what I would what I call an agent and it was doing a job. And now I’ve got this investigator that helps me fix that agent when it’s broken. Well now I’ve got a graph, right? Those those all of that together becomes the graph of how my AI is

not just operating, which which is what the agent does, but it’s also continuously improving, which is what the investigator is giving me, right? So so you can you can start to see how all of the AI operations in an

enterprise can be in a common graph, right? They can be in that kind of big picture graph that lays it all out. What

Shanti Greene (17:39)

big picture graph that lays it all out.

Mike (17:41)

are the infrastructure pieces, what are the observability pieces, what are the cost control pieces, what are the actual actions, right? just where that

wasn’t possible.

if you just

if you took a you know sort of a a a an agent s specific or s a a a chat bot specific view of enterprise AI, then you’re kind of stuck at that agent level without seeing the big picture.

Pete (18:01)

Yeah.

Is is last you were speaking, I thought, it feels a little bit more like single player mode and or multi multiplayer mode. Is that a I know it’s probably pr not perfect, but is that a reasonable, simple way to think about it?

Mike (18:14)

I like that a lot. Yeah, I know the the analogy is really good

and and it can be P V P or P V E, right? You could be playing against the environment so that so everybody

but you is an agent or it could be playing against other people where there are things in the

Pete (18:25)

Yeah, interesting.

Mike (18:26)

environment that are non player characters, right? so yeah, the analogy holds really well. And I I yep.

Pete (18:28)

Interesting. Yeah, yeah, yeah. Let me

let me throw some stuff I’ve been working on to you guys. And I’m just trying to see if I make sure I I got it. so for example, an agent might be so we built a bunch of tools around our

our own financial finance close process, end of month close, right? So one thing that it does is it we have all these credit card transactions that come in and the the the job that it does is it looks all the credit card transactions, look at the GL codes, it e even looks at

People’s calendars to just see what they were what what they were doing to figure out, all right, what’s the best classification for this expense? And it runs from top to bottom. Sounds like agent, like or just a simple may and maybe I don’t know if it’s even a loop necessarily.

It’s just a simple i agent loop.

What do you guys think?

Mike (19:16)

Yeah, so th I I would call that an agent. And the the main reason is because it it’s gonna run f from beginning to end, and it it’s gonna it knows when to start and it stops when it decides to and that’s it. That’s all it does. Yep.

Pete (19:18)

Yeah, it’s just a simple agent, right? Yeah, yeah.

Yeah, okay. Okay. Yeah, yeah. All

right. Now now b up a level, I have nine

of these different things. We have,

Mike (19:32)

Mm-hmm.

Pete (19:33)

you know, what should be invoice and you know what categorize those expenses and what what does revenue look like? There’s just a there’s about nine of these different things that finance does. All is under this close agent essentially. if I just say, instead of just say go do the ramp thing, I say hey close the month, and it runs each of those.

Mike (19:55)

Right. Yeah. I I mm.

Pete (19:55)

Sounds like graph, right? Is that a way to

Shanti Greene (19:57)

Yeah, and

and maybe that first one is an orchestrator that calls each of the other ones

Pete (20:01)

Mm, yeah. Right.

Shanti Greene (20:03)

in sea potentially in sequence, maybe in parallel, depends on whether they need

Pete (20:06)

Right.

Shanti Greene (20:07)

their the outputs from one to feed into the other. If it doesn’t,

Pete (20:10)

Yeah. Yeah.

Shanti Greene (20:11)

then yeah, that having nice fan out and being able to do them in parallel will be much faster. So you get some

Pete (20:14)

Yeah. Yeah.

Shanti Greene (20:16)

nice savings there. They don’t need to actually talk to each other. And then you can take that logic of

Did I get the finished and correct output from this agent? Move that into your orchestrator, and when

Pete (20:24)

Right. Audit this thing. Yep.

Shanti Greene (20:27)

all of those are complete, it can call a synthesizer and say, great, here is everything you need to now close the month.

Stew (20:33)

I was just gonna say I think that’s the next that’s gonna be the next buzz term, by the way. Like you kind of joked about it earlier, Pete, where we went from like prompt engineering to context engineering to har harness engineering to loop engineering to graph engineering.

Pete (20:45)

Yeah, yeah.

Stew (20:45)

I think orchestration engineering is is, you know, a good candidate for an upcoming one. Inclicit in a graph

Pete (20:48)

Hmm, okay. Yeah. Next that’s your vote, yeah.

Stew (20:54)

is that there is some orchestration between

it, but I think, you

Pete (20:57)

Yeah.

Stew (20:57)

know, as you make that graph bigger and bigger,

it’s

Pete (21:02)

Right.

Stew (21:02)

of these orchestration challenges really become such a such a focus on on how do you get get that right and in and

Pete (21:10)

Yeah. Yeah, it’s almost

like the the agent is sort of the the individual, the graph is

maybe the department, and then there’s there’s corporate level, there may be other levels actually depending on on what you’re going.

Mike (21:21)

Well, and and this is where

it it’s interesting and it’s sci-fi, but it’s also very real,

right? I don’t know if you guys remember in two thousand one the Space Odyssey when they sent the guy in to fix the computer. He wasn’t an engineer. He was a psychologist, right? because the AI was broken and they needed to find out why. And and I don’t think we’re

Pete (21:36)

that’s right.

Mike (21:40)

too far from that. I think there is a point at which this context engineering or graph

engineering or process engineering, I mean it’s all it’s management.

It’s like managing people. The humans

are augmented by the machines. The machines are doing some of the tasks by themselves. The whole thing has to get laid out and

Pete (21:54)

Mm-hmm.

Mike (21:55)

made to work together in a way that’s auditable and and and improved, relative to what you could do with either one alone, right?

Pete (22:02)

So how would you summarize the benefits of this this approach? Who wants to take a stab at that?

Shanti Greene (22:07)

Actually that

that auditability is a nice one. When you need to give something to a compliance officer and you have that graph,

Pete (22:13)

Mm-hmm.

Shanti Greene (22:13)

it’s a lot easier for them to understand what’s happening because then it becomes like an agentic workflow where here are all of the different steps that happen and here’s how you get from one step to the other. Like what’s it looking

Pete (22:24)

Mm-hmm.

Shanti Greene (22:25)

for to proceed from step one to step two? Versus if it’s inside an agentic like a s just a single agent loop, right? We asked the model to keep going until it got to where it needed to go. It got there.

Pete (22:36)

And it did. Yeah.

Shanti Greene (22:37)

We think

Stew (22:39)

Yeah, and it’s

it’s saying the same thing in a different

in an in another thing, it’s auditability and predictability as well, right? Like with these agents, when you give them

Pete (22:47)

Hmm. Yeah.

Stew (22:50)

a task, you know, give the same task to clawed code five different times, right? It

Pete (22:56)

Right.

Stew (22:56)

it may reach the same result, but but do it five different ways, right? And and that’s

Pete (23:00)

Yeah. Right.

Stew (23:02)

kind of

and there’s still gonna be that inside of each circle in the node,

right? But

Pete (23:08)

Yeah.

Stew (23:09)

there’s more predictability in the lines, if that makes sense.

Pete (23:12)

Well, and it sounds like because you’re isolating

the job, and it sounds like because you’re isolating the job of that agent, you that agent itself should be much more reliable.

Shanti Greene (23:21)

Yeah.

Pete (23:21)

Right. And

Mike (23:22)

Yeah, mean, especially

if you if you could sandbox it, right? Really a you know, agents like following the open claw standard and the the sort of agents that can write

their own content, they’re almost like the Marines, right? They they just whatever you got, give the job, they’re gonna

go get it done, right? But you still need the general telling you where to send the Marines, right? other otherwise you could have a lot of stuff going on in a lot of places, right? And so that’s that’s really where these benefits come in.

Pete (23:39)

Yeah, where to go. Right, right, right, right. Yep.

Mike (23:45)

And, you know, it it’s interesting.

Because so many of the we we’ve been we’ve been amazed by all the work that the AIs can do, but now that we’re getting into the

harder, thornier problems, we’re getting into the

Pete (23:55)

Mm-hmm.

Mike (23:55)

greater needs. And so, you know, a a simple example like if you were designing a new employee onboarding, right? You’re gonna you’ve gotta you’ve gotta get their business cards printed, you’ve gotta make sure somebody greets them at the door, make sure they go through training, get their i9 filled out, make sure they’re in pay all those things, right? A lot of those are gonna involve a human, maybe verifying documents or a human approving a payroll step.

So you can’t just say, well, run the agent until it’s all done. The agent has to run and go to sleep and wait for an event and then come back alive again. That’s not something that an agent by itself does. That’s a graph, right? So this graph capability, you might have other other

Pete (24:23)

Mm-hmm. Yeah.

Mike (24:29)

cases where you have, you know, within one organization, you have agents need to run in a factory, agents need to run in a data center, and and the results of those two need to get brought together. Again, it’s a graph. The agents are gonna be in one of those places at a time.

not in all those places and and they can only sort of reach out and control to s to a limited extent. So just the idea that that we can have these long running processes that are curated

Pete (24:48)

Right. Right.

Mike (24:55)

over time where the process itself is changing and the results are being checked, right? That’s that’s the richness that you get out of a out of a graph capability.

Pete (25:03)

I was just gonna say I didn’t hear anybody talk about speed and if I did I missed

it. Is that is that a a benefit or no?

Mike (25:10)

To the extent that you get repeatability, right? A lot of times

repeatability comes from agents through iteration, which then slows things down. So to the extent that a graph is there to say, no, no, we have a specific way that we

Pete (25:16)

Mm-hmm. Mm-hmm.

Mike (25:22)

know how to do this particular task. That’s not an agent.

Pete (25:23)

Yeah. Yeah.

Mike (25:25)

We have this other task that’s a little bit more unknown. Let’s figure out how to do that part, right? That that’s

Pete (25:29)

Yeah.

Mike (25:30)

the that that’s definitely an improvement you can get from graphs. The other thing with a with a

It’s important when you’re assessing the performance of the AI is how much did it cost and how long did it take? You know? Yeah. Right, right, right. Yeah. Well it and and and I I think I think the reality is in

Pete (25:40)

Right. I was just gonna come to that. This sound sounds expensive.

Mike (25:47)

a graph scenario, because it is better defined, you can put a price tag on it. You can say, What is it gonna cost to build this? Whereas if you just hand it all to an agent and say, Go nuts, right? You you you don’t know how long it’s gonna take. You don’t know how what budget it’s gonna have, and the quality of the result that’s gonna come out, right? So

By making those leave the pieces that need agency in an agent box, but make that box as small as you can. Put those boxes together with a graph that gives you the predictability, the cost

control, the reliability and the testability.

Pete (26:15)

And and what what makes

the what makes the cost more predictable in that example?

Mike (26:21)

Yeah. So so think of it as if you you have a task that an agent’s gonna be asked to do and nine times out of ten it’s gonna do it consistently the same way. But but that tenth time it’s gonna go wrong and repeat the task. Well, if you already know how you

want it to do the job, don’t make an agent do it because then you’re you’re gonna have a ten percent cost overrun, right? Instead, do it in a way that’s prescribed and that’s gonna lower your cost. And

And that that effect

compounds because we’re not talking about one agent, we’re talking about a hundred agents that are in these systems. And there’ll be massive increases in the cost and the time that comes through. It you either have to accept unreliable outputs or you have to accept higher cost. If you want to just stick to that sort of hands off approach, let the agent do it. You put it into a graph and now it’s well defined, the sequence of steps that it’s gonna go through, the points at which you can measure inside that process, the the dashboards that you can get out of it, right? That’s what makes it

confinable. You can measure it, therefore you

can confine it.

Pete (27:18)

I would imagine though there’s plenty of scenarios like when you’re fanning things out, for example, to have multiple multiple agents do something at the same time, you could significantly drive your

expense up.

Mike (27:30)

well so I think you would drive your expense up if you didn’t need to run those all at once, but you chose to for speed. but if you need to run them all, you just run them for cont yeah. So so the the if if to me it it always comes down to the first thing you have to do is get the job done and then you

Pete (27:40)

For context clarity, maybe.

Mm-hmm.

Mike (27:51)

can start optimizing, right? So then that if once you can prove that the system is capable of the job.

Shanti Greene (27:55)

system is capable of the

job.

Mike (27:56)

Now you can start saying,

all right, what’s extra? What’s being repeated? Where are the efficiency points? Right? Where can I probe? Right. It’s it’s always my my favorite is when you know you see somebody who’s running wires, maybe for just speakers or something, and they tie it all down perfectly, and then it doesn’t work, right? Now run the wires first, make it play some cool music, and then tie it down, right? Then tighten it up. So same same thing here. Prove that it can do the job, right? And then once you’ve proven that it can do the job, now you can go and it’s called pertub.

Pete (28:12)

Mm-hmm.

Mike (28:25)

perturbation theory, right? You you change one thing at a time, right? Change the model to a cheaper one. Did that fix it? Did that make it cheaper or not,

right? Because sometimes a cheaper model runs longer, right? so change the, you know, change JSON to YAML, whatever the things are that you could do. Change one thing at a time to

drive that cost down and and you use regression to never lose the functionality that you’ve got.

Pete (28:47)

Yeah.

Any any guidelines you guys would offer like

you will when agent, when graph, when and obviously there’s some very some variations here, but what any rules of the road you would sort of offer to folks?

Shanti Greene (28:59)

I mean I think if you look at the complexity of the job, would it benefit from parallelism or not? Like if it’s not going to benefit from parallelism, if it’s not that complex, you

Pete (29:09)

Yeah.

Shanti Greene (29:09)

don’t need to bother with the graph potentially. But it doesn’t really hurt to start thinking with that framework, because if you think about everything that way, some things are just simpler graphs than other things. So if you had to just pick a method of thinking about problems, it’s not a bad way to think about.

Mike (29:25)

Mm-hmm. Yeah, I would say an easy

Pete (29:27)

Build?

Mike (29:28)

one for me is if if one human is going to do something and wait for it to finish, that’s probably an agent. Maybe even just a loop, maybe maybe even just a prompt, right? if many possible humans need to do something and many possible agents could be involved, that’s probably a graph, right? It’s for if it’s a multi-day, if it’s a long-lasting thing that nobody’s that I that I need done but I’m not waiting for, you know, then it’s going to be more like a graph.

Stew (29:52)

You know, I graph is a little bit of a confusing term. If you say flow chart, I’d like to say like you know, I’m building a visio diagram of this

Pete (29:58)

I know. That’s what I look at and I’m like, Well that’s a flow chart.

Stew (30:05)

of of this process that I’m trying to flow, you know, like I I think if it’s if if it’s simple enough that that you

that that you feel like th that’s not really a flow chart so much, right? It’s just a checklist. that’s really, I think, when

you go more to just the single agent, when when you start to think about well, how is this getting done, regardless of who gets it done or

or how

you you start to wanna break that down into a flow chart. I think that’s when you need to start thinking about like, well, should I just just you know, a graph is just basically encoding that flow chart into what you’re you’ve engineered. So I think that’s kind of the way of of thinking about it.

Pete (30:48)

Make sense.

Mike (30:48)

Yeah, I mean as as as Stu was saying that in my brain, I was thinking about, you

know, a power source. Let’s say I I want a power source. Great. Here’s your here’s your Duracell double A battery, right?

That well no, but I need, you know, triple redundancy and hundreds of thousands of watts and

distributed anywhere. Okay, well I’m like another power source, right? The first one is a very simple power source, the second one is much more complex. They’re

both doing the same thing at the end of the day as far as the the micro level action of the result.

But clearly one is a much bigger job and needs a a much bigger definition. And to me, to me the the the ultimate thing that that proves something is a graph is when it needs to correct itself, when it needs to observe its results and make changes for next time. you know,

Pete (31:28)

Mm-hmm. Mm-hmm. Yeah. I think that’s good.

Mike (31:31)

which again makes me draws me back to that’s like humans. It’s the performance review cycle, you know, that comes. It’s the organizational transformation cycle that we go through.

Pete (31:38)

Let’s come

l I I like that you went there because I could envision some people listening to this and go, Well, wait a minute. Sounds like an org chart, sounds like replacing me. Like what what help me understand that and how do humans relate to this new org chart, if you will.

Mike (31:55)

Look, I it’s been said many times that that, you know, AI won’t replace people, but people that use AI will replace people. And I and I think that continues

Pete (32:04)

Yeah, yeah.

Mike (32:05)

to be true. you know, the AI by itself is is a is a it’s a bulk tool to handle a lot of things and humans that wield it well and it and it’s bidirectional, by the way. Humans teach the AI and the AI teaches the humans, and when that works synergistically you get superhuman performance that that is, you know,

supercharging companies. But one opinion that I have on that space is that it doesn’t create a need for fewer humans. In fact, I think it creates a need for more human labor because well think of it as a you know, if if a let’s say a very specialized skill set was needed to do a certain task, y you would be limited to how much of that task you could do by the specialized skills. Now imagine that anybody could do that task if they have AI helping.

Well, you’re gonna have a lot more bandwidth to do that task. And now a lot more people

Pete (32:51)

Yeah.

Mike (32:52)

are gonna want that task done, right? Whether it’s you know, whether it’s a specific kind of writing, whether it’s a specific kind, you know, of of tax advice or legal advice, all kinds of things, right? That that, you know, product design. I mean, there’s just a lot of activities where where people augmented by AI, like programming is a great example. There are more people programming than ever because

more people can do it now, does it mean that we need fewer programmers? Probably not. It just means we have more software, right? That’s a that’s

Pete (33:22)

Mm-hmm.

Mike (33:22)

the that’s the way this, you know, I mean, if we had if we had the the the the marginal value of an additional accounting clerk in the in an average corporation, right? Is whatever that is. And they they they we live with the we live in this sort of constant tension of too much work and people burning out versus what the business needs. And we just we just find a kind of a minimum that everybody can agree on.

Well, if each accounting clerk can do a lot more, you’re gonna want more accounting clerks, not fewer. because you you can because every one that you add

Pete (33:51)

Yeah.

Mike (33:52)

is more valuable to you until you saturate, until you get the job done. But we haven’t found that end in a lot of job functions. We haven’t found certainly

Pete (33:58)

Yeah, it is it is interesting.

You know, Stu, you sent something out, I don’t remember the name of the article or what it was called, but it had some really interesting examples of, you know, there’s a cook. There’s cook A and cook Chef A and Chef B. Chef A just follows the recipe top to bottom and and does the thing. Chef B realizes that, the oven’s running too hot, I need to make this change, or the the olive oil was was the wrong w not what I wanted, or I ran out, so now I have to improvise and so on.

And they’re they’re both going to end up in a very, very different place. Well, because one just followed the recipe and one was actually connected to reality and was able to sort of adjust on the fly. I feel like in just my own working with agents, I just feel like that doesn’t go away. Like big the business is constantly changing. We have new products, there’s a new pricing model, there’s a new, you know. So I just think it becomes more and more cre and and it’s and it takes away from this.

The work

that’s happening, like c so who wants to sit and categorize 300 receipts and what and what GL code they belong to? Like nobody.

so I I’m I’m I’m with you, Mike. I think very much requires a lot more human. I think it’s more engaging, more interesting work too.

Stew (35:08)

Yeah,

I think this is all a little theoretical in our heads a year

or two ago and we didn’t know how it would go. But I I think that with

Pete (35:15)

Mm-hmm.

Stew (35:15)

more time that goes, you know, the better we get at the better AI gets at software

engineering, the more engineering people want in the world and the more humans it takes to drive that. And I think that’s

Pete (35:24)

Yep. There’s no doubt.

Stew (35:26)

happening through a lot of other domains. It’s kind of like

Shanti Greene (35:28)

Yeah, I I

think that’s where we’ll also see the benefit of like when Hermes Agent came out, it started introducing the idea of self-improving skills. Like it’ll do it once, ask you how it did, or look try to evaluate its success, and then take the corrections into account, rewrite the skill with those things in there, and do it again so that every time you actually use the agents and they call the tools, each of those components can get a little bit better. Because the hardest thing to do, right, is write true automated evals.

To understand. Some of the things are difficult to evaluate. But if we can have humans doing that and saying, like, yeah, this is pretty good, but even implicitly, because if their follow-up is do it again or do it again, but do this, you don’t even have to ask. You now know it wasn’t that good, and you can go and improve that skill once you get to that final result where the humans accepted it, and our AI gets more powerful, and we can start thinking about the next set of problems and how we

Pete (36:22)

Mm-hmm.

Shanti Greene (36:22)

want to structure those, how we want to set them up.

as opposed to trying to think about the mechanics of the problems and what is it going to look like to do this.

Pete (36:31)

Okay.

Mike (36:31)

That’s right.

Pete (36:31)

Yeah, I agree. final maybe final question and give you guys each your your final word on. So b based on this entire topic, what are some of the thoughts you might just want to keep one or two things things to keep top of mind for executives that might be listening to this? Shanti, why don’t we go Shanti Stew Mike?

Shanti Greene (36:49)

Yeah, I think graphs are a really useful way for thinking about problems, thinking about the solution to your problem. They’re very visual, which makes it easy for architects to build and understand and explain them to lots of different people. You can see what’s happening, and that observability makes it easier to maintain these things in production.

Stew (37:06)

I I think it’s

Pete (37:06)

Two.

Stew (37:07)

yeah I think you’re just seeing as

we build up this long chain of buzzwords from prompt engineering to context engineering to harness engineering, yada yada, all the

Pete (37:16)

Yeah. All right.

Stew (37:18)

way down to graph engineering. I think you’re just the the meta thing is agents are getting more

capable. And and you know, and as they get more capable, we’re able to obviously give more and more to them.

But that is its own level of architecture and engineering, right? That’s that’s having to grow up and mirror a lot of human constructs over time. But I think the big takeaway is

it’s we can build so much more sophisticated things

today than than we realistically could have a year ago and enormously more than two years

Pete (37:51)

A year ago. Yeah.

Stew (37:54)

ago. And I I think this is just the next phase of that evolution.

And and and it’s really starting to get real now, you know, like it was

Pete (38:04)

Yeah.

Stew (38:06)

i in terms of of how effective these tools can be.

Pete (38:09)

Yeah, yeah, we’re beyond

help me craft my email to help me

Mike (38:12)

Right.

Pete (38:13)

just close close the books for the month.

That’s yeah, for sure.

Mike (38:15)

Yeah, yeah, exactly.

And and which which takes me to what I would

be talking about of to to people that are making important decisions that need to lean into this. And that is to focus on this new new animal in the zoo called the sandbox, right? So these agents, as they get more powerful, you’re gonna wanna give them more freedom. Let them do more things on their own. But to do that, you also have to restrain them from

doing things that they shouldn’t do, right? So the answer is a sandbox. And it’s a it’s an IT concept like a server, like a Docker image, all lots of

Concepts that are familiar, but it’s just a place where an agent can live and work and do its job, right? And it’s the famously the the the Mac Minnie craze around OpenClaw

was basically just creating a bunch of sandboxes where a physical sandbox where things could run.

Pete (38:56)

A physical sandbox. Yeah.

Mike (38:59)

So so if you’re an IT executive, right now you should be thinking about how do I ramp up my sandbox strategy because I’m gonna want those agents to be super

Pete (39:05)

Mm, that’s a good point.

Mike (39:08)

powerful. I need to know, you know,

Pete (39:09)

Yeah. Yeah.

Mike (39:10)

who’s my vendor for that and and

And

the other interesting thing that you’re gonna want to figure out is cloud managed agents, right? So so this the idea of having it do dangerous things in a box is

one thing. dangerous things I mean like deleting files and things like that. It’s you know, right,

Pete (39:25)

Yeah. giving it permission to do what it wants. Yeah.

Mike (39:28)

exactly right. Giving it instead of having a human say, Yes, you may change to that directory every time, just put it in a box

and let it do it and and get the result out, right? So that’s one piece, but then

How much of the software lives up there? That that’s where we get into cloud managed agents and and that’s where agents become a service of their own rather than being a software that you run that uses the APIs, which are a service. So I think those are the two IT things to think about on the on the business side for enterprise customers. I’m gonna lean into what I’ve said drumbeat forever, which is the evergreen.

the evergreen story in AI is capture your business processes. And when I say capture that, sometimes people get lost in what that means. I mean write it down.

Just write down what you do. You know, at the end of the month, I fill in the blank, you know, write

it down on a notepad for all and scan it in, right? Make

a videotape of yourself talking about what you do when you have a difficult customer situation. Capture every one of those stories that you possibly can and do that across your enterprise because th that’s what’s going to turn into the gold that makes your AI better than your competitors’ AI, right? And and it doesn’t have to be

this massive organizational level hierarchy of approval processes to define the standards.

You’ll never get there if you do that. Instead, get your best people to c to write down what they

do and the AI will be able to repeat that and help them and that’s how the spire f that’s how the fire spreads.

Pete (40:47)

Awesome. Well, gentlemen, thank you very much. Great to see you. Awesome episode.

And we’ll see you guys on the flip side.

Shanti Greene (40:55)

Sounds good?

Mike (40:57)

Bye all.

Shanti Greene (40:58)

See ya.