WEBVTT 00:00:00.000 --> 00:00:02.040 Why Durable? 00:00:02.040 --> 00:00:08.580 So Pi, the coding agent, has historically been you are a single human, 00:00:08.580 --> 00:00:12.640 you sit in front of a computer or you have a remote machine and you SSH into it. 00:00:12.640 --> 00:00:18.600 And you start your Pi session, you do a thing and you stop the Pi session and then you usually don't care about the session anymore. 00:00:18.600 --> 00:00:22.620 So it's more like a one-to-one relationship, the sessions are ephemeral, 00:00:22.620 --> 00:00:28.680 more or less, and everything fits snugly into memory as well because Pi 00:00:28.680 --> 00:00:35.580 historically loads the entire session into memory just to do a bunch of things. 00:00:36.020 --> 00:00:42.540 It's not entirely necessary, it's just a historical artifact stemming from the fact that it's a one-on-one ephemeral session, 00:00:42.540 --> 00:00:44.240 usually not super long, right? 00:00:44.240 --> 00:00:49.480 However, does that fit all agentic applications? 00:00:49.480 --> 00:00:51.400 I guess the answer is no. 00:00:52.620 --> 00:00:58.900 If you want to have a Slack bot that stays with you for years and years and can answer multiple 00:00:58.900 --> 00:01:03.480 people in parallel or have threads and answer there, 00:01:03.480 --> 00:01:04.920 Pi is not your best option. 00:01:04.920 --> 00:01:09.140 It can be done and we have like Piedolf, our Discord bot, 00:01:09.140 --> 00:01:13.140 who's been asleep for a couple of months because it was not so useful. 00:01:14.240 --> 00:01:18.740 But it's kind of not what Pi the coding agent was built for. 00:01:19.780 --> 00:01:24.580 So earlier this summer we decided we needed a different kind of substrate, 00:01:24.580 --> 00:01:27.360 a different kind of framework or library or whatever you want to call it, 00:01:28.200 --> 00:01:33.820 that allows you to model this kind of agent or agentic application as well. 00:01:33.820 --> 00:01:39.480 Long-running agents, multiplayer, there's multiple users or other agents can interact 00:01:39.480 --> 00:01:40.840 with each other. 00:01:42.740 --> 00:01:46.460 And not everything should be in RAM, in memory in total. 00:01:47.140 --> 00:01:48.880 And it should be able to run anywhere. 00:01:49.200 --> 00:01:53.500 Like Pi is obviously mostly bound to a computer-like system. 00:01:53.500 --> 00:01:57.320 We also want to run on Vercel, on Cloudflare, 00:01:57.320 --> 00:02:00.520 on E2B, on whatever you can come up with, 00:02:01.060 --> 00:02:07.700 where the computing substrate is potentially different than a process in an operating system. 00:02:07.840 --> 00:02:14.100 For example, Cloudflare, Durable Objects, it's an entirely different execution model than what you have in a process in Node 00:02:14.100 --> 00:02:15.940 on your Mac or Windows machine. 00:02:16.860 --> 00:02:19.080 So yeah, and that was how Durable was born. 00:02:19.080 --> 00:02:25.380 And I think it's also kind of useful to just point out what maybe some of the 00:02:25.380 --> 00:02:28.900 classical limitations are of Pi's existing session model. 00:02:29.460 --> 00:02:34.640 And why it works quite well for the Pi coding agent, but like why it shows some limits. 00:02:34.880 --> 00:02:39.520 And actually I think like to a large degree, those limitations are also shared with a lot of other agents. 00:02:41.280 --> 00:02:44.920 So for a start, it writes JSON-L files. 00:02:45.460 --> 00:02:46.480 And they're definitely useful. 00:02:46.980 --> 00:02:48.440 There's nothing fundamentally wrong with them. 00:02:48.440 --> 00:02:53.260 But they're append, right? 00:02:53.340 --> 00:02:57.260 So it's like it's not very easy to rewrite those files. 00:02:58.280 --> 00:03:04.620 And as you're putting larger and larger workloads into them, they're accumulating a lot of intermediary 00:03:04.620 --> 00:03:06.480 results potentially. 00:03:08.660 --> 00:03:12.760 That, yeah, are potentially wasteful. 00:03:12.760 --> 00:03:16.840 But the second thing is also that if you are going to have sub-agents, 00:03:16.840 --> 00:03:21.440 you would actually have to split that across dependent sessions. 00:03:22.280 --> 00:03:25.460 So Pi's tree works really, 00:03:25.460 --> 00:03:29.720 really well, particularly if you are a single writer, single user. 00:03:29.900 --> 00:03:32.700 You can navigate to different parts of the tree, rewind the session, do some stuff. 00:03:32.700 --> 00:03:36.940 But modeling sub-agents into a single session 00:03:36.940 --> 00:03:41.340 would interleave lines, 00:03:41.340 --> 00:03:44.160 would require some locking that doesn't exist today. 00:03:44.160 --> 00:03:50.240 So like the entire tree system as it exists is very cool, 00:03:50.240 --> 00:03:53.200 but has some limitations, let's say. 00:03:53.200 --> 00:03:58.880 So a lot of the work towards durable started, 00:03:58.880 --> 00:04:04.300 I think, in some way or another by figuring out how to make this stuff work with a database. 00:04:04.300 --> 00:04:08.660 And really, I think there are multiple things. 00:04:08.800 --> 00:04:11.740 Like durable means you should be able to stop and start it again. 00:04:12.000 --> 00:04:14.300 So it's like it suspends and picks up work. 00:04:15.240 --> 00:04:20.580 And that if you were to have sub-agents in it or you were to have multiple parallel things in it, 00:04:20.600 --> 00:04:25.100 it would extend to all the branches which are active at any point in time. 00:04:25.100 --> 00:04:31.440 And then the second part is that while 00:04:31.440 --> 00:04:34.200 Pi is quite durable even today, 00:04:35.140 --> 00:04:41.640 it is very easy to build stuff with it where you 00:04:41.640 --> 00:04:42.860 cannot pick up the work. 00:04:43.060 --> 00:04:49.100 And even the core agent look that we have today, if you were to continue, it could not continue on its own. 00:04:49.100 --> 00:04:55.840 So like if you were to shut down and you bring it up, you would have to write continue or something into it to actually continue to work. 00:04:56.660 --> 00:05:03.140 And there are a lot of features that you might want to put into Pi that require extra state to be 00:05:03.140 --> 00:05:04.320 persisted along the way. 00:05:04.580 --> 00:05:10.240 So the classic example would be if you interrupt a tool execute 00:05:10.240 --> 00:05:14.860 in order to ask the user for approval, 00:05:14.860 --> 00:05:21.160 which again, not really the most useful case right now because I think tool 00:05:21.160 --> 00:05:22.200 approvals are largely gone. 00:05:22.580 --> 00:05:25.400 But you could not really suspend the agent at this point in time and say like, 00:05:25.460 --> 00:05:30.140 hey, we need to figure out if we can go ahead or not and then continue from there. 00:05:30.280 --> 00:05:35.800 So it's like the system is quite limited in being able to suspend at points where we didn't intend 00:05:35.800 --> 00:05:37.800 it unsuspending before. 00:05:38.480 --> 00:05:44.720 Yeah, I think we should probably just go through what durable means because we throw it around. 00:05:44.860 --> 00:05:49.540 But not a lot of people are actually very familiar with the concept. 00:05:49.840 --> 00:05:53.680 So in old Pi, not old Pi, in Pi coding agent, 00:05:53.680 --> 00:05:58.480 you start a session, you let the agent do something, your machine dies. 00:05:58.900 --> 00:06:03.440 You spin up Pi again, the same session, and that Armin said you have to write continue. 00:06:04.000 --> 00:06:11.260 If there was an extension that did something in the background, that extension needs to be written in such a way that it actually spins up the background work again. 00:06:11.820 --> 00:06:14.780 And Pi coding agent didn't help the extension with that at all. 00:06:14.860 --> 00:06:17.520 Like everybody had to reinvent how to do that. 00:06:17.920 --> 00:06:23.840 If you had a tool that was being executed when the process crashed, 00:06:23.840 --> 00:06:26.540 that tool is just aborted and not run again. 00:06:26.540 --> 00:06:32.260 And the agent has to figure out if it should run it again and if so how and if the tool execution was a 00:06:32.260 --> 00:06:35.560 destructive thing or not and so on and so forth. 00:06:35.560 --> 00:06:41.060 And then Pi doesn't have real concurrency, 00:06:41.060 --> 00:06:45.680 meaning you always have a single agent loop. 00:06:46.640 --> 00:06:51.460 And if the process dies, we do not recover within that agent loop. 00:06:51.460 --> 00:06:54.700 So an agent loop is basically get the user input, 00:06:54.700 --> 00:06:59.260 send it to the LLM, get the response back, check if there's tool calls, 00:06:59.260 --> 00:07:02.740 execute the tools, and then do that in a loop until the LLM says I'm done. 00:07:02.740 --> 00:07:06.740 In Pi coding agent, that just goes away. 00:07:06.880 --> 00:07:11.020 We do not keep track of where we are in that loop basically, right? 00:07:11.020 --> 00:07:17.680 So with Pi durable, we introduced a bunch of things that make this automatic basically. 00:07:18.100 --> 00:07:21.380 So it kind of forces you into a durable way of thinking. 00:07:21.380 --> 00:07:27.180 So first of all, that loop I just described, that is actually decomposed into what we call tasks. 00:07:27.460 --> 00:07:31.200 And the task is basically just saying I intend to do a thing. 00:07:31.520 --> 00:07:35.040 I store this in a storage so I can remember it later. 00:07:35.340 --> 00:07:41.040 I do the thing and then I store the result of the thing I did, right? 00:07:41.040 --> 00:07:46.140 So this gives me kind of like a sandwich and I can resume at any point here. 00:07:46.260 --> 00:07:50.320 I can resume in between intent and execution of what we call the effect. 00:07:50.320 --> 00:07:57.260 And in between the effect and the final result and after the result, right? 00:07:57.820 --> 00:08:02.700 And this is very nice because now you just write the thing that says I'm about to do the thing. 00:08:02.820 --> 00:08:03.720 I am doing the thing. 00:08:03.860 --> 00:08:04.560 I did the thing. 00:08:05.280 --> 00:08:10.200 And if you resume, we know exactly where in this sandwich you were and can resume from there. 00:08:10.500 --> 00:08:12.100 And this extends to everything. 00:08:12.260 --> 00:08:13.560 This extends to the request. 00:08:13.660 --> 00:08:15.560 It's being made to the LLM provider. 00:08:15.560 --> 00:08:21.220 This extends to tools which can now mark themselves as replay safe. 00:08:21.720 --> 00:08:24.340 For example, a read tool is replay safe. 00:08:24.580 --> 00:08:27.600 You can just call it again and you get whatever. 00:08:28.360 --> 00:08:34.700 A sub-agent tool is also replay safe because that same agent runs the same 00:08:34.700 --> 00:08:36.060 tasks as the main agent. 00:08:36.060 --> 00:08:40.400 So all of this durability of the execution graph applies to it as well. 00:08:40.620 --> 00:08:44.300 So that agent can just be spun up again and be told, 00:08:44.300 --> 00:08:48.200 not even be told, it automatically does continuous from where it left off. 00:08:49.660 --> 00:08:53.360 And so this is probably the biggest selling point of Pi Durable. 00:08:53.900 --> 00:08:59.320 We took some inspiration from structured concurrency. 00:08:59.320 --> 00:09:04.940 It doesn't cleanly apply to everything in an agent, which to me was a very interesting observation. 00:09:05.100 --> 00:09:10.740 So structured concurrency is basically something owns another thing that does something. 00:09:11.240 --> 00:09:16.300 And this other thing is not allowed to survive past the lifetime of this. 00:09:16.880 --> 00:09:19.640 This works for like 90% of tasks inside Pi Durable. 00:09:19.640 --> 00:09:25.680 But there's a handful of things that need this child execution to survive 00:09:25.680 --> 00:09:27.360 the lifetime of the parent. 00:09:27.360 --> 00:09:32.400 So it's not 100% structured concurrency, but it's very close to that. 00:09:32.500 --> 00:09:34.580 And it gives you an escape hatch if you don't need it. 00:09:36.060 --> 00:09:37.200 Yeah, and that's durability. 00:09:37.420 --> 00:09:38.400 I hope any of that makes sense. 00:09:38.420 --> 00:09:40.400 And I think there's another thing we're mentioning. 00:09:40.660 --> 00:09:45.240 So if you go to the earendil works GitHub repository, you will find that there's a project called Absurd, 00:09:46.260 --> 00:09:49.580 which is a durable workflow engine sitting on top of Postgres. 00:09:49.580 --> 00:09:55.500 And we built that originally to support a Pi-driven agent, 00:09:55.500 --> 00:09:56.960 which is called Levos, 00:09:57.880 --> 00:10:04.800 to basically continue and suspend across basically a cloud deployment. 00:10:05.260 --> 00:10:09.360 So when we built this originally, the assumption was, well, sandbox are quite expensive. 00:10:09.740 --> 00:10:14.920 So we want to build the agent in a way where you just distribute it to any worker that wants to work on it 00:10:14.920 --> 00:10:16.440 and continue work doing there. 00:10:17.380 --> 00:10:22.400 And unsurprisingly, the problem that we ran into quite early on, 00:10:22.440 --> 00:10:27.120 I think a lot of people ran into it, is that there's a lot of states that's not just the agent state. 00:10:28.240 --> 00:10:33.480 So initially just transmitting the session transcript around was obviously not enough. 00:10:33.960 --> 00:10:35.900 Agents love to write to the file system and so forth. 00:10:35.900 --> 00:10:43.240 And so you very quickly discover that durable workflow engines are great, 00:10:43.600 --> 00:10:49.960 but unless the agent harness actually really supports them well, that's not enough. 00:10:50.080 --> 00:10:52.000 So like the agent harness in itself needs to be durable. 00:10:53.360 --> 00:10:56.520 Now, that doesn't mean that everything becomes durable, right? 00:10:56.580 --> 00:11:00.380 So it's like if the file system now holds on to really important data, 00:11:00.900 --> 00:11:02.720 you also need to find a way to either snapshot that, 00:11:02.720 --> 00:11:05.060 or you build the agent around in a way that it can go away. 00:11:06.900 --> 00:11:10.140 And so I think for someone who is looking at this and saying like, 00:11:10.220 --> 00:11:12.640 hey, we have a Dora workflow engine, all of our problems are gone, 00:11:12.720 --> 00:11:15.400 it's like you might still want to use it because it might still be useful. 00:11:16.260 --> 00:11:21.060 But just having that did not actually make your agent durable. 00:11:22.880 --> 00:11:29.040 And in fact, there's still a chance that the agent will not come out particularly durable out of the box 00:11:29.040 --> 00:11:35.300 if you let the agent just write files to file systems that you never bring back or stuff like this. 00:11:35.900 --> 00:11:40.100 So there's things inside the harness that can all be durable, 00:11:40.480 --> 00:11:43.360 but the harness also operates on an external environment. 00:11:43.980 --> 00:11:46.500 And in the effect sandwich, which I described earlier, 00:11:46.760 --> 00:11:48.840 the intent is the durable part inside the harness, 00:11:48.960 --> 00:11:53.720 the effect, which is write to a file, execute bash, call some service, deploy something, blah, blah. 00:11:54.120 --> 00:11:56.820 That is the external system and you do not have control over that. 00:11:56.820 --> 00:11:58.160 Well, usually. 00:11:58.780 --> 00:12:02.780 So the only way to make this part durable as well is to have some kind of, 00:12:02.820 --> 00:12:04.700 and I can't pronounce the fucking word, 00:12:04.940 --> 00:12:11.780 basically, if I say deploy with the ID, blah, blah, blah, 00:12:11.780 --> 00:12:15.800 and I call this again and the deploy for that ID is already running, 00:12:16.020 --> 00:12:17.140 it shouldn't trigger again. 00:12:17.300 --> 00:12:18.280 That's what I'm saying. 00:12:18.280 --> 00:12:20.340 So if your external system works like this, great. 00:12:20.460 --> 00:12:24.380 File systems don't work like this unless you have like a snapshotting file system. 00:12:25.120 --> 00:12:31.580 But yeah, the thing with durability is you can have a workflow engine like temporal or absurd 00:12:31.580 --> 00:12:34.400 or I don't know what other stuff exists out there. 00:12:35.280 --> 00:12:37.280 But then you run into issues of atomicity. 00:12:38.260 --> 00:12:42.960 Like for durability to work, everything that needs to be durable in one step 00:12:42.960 --> 00:12:45.280 needs to be atomically persisted. 00:12:45.280 --> 00:12:49.800 And if you have this outside of the agent, 00:12:50.000 --> 00:12:54.640 then anything inside of the agent that needs atomic persistence for durability 00:12:54.640 --> 00:12:57.220 needs to be shoehorned into that somehow. 00:12:57.400 --> 00:12:59.200 And that really doesn't quite work. 00:12:59.280 --> 00:13:03.920 So that's why we decided for Pi Durable to actually make it a durable workflow engine itself, 00:13:04.360 --> 00:13:09.220 which solves a lot of problems because it also allows you to build your own tasks 00:13:09.220 --> 00:13:14.540 that run inside this scheduler and durable scheduler for your own application. 00:13:14.540 --> 00:13:17.880 And it also allows you, and that's the second part of durability, 00:13:18.620 --> 00:13:22.640 persistence of state, not only of the harness, but also of your application. 00:13:23.160 --> 00:13:29.460 Now, obviously, you do not want to stuff every application state you have into the harness storage. 00:13:30.240 --> 00:13:36.660 But what you can do is you persist large blobs, for example, into your own storage. 00:13:37.020 --> 00:13:40.920 And then inside the harness storage, you persist a pointer to that persisted blob. 00:13:40.920 --> 00:13:47.560 And with that, it's very easy to have durable execution and durable storage of state. 00:13:47.560 --> 00:13:53.720 I think another thing worth remembering here is 00:13:53.720 --> 00:13:57.320 so absurd, like temporal, 00:13:57.940 --> 00:14:02.380 and I think that gazillions of other durable workflow execution engines out there, 00:14:02.980 --> 00:14:09.060 they were actually largely built on one premise that is now quickly being invalidated. 00:14:09.180 --> 00:14:14.660 And that premise was that you can actually cheat on durability quite a bit 00:14:14.660 --> 00:14:21.760 by assuming that the code that is durable is sort of maintained by an expert programmer. 00:14:22.580 --> 00:14:28.400 And I'm saying this because the way almost all durable workflow engines work today 00:14:28.400 --> 00:14:35.240 is they either say like your code has to be fully deterministic on the workflow orchestration part, 00:14:35.300 --> 00:14:39.660 so that if I suspend for like a week and I come back in a week later, 00:14:39.660 --> 00:14:43.460 the code is the same, so I just run through all the checkpoints, 00:14:43.640 --> 00:14:46.280 I'm pulling the data as it was written into durable storage, 00:14:46.860 --> 00:14:48.120 and I'm continuing from there. 00:14:49.620 --> 00:14:52.720 Or it has to be written in a way that is maybe non-deterministic, 00:14:53.220 --> 00:14:58.000 but all the branches that you were taking on the durable information is still compatible 00:14:58.000 --> 00:15:00.140 or has like explicit code in it for upgrades. 00:15:00.140 --> 00:15:06.640 And the problem with this with LLMs is that we are 00:15:06.640 --> 00:15:08.180 now living in a world 00:15:08.180 --> 00:15:09.880 where the agents are writing their own code. 00:15:10.000 --> 00:15:11.800 And it's not that the agents cannot be expert programmers, 00:15:12.440 --> 00:15:18.080 but the agents, it is very hard for them to basically maintain 00:15:18.080 --> 00:15:22.660 like a perfect understanding of all the possible tasks 00:15:22.660 --> 00:15:24.600 that will ever come by again in this durable state. 00:15:25.240 --> 00:15:27.900 And so this, I would say like this is an unsolvable problem, 00:15:27.900 --> 00:15:31.580 but the trade-off now is less to make it friendly for a programmer 00:15:31.580 --> 00:15:35.040 by abstracting it away, but to make it a lot more explicit. 00:15:35.900 --> 00:15:38.020 Because if it's explicit, then the agent can see it. 00:15:38.080 --> 00:15:40.040 It's like, oh, I wrote this durable information away 00:15:40.040 --> 00:15:41.120 and we need to load it again. 00:15:41.540 --> 00:15:45.380 So the trade-off actually now is sort of almost entirely inverse 00:15:45.380 --> 00:15:46.440 to what we did with Absurd, 00:15:46.960 --> 00:15:49.340 where we tried to make it as friendly for a human programmer to do. 00:15:49.840 --> 00:15:52.240 And now it's like S in your face. 00:15:53.400 --> 00:15:54.380 So did the LLM. 00:15:55.100 --> 00:15:56.500 There's going to be a little bit more code, 00:15:56.500 --> 00:16:00.760 but the LLM is going to be explicitly going through checkpoints. 00:16:00.920 --> 00:16:05.920 So like as an example, state suspension in Absurd was like, 00:16:06.300 --> 00:16:08.020 yeah, like every time you went to a step, 00:16:08.440 --> 00:16:09.520 we would increment the counter. 00:16:09.840 --> 00:16:12.280 And so like if you, if a loop for this four times, 00:16:12.400 --> 00:16:15.680 then there's a hidden like first time, second time, 00:16:15.760 --> 00:16:16.560 third time and so forth. 00:16:17.560 --> 00:16:20.160 In Pi Durable, what is just a memo function, 00:16:20.820 --> 00:16:24.240 where you give it a name and that's the name. 00:16:24.240 --> 00:16:26.160 And if you want to version it, you version it. 00:16:27.540 --> 00:16:30.820 So there's a lot less hidden magic in durable 00:16:30.820 --> 00:16:32.700 than there is in temporal or Absurd. 00:16:33.160 --> 00:16:33.620 Yeah, definitely. 00:16:33.800 --> 00:16:39.000 Like the workflow scheduler is like 1,200 lines of code 00:16:39.000 --> 00:16:40.680 that you can actually read and understand. 00:16:41.240 --> 00:16:44.680 And it's very easy to reason about the execution graph. 00:16:44.820 --> 00:16:47.320 And that is actually the difference to temporal or Absurd. 00:16:47.900 --> 00:16:49.420 In temporal or Absurd, 00:16:49.420 --> 00:16:52.840 you kind of pretend you're writing a box standard JavaScript program. 00:16:53.160 --> 00:16:56.380 And it just will magically pick up from where you left off, right? 00:16:56.520 --> 00:16:58.800 That's more true for temporal than Absurd, I think. 00:17:00.020 --> 00:17:03.060 And that is, as you said, for humans, a great model 00:17:03.060 --> 00:17:04.580 because you kind of sort of understand. 00:17:04.580 --> 00:17:07.160 But then if you actually use it in anger, 00:17:07.260 --> 00:17:08.120 you figure out, 00:17:08.760 --> 00:17:12.260 they actually have to do a lot of shit to make this work. 00:17:12.420 --> 00:17:14.740 Like something as simple as set timeout, 00:17:14.840 --> 00:17:18.320 they have to kind of hook into 00:17:18.320 --> 00:17:19.900 and ensure that it works with this. 00:17:20.460 --> 00:17:22.400 And the biggest problem, actually, 00:17:22.460 --> 00:17:23.720 with these kinds of workflow engines 00:17:23.720 --> 00:17:25.940 is that you have an execution graph. 00:17:26.120 --> 00:17:27.660 Like function A calls function B, 00:17:27.760 --> 00:17:28.620 calls function C. 00:17:29.200 --> 00:17:30.980 And then somewhere in there, 00:17:31.320 --> 00:17:34.240 we stopped the process 00:17:34.240 --> 00:17:35.340 and now we restart it. 00:17:35.360 --> 00:17:38.200 And now we have to reestablish the call stack, right? 00:17:38.340 --> 00:17:40.660 Function A, function B, function C. 00:17:40.920 --> 00:17:41.820 So how do you do this? 00:17:42.220 --> 00:17:43.720 You have to replay the fucking shit. 00:17:44.460 --> 00:17:45.700 And the replay works like this. 00:17:45.740 --> 00:17:47.220 You go into function A again 00:17:47.220 --> 00:17:49.280 and function A calls something that says, 00:17:49.420 --> 00:17:50.240 do the thing. 00:17:50.720 --> 00:17:52.040 And instead of actually doing the thing, 00:17:52.120 --> 00:17:53.360 it gives you the last result. 00:17:53.940 --> 00:17:54.560 The thing did. 00:17:54.840 --> 00:17:56.080 The last time you ran it. 00:17:56.340 --> 00:17:57.120 Then you take that 00:17:57.120 --> 00:17:58.680 and you call function B with that. 00:17:58.900 --> 00:17:59.940 And that does the same thing. 00:18:00.000 --> 00:18:00.600 And then you take that 00:18:00.600 --> 00:18:01.400 and call function C. 00:18:01.700 --> 00:18:03.420 And it's a terrible model, actually, 00:18:03.580 --> 00:18:04.440 if you think about it. 00:18:04.480 --> 00:18:06.480 Because it requires a journal of everything 00:18:06.480 --> 00:18:09.040 and it reruns the entire code. 00:18:09.380 --> 00:18:10.740 And if anything, as Armin said, 00:18:10.860 --> 00:18:12.820 changes in any of these functions, 00:18:13.240 --> 00:18:14.080 you are fucked. 00:18:14.320 --> 00:18:16.440 One of the biggest criticisms of Temporal 00:18:16.440 --> 00:18:18.080 is that versioning is just terrible. 00:18:18.640 --> 00:18:21.260 So what does Pi Durable actually do? 00:18:21.780 --> 00:18:23.020 Pi Durable doesn't give you 00:18:23.020 --> 00:18:25.880 a classic call stack of JavaScript functions. 00:18:25.880 --> 00:18:29.860 Pi Durable gives you a tree of tasks. 00:18:30.380 --> 00:18:31.900 And each task is self-contained. 00:18:32.380 --> 00:18:34.840 It can call other tasks and wait for them. 00:18:35.960 --> 00:18:38.380 And that's trivial to reestablish 00:18:38.380 --> 00:18:42.720 compared to a normal JavaScript function call graph. 00:18:43.140 --> 00:18:44.420 And that's the big difference here. 00:18:44.800 --> 00:18:46.460 The units of work are very small, 00:18:46.740 --> 00:18:47.320 are very easy. 00:18:47.460 --> 00:18:52.700 The intent, effect, storage kind of model. 00:18:52.700 --> 00:18:55.740 And if you want to do more things in parallel, 00:18:55.920 --> 00:18:56.300 you can. 00:18:56.380 --> 00:18:56.860 It's very easy. 00:18:56.960 --> 00:18:57.920 You just create a new task 00:18:57.920 --> 00:18:59.440 and other tasks can wait for that. 00:18:59.560 --> 00:19:00.040 And that's it. 00:19:00.200 --> 00:19:02.780 And it's very super easy to kind of rewind, 00:19:03.120 --> 00:19:06.220 to kind of reestablish that virtual call graph, 00:19:06.360 --> 00:19:07.180 if you want to call that. 00:19:08.080 --> 00:19:09.340 So since we already talked about the effects. 00:19:09.800 --> 00:19:10.140 Oh, yeah. 00:19:10.140 --> 00:19:14.200 We should maybe talk about the other elephant in the room, 00:19:14.320 --> 00:19:15.800 which is effect.js. 00:19:16.800 --> 00:19:18.260 And before we talk about effect.js, 00:19:18.560 --> 00:19:22.820 I also made a little side note here 00:19:22.820 --> 00:19:26.460 on the sort of constantly implied elephant in the room, 00:19:26.540 --> 00:19:27.680 which is JavaScript in the first place. 00:19:27.680 --> 00:19:31.340 And I think this, again, is TypeScript. 00:19:32.240 --> 00:19:34.340 And it's TypeScript because we really like 00:19:34.340 --> 00:19:36.360 the idea of agents extending themselves. 00:19:36.500 --> 00:19:37.220 So it's like right now, 00:19:37.280 --> 00:19:38.680 JavaScript seems like a good environment. 00:19:40.180 --> 00:19:41.240 But I also want to say, 00:19:41.360 --> 00:19:43.380 in many ways, 00:19:43.440 --> 00:19:44.540 what we are trying to do here 00:19:44.540 --> 00:19:45.780 is we're trying to build 00:19:45.780 --> 00:19:47.820 almost like 00:19:47.820 --> 00:19:50.740 a way of building an agent 00:19:50.740 --> 00:19:52.940 and that stores itself durably. 00:19:53.560 --> 00:19:54.560 Durable storage means 00:19:54.560 --> 00:19:57.740 commitment to some stable system 00:19:57.740 --> 00:20:00.200 that you should be able to describe 00:20:00.200 --> 00:20:01.340 in a way where you could rebuild 00:20:01.340 --> 00:20:02.380 in a different language, right? 00:20:02.440 --> 00:20:02.820 So it's like, 00:20:03.560 --> 00:20:04.480 keep in mind, 00:20:04.560 --> 00:20:06.260 it's like this is for sure JavaScript right now 00:20:06.260 --> 00:20:07.620 and might even be JavaScript 00:20:07.620 --> 00:20:08.760 for a very, very long time. 00:20:09.300 --> 00:20:12.140 But the goal is that the system in itself is so, 00:20:13.040 --> 00:20:13.200 like, 00:20:13.700 --> 00:20:15.600 you should be able to write an agent in this 00:20:15.600 --> 00:20:18.120 that maintains a two-year-old Slack channel 00:20:18.120 --> 00:20:20.020 and then you eventually want to rewrite it in TypeScript, 00:20:20.140 --> 00:20:21.600 but you can pick up the data as it was. 00:20:22.100 --> 00:20:22.180 Right? 00:20:22.220 --> 00:20:23.500 So the idea here is like 00:20:23.500 --> 00:20:24.780 that the underlying system 00:20:24.780 --> 00:20:29.700 is a trustworthy foundation to build on. 00:20:30.740 --> 00:20:32.920 So why did we not use Effect.js? 00:20:34.360 --> 00:20:36.360 Because it didn't solve any of our issues. 00:20:36.820 --> 00:20:37.740 So out-of-the-box, 00:20:37.820 --> 00:20:40.400 Effect.js is great for in-memory, 00:20:40.500 --> 00:20:40.980 in-process, 00:20:41.180 --> 00:20:42.040 structured concurrency, 00:20:42.160 --> 00:20:42.580 for example, 00:20:42.720 --> 00:20:44.780 along with a metric ton of other stuff 00:20:44.780 --> 00:20:46.080 like structured errors, 00:20:46.420 --> 00:20:47.300 dependency injection, 00:20:48.120 --> 00:20:49.340 cancellation and logging 00:20:49.340 --> 00:20:50.640 or telemetry traces 00:20:50.640 --> 00:20:51.200 and all of that. 00:20:51.280 --> 00:20:52.120 It's fantastic. 00:20:52.120 --> 00:20:53.380 Truly fantastic. 00:20:53.800 --> 00:20:54.580 Although I have to say 00:20:54.580 --> 00:20:56.900 that one of the maintainers recently said 00:20:56.900 --> 00:20:58.800 he found out 00:20:58.800 --> 00:21:01.540 he can no longer write Effect.js himself. 00:21:03.460 --> 00:21:04.480 But it's not a problem 00:21:04.480 --> 00:21:05.700 because agents understand it. 00:21:08.260 --> 00:21:09.040 So again, 00:21:09.240 --> 00:21:10.280 we love Effect.js 00:21:10.280 --> 00:21:11.620 and I think it's the right solution 00:21:11.620 --> 00:21:12.680 for many, many problems. 00:21:12.820 --> 00:21:13.360 But for us, 00:21:13.440 --> 00:21:14.400 it didn't solve a problem 00:21:14.400 --> 00:21:16.340 because Effect.js are out-of-the-box. 00:21:16.820 --> 00:21:18.460 That's exactly zero 00:21:18.460 --> 00:21:19.820 for durable execution. 00:21:20.960 --> 00:21:22.500 So the function A, 00:21:22.560 --> 00:21:22.960 function B, 00:21:23.080 --> 00:21:24.040 function C thing, 00:21:24.760 --> 00:21:27.540 that works great in memory, 00:21:28.220 --> 00:21:30.440 but Effect.js doesn't give you any primitives 00:21:30.440 --> 00:21:32.300 to reestablish that call graph 00:21:32.300 --> 00:21:34.660 if you have to resume the process. 00:21:35.580 --> 00:21:37.400 So then the question is, 00:21:38.040 --> 00:21:38.300 okay, 00:21:38.340 --> 00:21:39.780 if it doesn't help us with durability, 00:21:39.780 --> 00:21:42.140 can we get anything else out of it 00:21:42.140 --> 00:21:43.260 and how hard would it be 00:21:43.260 --> 00:21:44.720 to add durability to this? 00:21:45.080 --> 00:21:45.880 And it turns out 00:21:45.880 --> 00:21:47.260 adding durability to this 00:21:47.260 --> 00:21:49.420 basically ends up 00:21:49.420 --> 00:21:50.680 looking like what we have. 00:21:51.460 --> 00:21:53.780 A task-based workflow engine. 00:21:54.260 --> 00:21:55.720 There's Effect.js workflow, 00:21:55.940 --> 00:21:56.660 the package, 00:21:56.840 --> 00:21:57.220 or effect 00:21:57.220 --> 00:21:58.880 slash workflow, 00:21:59.480 --> 00:22:00.360 and it's basically that. 00:22:00.440 --> 00:22:01.520 They call tasks activities 00:22:01.520 --> 00:22:03.180 and it's basically 00:22:03.180 --> 00:22:04.800 the same thing, 00:22:05.220 --> 00:22:07.660 except you also inherit all of Effect.js 00:22:07.660 --> 00:22:09.280 and we didn't want to make that. 00:22:09.460 --> 00:22:10.080 It's not a bet 00:22:10.080 --> 00:22:12.040 because a lot of people build on Effect.js, 00:22:12.340 --> 00:22:14.560 but we want it to be self-contained 00:22:14.560 --> 00:22:15.900 and in terms of portability, 00:22:16.760 --> 00:22:18.960 it's actually very easy to port 00:22:18.960 --> 00:22:20.760 Pi Durable to something else. 00:22:20.940 --> 00:22:21.680 Ask me how I know. 00:22:24.420 --> 00:22:25.780 It's less trivial 00:22:25.780 --> 00:22:27.120 to port Effect.js 00:22:27.120 --> 00:22:31.020 and its generator-based semantics to something else. 00:22:31.780 --> 00:22:32.700 So ultimately, 00:22:32.860 --> 00:22:34.540 that's why we decided against Effect.js. 00:22:35.240 --> 00:22:35.960 And I also think 00:22:35.960 --> 00:22:37.280 a useful, 00:22:37.720 --> 00:22:38.500 so I actually think 00:22:38.500 --> 00:22:39.400 Effect is quite useful. 00:22:39.720 --> 00:22:41.800 I don't actually think it's particularly useful for Durable 00:22:41.800 --> 00:22:42.520 for a bunch of reasons, 00:22:42.660 --> 00:22:43.580 but I did 00:22:43.580 --> 00:22:44.900 in an attempt 00:22:44.900 --> 00:22:47.060 to use what we built and also to, 00:22:49.100 --> 00:22:50.780 like a lot of the side projects that I'm doing 00:22:50.780 --> 00:22:53.100 just to figure out how to build a software factory 00:22:53.100 --> 00:22:54.060 and things like that 00:22:54.060 --> 00:22:56.440 are in fact based on Effect. 00:22:57.440 --> 00:22:58.080 And the reason 00:22:58.080 --> 00:22:58.780 they're based on Effect 00:22:58.780 --> 00:22:59.400 is because I think 00:22:59.400 --> 00:23:00.920 Effect does actually 00:23:00.920 --> 00:23:04.400 solve some of the guardrail problems today because like, 00:23:04.540 --> 00:23:04.940 for instance, 00:23:05.520 --> 00:23:06.560 it strongly encourages 00:23:06.560 --> 00:23:07.640 you not to throw exceptions. 00:23:08.000 --> 00:23:09.340 It strongly encourages 00:23:09.340 --> 00:23:10.380 you to compose services. 00:23:11.100 --> 00:23:14.760 So it is a pretty useful crutch almost 00:23:14.760 --> 00:23:16.820 to get a little bit 00:23:16.820 --> 00:23:18.720 of structure 00:23:18.720 --> 00:23:19.920 into a TypeScript project 00:23:19.920 --> 00:23:20.880 where it would otherwise 00:23:20.880 --> 00:23:21.800 not have a lot of it 00:23:21.800 --> 00:23:23.400 when you leave an agent to it. 00:23:23.400 --> 00:23:25.540 But simultaneously, 00:23:26.800 --> 00:23:29.280 if I then start the same project 00:23:29.280 --> 00:23:30.840 without Effect, 00:23:30.980 --> 00:23:31.580 like I did this 00:23:31.580 --> 00:23:32.220 a bunch of times now, 00:23:32.300 --> 00:23:33.060 like build this with Effect, 00:23:33.180 --> 00:23:33.920 build it without Effect, 00:23:34.500 --> 00:23:35.880 it's actually not entirely clear 00:23:35.880 --> 00:23:36.640 that the Effect solution 00:23:36.640 --> 00:23:38.220 is also definitely 00:23:38.220 --> 00:23:39.120 better for the agent 00:23:39.120 --> 00:23:41.060 because for the very least, 00:23:41.120 --> 00:23:42.040 you actually do end up 00:23:42.040 --> 00:23:43.560 ironically with more code usually. 00:23:44.400 --> 00:23:45.220 And more code 00:23:45.220 --> 00:23:47.020 is still a little bit of a challenge 00:23:47.020 --> 00:23:48.840 for this. 00:23:49.540 --> 00:23:50.960 But I think the Effect people 00:23:50.960 --> 00:23:51.820 are on the right track 00:23:51.820 --> 00:23:54.340 in general about expressing software like this. 00:23:55.140 --> 00:23:55.920 And I already 00:23:55.920 --> 00:23:57.740 proposed to 00:23:57.740 --> 00:23:59.500 at least two of them, 00:23:59.740 --> 00:24:01.480 why not try to build 00:24:01.480 --> 00:24:04.400 it into TypeScript, 00:24:04.680 --> 00:24:06.320 have a version of TypeScript 00:24:06.320 --> 00:24:08.280 that expresses effects. 00:24:08.920 --> 00:24:09.720 Because then 00:24:09.720 --> 00:24:11.180 a lot of these hacks 00:24:11.180 --> 00:24:12.200 that are currently there 00:24:12.200 --> 00:24:14.800 with Effect Gen 00:24:14.800 --> 00:24:18.120 and the service declarations might actually become 00:24:18.120 --> 00:24:20.360 core permittives of a language. 00:24:20.360 --> 00:24:22.280 I definitely want someone 00:24:22.280 --> 00:24:23.240 to do some experiments 00:24:23.240 --> 00:24:24.580 there and see where it goes. 00:24:25.960 --> 00:24:26.580 But I think 00:24:26.580 --> 00:24:29.100 the portability part about it is a pretty 00:24:29.100 --> 00:24:30.940 big part. 00:24:31.120 --> 00:24:32.080 The Effect 00:24:32.080 --> 00:24:35.480 adds a lot of effect to the 00:24:35.480 --> 00:24:37.620 effect ecosystem. 00:24:38.020 --> 00:24:38.720 It's viral, 00:24:38.860 --> 00:24:39.200 basically. 00:24:39.380 --> 00:24:40.380 It is very viral. 00:24:40.780 --> 00:24:43.400 Even using a database usually goes through other effect libraries. 00:24:43.660 --> 00:24:43.920 And so 00:24:43.920 --> 00:24:47.300 you inherit all of that, 00:24:47.300 --> 00:24:49.340 which you might 00:24:49.340 --> 00:24:50.200 individually want to do 00:24:50.200 --> 00:24:51.100 if you build an application. 00:24:51.520 --> 00:24:53.240 I think it's a much tougher sell 00:24:53.240 --> 00:24:54.400 if you want to build 00:24:54.400 --> 00:24:55.680 a foundation framework 00:24:55.680 --> 00:24:57.780 that other people then use. 00:24:58.000 --> 00:24:59.400 Because you would basically 00:24:59.400 --> 00:25:01.980 force that choice onto the user. 00:25:02.940 --> 00:25:03.960 Which is pretty tough. 00:25:04.920 --> 00:25:05.400 Again, 00:25:05.640 --> 00:25:08.080 we are absolutely not dunking on Effect. 00:25:08.440 --> 00:25:09.340 We actually really, 00:25:09.440 --> 00:25:10.060 really like it. 00:25:10.180 --> 00:25:11.480 And if you're building applications, 00:25:11.640 --> 00:25:12.740 you should use it. 00:25:13.140 --> 00:25:14.620 If you build applications with TypeScript. 00:25:15.940 --> 00:25:16.100 Yeah. 00:25:17.860 --> 00:25:18.260 Yeah. 00:25:18.680 --> 00:25:19.920 So that's what we have today. 00:25:20.020 --> 00:25:21.240 I think one thing that 00:25:21.240 --> 00:25:23.980 we want people to do 00:25:23.980 --> 00:25:25.980 is to play with it and particularly 00:25:25.980 --> 00:25:27.840 see if the data model that we have makes sense. 00:25:29.640 --> 00:25:31.120 And I think the vacation 00:25:31.120 --> 00:25:32.420 example that we have 00:25:32.420 --> 00:25:33.660 in the repository 00:25:33.660 --> 00:25:34.660 is a good starting point. 00:25:35.680 --> 00:25:37.880 Because we would like to know 00:25:37.880 --> 00:25:41.100 that this solves your problems 00:25:41.100 --> 00:25:44.540 before we fully commit to it and then rewire the world 00:25:44.540 --> 00:25:45.760 on top of that. 00:25:46.780 --> 00:25:50.300 But that you mean rewiring Pi Coding Agent? 00:25:50.720 --> 00:25:51.060 For instance, 00:25:51.180 --> 00:25:52.320 the Pi Coding Agent on it 00:25:52.320 --> 00:25:54.620 or just to build more 00:25:54.620 --> 00:25:58.100 functionality that people 00:25:58.100 --> 00:26:00.080 might want to have because the core 00:26:00.080 --> 00:26:00.900 doesn't support it yet. 00:26:01.740 --> 00:26:02.440 I know that 00:26:02.440 --> 00:26:03.860 some of the people that were playing 00:26:03.860 --> 00:26:04.860 with it already said like, hey, 00:26:04.960 --> 00:26:08.660 maybe there needs to be more abstraction over the state system. 00:26:09.840 --> 00:26:11.620 Whatever we would want to build 00:26:11.620 --> 00:26:12.380 on top of that 00:26:12.380 --> 00:26:14.480 even just to assist people to building with it. 00:26:16.120 --> 00:26:16.600 Ideally, 00:26:16.700 --> 00:26:18.100 we'd know that the core 00:26:18.100 --> 00:26:21.280 data storage abstraction system works. 00:26:22.280 --> 00:26:22.760 So, 00:26:22.980 --> 00:26:24.500 I have high confidence that 00:26:24.500 --> 00:26:26.540 this kind of low-level API 00:26:26.540 --> 00:26:30.560 that we now have is a good primitive pending some plus and minuses. 00:26:31.100 --> 00:26:33.020 But I do agree that, for example, 00:26:33.560 --> 00:26:35.020 some sugar on top of it, 00:26:35.240 --> 00:26:36.980 specifically with regards to state management, 00:26:36.980 --> 00:26:38.280 is probably a good idea. 00:26:38.400 --> 00:26:40.000 Not just for humans, but for agents as well. 00:26:40.660 --> 00:26:41.720 It's just unclear 00:26:41.720 --> 00:26:45.400 to me what a nice abstraction there would look like. 00:26:45.940 --> 00:26:46.520 And also, 00:26:46.740 --> 00:26:48.960 we are not just relying on external builders using 00:26:48.960 --> 00:26:50.820 Pi Durable to figure out what works and what doesn't. 00:26:51.080 --> 00:26:52.400 We are actually betting on this 00:26:52.400 --> 00:26:55.400 for all of our Earendil products going forward. 00:26:56.220 --> 00:26:57.220 And in the future, 00:26:57.400 --> 00:26:57.980 that means in the 00:26:57.980 --> 00:26:58.640 next couple of weeks, 00:26:58.860 --> 00:27:00.360 we have a bunch of team members who 00:27:00.360 --> 00:27:01.400 will use it in 00:27:01.400 --> 00:27:04.080 anger, build some tools that we actually use in 00:27:04.080 --> 00:27:05.280 production that are 00:27:05.280 --> 00:27:12.180 very focused on a specific task, and we will document our journey and what we learned and obviously also provide the code 00:27:12.180 --> 00:27:13.540 that we created 00:27:13.540 --> 00:27:17.420 in a part of that process so other people can get inspired by that. 00:27:18.660 --> 00:27:23.260 Also, don't be surprised if there are some new faces coming up on these meditations in the future. 00:27:23.400 --> 00:27:23.700 That's all. 00:27:23.820 --> 00:27:24.240 Bye! 00:27:24.840 --> 00:27:25.220 Bye-bye.