Anthropic 的工程师说：我们内部已经不怎么写 prompt 了，写的是循环。 她在��

Transcript

0:02

Hey everyone, great to see you all today

0:05

and hope you're all having a wonderful day

0:06

so far here at AI DevCon

0:09

I certainly am

0:11

I hope that your context windows are not too full

0:13

and you have a bit of space for

0:15

a little bit more information about context

0:18

by way of introduction

0:19

my name is Lamis

0:20

I am a member of technical staff at

0:22

Anthropic

0:23

I work on our applied AI team

0:26

and this is a team which sits between research

0:28

product, and go-to-market

0:30

So we do a mixture of working on internal projects

0:32

as well as directly with customers

0:34

who are building agents at the frontier

0:37

Me specifically, I work with startups and founders

0:40

many of whom I'm sure are in this room today

0:43

and

0:44

I think I have the best seat in

0:45

the house because

0:47

these are the users that are constantly pushing

0:48

us

0:49

right up against the boundary

0:51

of what is possible with models and products today

0:54

and as such, we just get to

0:56

like, ride the exponential together

1:00

One thing that constantly comes up

1:02

as I'm sure you're all aware

1:04

is what it really takes

1:05

to take the raw

1:07

model intelligence that we have today and translate that into durable

1:11

scalable, useful products

1:14

And one of the main levers

1:15

that we have in order to do this is context engineering

1:18

which will be the focus

1:19

of my talk today

1:23

So on this journey

1:25

I want to quickly

1:26

do a recap of where context engineering has

1:29

gone in the past year

1:30

It's a space that's completely blown up

1:33

And through that

1:34

we'll kind of distill the primitives that have

1:36

proven to be really useful

1:37

some stuff that has been a little bit less useful

1:40

Secondly

1:41

we'll talk about what the state of the

1:42

art is for memory management today

1:46

And thirdly, and in particular with that

1:49

we'll talk about not just what nice theoretical principles are

1:52

but what it takes to actually build these

1:53

systems in production

1:55

And then finally

1:56

we'll talk about where this will go on

1:58

the path to continual learning

1:59

and in particular, touching on a paradigm called dreaming

2:06

So, we've said this before

2:08

and models, we release new models all the time

2:12

they are more and more intelligent

2:14

but when it comes to actually deploying these

2:16

models in your agents

2:17

in your environments, in your organization

2:21

the intelligence alone is not going to compound

2:23

because they

2:24

need this context that helps them perform the

2:25

specific tasks that you need them to

2:28

In particular

2:29

a lot of this context is often kind

2:31

of orthogonal to the model intelligence

2:33

right?

2:33

Like, the newest model, we've just released one

2:37

isn't gonna

2:38

isn't gonna out of the box

2:39

know exactly what it takes to succeed in

2:41

your organization

2:42

and with the tasks that you want them to

2:44

And so

2:45

it's a really great investment

2:47

to work on the context engineering part

2:49

because this, over time has the effect of multiplying the intelligence

2:53

even as models get smarter

2:56

So I'm sure you'll all be familiar with

2:57

these problems. It's like agents not knowing

2:59

their way around a code base or

3:00

knowing enough about your own user preferences

3:05

and then additionally, like

3:06

you don't have the effect where

3:08

agents are better at the task the next

3:10

time they perform it

3:11

so they, they might not learn from their mistakes

3:13

and as such, you don't have this

3:16

continual learning effect. So

3:18

just to recap

3:22

where we've got so far on this journey of context engineering

3:27

at Anthropic, we like to say

3:30

do the simple thing that works

3:32

And this is a timeline that's only really

3:33

spans the past year

3:35

and where we started

3:36

was with these ClaudeMD files

3:38

that we launched with Claude Code

3:41

And what we learned from this was that

3:42

it was kind of unreasonably effective

3:44

like this markdown file that just gives the

3:45

agent a couple of instructions about maybe

3:48

your way around the code base

3:49

the organization, your own user preferences

3:53

That injected into the beginning of a model

3:55

context at the beginning of a session

3:57

was so good at steering it towards the

3:59

things that mattered and helping it navigate

4:01

and align its actions towards your preferences

4:05

However

4:05

we also learned a couple of things about

4:07

what doesn't work here

4:08

So when we're injecting this at the beginning

4:11

at the beginning of the session into context

4:13

we obviously start to run into problems

4:15

where you get problems with context

4:18

like what happens when this

4:19

file with very important preferences gets very

4:21

very long?

4:23

how do I manage that over time?

4:26

And so we went back to the drawing

4:27

board and thought about like ways that we

4:28

could improve this

4:30

Separately though

4:31

what was true was that having just a

4:32

very simple markdown file

4:34

which is human readable

4:35

which your agent can write to

4:37

which you can write to

4:38

is really, really effective

4:41

So a second avenue that we investigated was

4:44

these memory tools

4:46

And this is interesting because it leans into this idea of

4:51

okay

4:51

what happens if we

4:53

let agents autonomously manage their own memory systems?

4:56

So we let them decide when they read

4:58

when they write, and when they update memories

5:02

And this is all happening in band

5:04

which means that it's within the context of

5:06

a session. So during a session

5:08

a agent is thinking about

5:10

like, what might be interesting to pull from memory

5:12

what might be interesting to write to memory

5:15

So

5:16

autonomy

5:17

proves to work really well in this case

5:21

And over time

5:21

we've kind of developed that into systems where

5:23

we're even less opinionated about what these tools

5:25

need to look like

5:26

and I'll touch on that in a second

5:30

So the next

5:31

The next stop on this journey was skills

5:33

which I'm sure you've heard a bunch about

5:35

today

5:36

And what this solves is this problem of the

5:39

like, ever-growing context. So we have this really clever idea of

5:43

progressive disclosure

5:46

The way I like to think about it is

5:49

actually, first, on skills, what skills are good at

5:51

it's processes where you have

5:53

like

5:53

a procedural workflow. So

5:55

something where you have an opinion about

5:57

how the process works end to end

5:59

that you want the agent to follow

6:01

and what's very clever is that the agent

6:02

only looks at this front matter

6:04

a couple of sentences at the top of

6:06

the file before loading the skill

6:08

but you can still load as much detail

6:10

as you want into the main body of

6:11

the file. So you're able to

6:14

at the same time

6:14

have very deep levels of detail

6:17

whilst not overloading the model's context

6:20

And the way I like to think about

6:21

it is as if I've had a bookshelf in my room

6:24

and every time someone talks to me

6:27

I can kind of scan and look at

6:28

my

6:29

list of books and see if any of

6:30

the titles might be relevant to the conversation

6:32

and kind of pick that off the shelf

6:34

and read it when I need to

6:35

So, for example

6:36

if someone walked up to me and started

6:38

speaking to me in French

6:39

and I noticed that I have a French dictionary

6:41

I could pull that out

6:42

and it would give me context kind of

6:43

loaded during the conversation that would help me

6:47

without me having

6:48

paid attention in like seven years of French

6:49

classes at school and having that all loaded

6:51

into my context already

6:53

So, this was a really

6:55

really great innovation

6:57

But one bottleneck potentially with it

6:59

is that

7:00

it's still kind of driven by humans and

7:02

agents together. So you're still

7:04

even if you're using your agents to write

7:06

the skills with you

7:07

you're still being quite opinionated about like what

7:09

things need skills

7:12

So that takes me to the final step on this path

7:14

which is

7:15

what we perceive to be state of art

7:16

for memory systems today

7:19

And what we have done

7:21

and yeah, what we think is best practice

7:24

is modeling

7:25

these memory systems just as file systems

7:27

So this kind of aggregates a couple of

7:29

the learnings from this path

7:31

So file systems are great

7:32

you can just fill them up with Markdown

7:35

Agents are actually just very good at using

7:37

normal file system tools like Bash and grep

7:39

so just let them search over the file system

7:42

rather than being opinionated about the specific tools

7:45

that they should use to read and write

7:46

to memory

7:48

And then, yeah

7:49

that search kind of mirrors this idea of

7:51

progressive disclosure

7:52

You can index these memory systems really well

7:55

so that agents can intelligently search over them

7:57

And that's where we

7:58

have kind of got to so far

8:01

So just to recap the key learnings from that format

8:05

Markdown is great

8:06

For reading memories, you know, allow the memories to grow large

8:10

but

8:11

Give agents tools to quickly index and search for what's relevant

8:16

and finally, like

8:17

give agents autonomy when they're writing to memories

8:21

And if you were to go out and build this system

8:23

it would work really well

8:24

You would have the

8:25

feeling of continual learning because your agents would

8:28

get better at the individual

8:30

whatever individual tasks you're doing

8:34

However, as with everything

8:37

this very neat idea

8:38

runs into some problems when you try and

8:40

scale it to production

8:42

So we have an idea

8:44

we have a concept for theoretically what works

8:47

and when we then think about scaling these

8:49

to production

8:51

in environments where we have many agents collaborating

8:54

at the same time

8:55

where they run over very long periods of time

8:59

and where potentially these

9:01

like, code bases get really complicated

9:03

all manner of problems start to arise

9:05

and we've seen these in production time and

9:07

time again

9:08

So one, a couple of things just to like

9:10

yeah, spark your imagination

9:12

Think about multiple agents trying to write to

9:14

a memory file at the same time

9:16

How do you manage that?

9:17

Think about

9:18

one agent running into a problem and deciding

9:21

to like write to the organizational wide context

9:24

which every other agent is currently reading from

9:26

Like if something was incorrect there

9:29

that would scale to all of your agents

9:30

and be pretty disastrous

9:34

and think about when you have humans and agents collaborating on

9:37

on memory context together

9:38

like how do you

9:39

keep track of what's going on?

9:42

The final problem is that memories can go stale

9:44

of course. Something that was relevant in the past

9:47

might not be relevant today

9:48

or maybe it was written incorrectly

9:51

or even maliciously injected by someone trying to

9:55

prompt inject your agents to write

9:56

bad things to memory. So

9:58

you have to have a lot of guardrails

9:59

in place to make sure that these

10:02

nice autonomous memory systems actually work in production

10:08

And so

10:09

I'm going to talk through a couple of

10:11

key principles

10:12

that we

10:13

use when designing memory systems in production

10:15

to make sure that we do get

10:17

to use all of those nice effects that we've talked

10:19

that we've spoken about

10:21

So the very first thing is versioning

10:24

So when you're designing any kind of memory system

10:26

you need to be able to store versions

10:29

to keep track of what's going on

10:31

to allow you to roll back should you need to

10:33

if a new update isn't particularly good

10:37

Additionally, you probably want to think about

10:39

like, what context was this update based on?

10:42

So which agent session

10:43

which transcript

10:45

resulted in me wanting to make this update?

10:49

And additionally, like, you might want to track

10:51

like, who did it, which agent, which human, et cetera, et cetera

10:54

So this is really important

10:56

The second thing is concurrency

10:58

So we've talked about

10:59

okay

10:59

what happens when I deploy thousands of agents

11:01

all working off the same memory system

11:05

And the solution that we've adopted here

11:08

is to have this hashing system

11:10

So

11:11

when an agent decides that it wants to

11:13

write an update to a memory

11:15

it takes a hash

11:16

it then drafts its edit

11:18

and then before it writes the update

11:21

it takes another hash

11:22

If those two things do not match

11:24

then the agent cannot write it because

11:26

it means that some update was made in

11:28

the meantime

11:29

And in order to handle that

11:31

the agent repools the memory

11:33

drafts its new update

11:34

and then tries to commit this again

11:36

So these are the kinds of

11:37

just engineering practices that allow you to scale multiple agent architectures

11:42

scale memory to these kinds of architectures

11:48

Another couple key principles

11:50

so permissioning is really important

11:54

When you have large memory bases

11:56

you probably have a mixture of top-level

11:59

organizational wide knowledge. It might be like your key

12:03

like what your organization is trying to achieve

12:05

or key principles about the code base

12:07

which you've really carefully curated

12:10

all the way down to the level of

12:12

like a scratch pad for an agent

12:13

where it writes down its working memory

12:16

and it's very like individualized

12:18

And all the way in between

12:19

you could have things for specific organizations or

12:21

cross sections

12:23

And so it's really important that you have

12:25

guardrails when it comes to permissioning

12:27

these memories

12:28

So like I said

12:30

you wouldn't want

12:31

one agent to just decide that

12:32

it should update the organization wide context

12:34

Probably you might want that as read only

12:37

However, for its own scratch pad

12:38

you would want it to have write access

12:42

and yeah, yeah, that's, that's permissioning

12:47

final thing, which is kind of peripheral

12:49

but still really important when you design memory

12:52

systems

12:53

Is portability

12:55

So like I've mentioned before

12:59

your

12:59

curation of your memory systems

13:01

is going to be so important like

13:04

throughout the future. This is really

13:06

really important organization, user

13:08

or like work task specific context

13:12

And so it's likely that if you're putting

13:13

a lot of effort into curating this

13:15

you want it to be accessible across potentially multiple product surfaces

13:20

and access, accessible by multiple systems

13:24

So designing it in a way with a

13:25

clean API in which it's portable and you

13:27

can access it is really important

13:31

And so when you put all these things together

13:34

we have the kind of

13:36

learnings we have from allowing agents to creatively manage their memory

13:39

and then these production level

13:41

guardrails that allow them to like reasonably

13:44

use all of those principles in practice

13:47

And when you do this

13:48

you get very effective results

13:50

So

13:51

just sharing here a couple of learnings from

13:53

what we've seen deploying

13:54

these large-scale memory systems in production

13:58

And for example, we see you get better accuracy

14:02

so you have this effect

14:04

where the second time the agent does the task

14:06

it actually does it better

14:08

with higher results

14:10

because it's noted all of those memories

14:12

about what went wrong

14:14

Secondly, that then has second-order effects on the speed and latency

14:18

sorry

14:18

speed and cost of your

14:20

agents running because they're then spending fewer tokens

14:23

they can

14:24

more easily one-shot these tasks

14:26

because they actually know what they're doing

14:29

And you'll see that across all sorts of different processes

14:31

agents are just able to

14:33

do the task better and faster

14:34

Finally

14:35

having this

14:37

process where your agents are starting to autonomously

14:40

write their own memories

14:41

frees up

14:43

Capacity and context for you

14:44

as

14:45

product developers potentially to focus on product wins

14:47

while you know that the agents are doing

14:49

this kind of self-learning

14:51

continual learning loop in the background

14:53

And yeah, once that infrastructure is set up really well

14:56

this, this works very symbiotically

15:02

as ever, we do then reach a new bottleneck

15:06

and this specifically is about in-band memory

15:10

So in-band memory, as I mentioned before

15:13

is when agents are writing to and reading

15:16

from memory within a specific session

15:19

So if you think about Claude code

15:20

for example, when you like spin up a new session

15:24

it's

15:24

it's

15:24

largely like focusing on that specific context when it's reading to

15:28

reading and writing from memory

15:31

and this just architecturally or philosophically has limitations in the general

15:36

like agent fleets, continual learning objectives

15:39

There's two, two main reasons why

15:41

First of all

15:42

is that you have this inherent split of

15:44

focus and resources

15:46

So you're asking an agent to complete a task

15:49

but at the same time

15:50

you're also asking it to invest in memory curation

15:53

which would help it perform better in a

15:55

future run

15:56

So when you put these things together

15:58

it's just a very difficult optimization problem

16:00

because how much capacity should an agent put into

16:02

like, helping future versions of itself

16:04

versus doing the task that you actually asked

16:06

it to do

16:09

And also, there's, like, other effects, like latency

16:10

for example

16:12

The second thing is that

16:14

the agents just have an inherent visibility limitation

16:18

So

16:19

they

16:19

only have a context of what's going on

16:21

in their session

16:23

As such

16:23

they just won't see patterns that happen across

16:25

sessions. So when you get frustrated that your

16:27

agent keeps making the same mistake over sessions

16:29

it just doesn't understand how frustrating that is

16:32

because it

16:33

has a new context window in each of those

16:35

Secondly, when you're running multiple fleets of agents in different environments

16:39

You

16:40

these single agents just don't have the context

16:41

of what other failures other agents are running

16:44

into

16:45

So for these two reasons

16:46

we introduce this concept of some out-of-band memory curation

16:50

and this

16:51

helps to make these problems go away

16:54

And just to introduce an analogy for

16:56

why this in theory should work

16:59

I'd like you to think about a school

17:01

for example

17:02

where you have lots of students that submit

17:03

a lot of work

17:04

and you also have

17:07

teachers that market and a head teacher that

17:08

reviews everything

17:11

This is a system that we have in

17:12

the real world for good reason

17:13

and it's because when you have

17:15

certain individuals that have dedicated capacity for helping people learn

17:19

that's really effective

17:21

And when you also have people that have

17:22

visibility over the whole fleet of agents or learners

17:25

and they're able to spot patterns and then

17:27

kind of steer context or

17:29

let's say in this case

17:29

the curriculum, that's also really effective

17:32

So as always

17:34

we kind of look to the real

17:36

world to think about how to build these systems

17:40

sorry, I also didn't touch on a final limitation

17:42

which is that memories go stale

17:43

so you need something

17:44

some pass that checks that

17:47

everything that's written there is still correct

17:51

And so we introduced this concept of dreaming

17:55

which is like a second

17:56

a second order process over memory

17:59

So if we think about how that's been constructed

18:01

we have the, like, actual context, which agents reference and is

18:05

has useful information, the memory processes

18:08

which allow agents to kind of autonomously manage that context themselves

18:12

and then dreaming

18:13

which

18:14

is a process that runs

18:16

in batch and asynchronously

18:18

with its own allocated resources

18:20

to ensure that those memories themselves are effective

18:24

up-to-date, and helping the agents learn over time

18:31

So, what does dreaming look like?

18:35

Essentially, what we do Is we take an existing memory store

18:40

so this is a collection of memories

18:43

We then take

18:44

a bunch of sessions or transcripts from agent

18:47

interactions over a period of time

18:51

and we give these together to an agent

18:53

which reviews all of the transcripts

18:56

looks at the memory store

18:57

and starts to identify patterns

18:59

for where there could be

19:00

uplift in the memories

19:05

It then outputs a new memory store

19:08

where there are

19:08

proposed changes to the existing memory store

19:12

And what the agent is able to do

19:13

as I mentioned

19:14

is

19:15

spend tokens on solving this problem of making agents learn better

19:19

identify patterns for where agents are consistently failing

19:24

and then propose changes for what

19:26

might make a more effective memory store

19:28

such that next day

19:29

when you run these agents again

19:31

they'll actually feel smarter and they're running better

19:34

To go back to my analogy

19:36

just to paint some pictures of like

19:37

what this could look like in practice

19:39

Let's imagine

19:41

that the

19:42

head teacher reviewing all these transcripts notices that

19:45

every geography student has incorrectly answered a question

19:49

They're just all writing like complete garbage to

19:51

this question

19:53

The teacher notices that actually by kind of

19:56

in this case, analyzing the memory store

19:58

that entire topic is missing from the curriculum

20:01

So what the teacher was able to do

20:02

is notice that pattern

20:04

look at the memory store

20:05

and suggest a new change to that curriculum

20:07

such that the next day when these agents run

20:09

they now have that information that they needed

20:13

To give another example

20:14

the teacher might notice that in a certain maths exam

20:18

all of the answers are wrong in the

20:19

same way

20:20

All of the students are outputting radians when

20:23

it's meant to be degrees

20:24

I don't know if anyone else in like

20:25

GCSE maths had that problem too

20:28

but what they're able to do is

20:29

give an instruction saying like

20:31

this is how you should configure your calculators

20:33

And in the case of agents

20:36

that's like

20:36

noticing in the transcripts that there's something wrong

20:39

with a tool configuration

20:40

So you might notice that

20:42

something in the tool calls keeps failing

20:44

And what's important here

20:45

is that when we look at those transcripts

20:48

we're not just looking at kind of the passes of like

20:51

responses between agent and

20:54

the system or the user

20:55

We're also

20:56

really scrutinizing like those tool calls and all

20:58

of the other metadata

20:59

that is really central to the agent's performance

21:03

Finally

21:04

you could also notice something that's like fleet-wide

21:06

or organization-wide

21:08

So for example

21:08

like everybody's using too many M dashes and

21:10

you don't like that

21:11

So you want to add some organizational-wide organ

21:16

announcement or

21:17

context change that says not to do that

21:20

And so

21:20

I hope that paints the picture of like

21:22

why this could be really

21:22

really effective

21:24

And now I'll just talk about

21:27

how you

21:29

would go about designing such a system

21:31

in production

21:33

So you have some concept of your memory store

21:36

which is a concept

21:36

which is a collection of memories

21:39

Memories themselves might just be markdown files

21:42

organized in this directory

21:44

You then take a number of the transcripts

21:46

and like I mentioned

21:48

that is a mixture of

21:50

like, the back and forth passes between the agents

21:53

as well as metadata on tools

21:55

other skills they used

21:56

et cetera

21:58

And the way that we've designed it

22:00

we have the orchestrator deploy a fleet of

22:02

sub-agents that go and analyze all those transcripts

22:07

And one point worth making here

22:08

is that when you design these systems

22:11

You have the ability to steer how these agents

22:14

which both write memories and coordinate dreaming

22:17

go about the problem

22:18

And by steering, I mean that you're able to tell them

22:21

like, in your specific case

22:23

these are the kinds of things I think

22:24

are important and relevant

22:26

These are the kinds of things that are

22:27

not important and relevant

22:28

So you do have the

22:29

ability there to start to

22:30

curate that memory and dreaming process

22:33

to your organization specifically

22:37

And

22:38

the orchestrator then reviews all of the responses from the sub-agents

22:42

and it then decides

22:45

like

22:45

where there are prevalent enough patterns that it

22:48

thinks this warrants a change in the memories

22:51

From there

22:52

it proposes individual changes to the memory store

22:56

And in our case

22:57

the way that we design this in production

22:59

is

23:00

the agent will additionally give you examples

23:02

of transcripts where it's noticed this pattern has happened

23:05

and also some stats on like

23:07

how prevalent this issue is and why this warrants actually updating

23:10

the memory store

23:14

And so all of this works really neatly

23:16

You get this output

23:17

and you as the individual

23:19

can decide where you want to accept changes to the memory

23:24

where you want to reject them

23:25

et cetera

23:26

And this works really effectively

23:31

So together

23:33

we have these two processes that run in

23:35

parallel

23:37

The first is memory

23:39

and these agents are using some of their

23:45

Like in-band contacts and in-band

23:47

resources to write to memory where they think

23:49

it's important

23:50

And this is neat because it means that

23:52

in the actual next run

23:54

the next session, that agent will be better

23:56

So there's a shorter time to kind of

23:58

seeing that change

23:59

But

24:00

inherently these agents have competing resources

24:02

when they think about what to dedicate to memory

24:05

what to dedicate to completing the task

24:08

and additionally a lack of visibility

24:10

So on the other side

24:11

we have dreaming, which is this out-of-band process

24:16

and this allows, again, broader visibility and dedicated capacity

24:21

i.e. token spend

24:22

which is specifically directed towards helping agents

24:25

learn better

24:27

And you might think

24:28

okay, that sounds really expensive

24:30

Why would I want to chuck extra resources

24:32

at this?

24:33

But if we kind of go back to

24:35

the improvements we saw when you build effective memory stores

24:38

actually you can see a bunch of costs

24:40

go down because

24:43

agents are able to one-shot things more effectively

24:45

They have more information that they need

24:47

in order to perform a task well

24:53

So, to summarize

24:58

at the very least

24:59

do the simple thing that works

25:00

Context management

25:02

makes such a huge difference to your agent

25:04

performance

25:06

Implementing things like a CloudMD file

25:09

like skills, which I'm sure you've heard about a bunch

25:12

and allowing agents to autonomously manage these systems

25:14

themselves goes a really long way

25:17

Once you think about scaling those things

25:19

into architectures with many agents

25:23

agents that run over a very long time

25:27

situations where

25:28

you will continue to work and develop

25:30

on a workspace or a code base over

25:32

a long period of time

25:33

Or very complex domains

25:36

you should start thinking about

25:37

adding some features or some guardrails that allow

25:39

those agents to manage their memory

25:41

in a way that is safe

25:43

verifiable, auditable

25:47

I should also say here that whilst this kind of

25:51

these kinds of practices are really effective when

25:53

it comes to coding tasks

25:54

for example

25:55

this by no means is just specific to

25:57

coding. Like I use memory all the time

25:59

when I'm

26:00

producing presentations that has context on like

26:02

how I like to write things

26:03

how I like my slides

26:04

et cetera, et cetera

26:05

and that develops over time

26:06

So this is really not coding specific

26:09

The final thing

26:11

is

26:12

if you really want to kind of like

26:13

close the loop here

26:15

think about adding an additional out of band process

26:18

like dreaming, as we call it

26:20

to consolidate your memory and

26:23

cut things that are no longer relevant

26:25

add things that agents are missing

26:26

and clean up and organize memory systems

26:31

So to close

26:33

I want to say that this

26:35

journey that we've been on with Context Engineering

26:37

a lot of this stuff

26:38

has only happened in this past year

26:41

This is very much an open area of research and development

26:45

and one which we see huge value

26:46

in the future

26:47

So we're so excited to see the kinds

26:49

of things and contributions that you'll all make

26:51

to this space

26:52

So I encourage you to keep thinking

26:54

keep learning, and keep dreaming

26:56

Thank you

27:03

Wonderful

27:05

Thank you, Lamis. That was great

27:07

do you have any questions in the room?

27:09

Oh, golly gosh

27:11

I'll dive straight here first

27:14

Thank you

27:15

Thanks for the presentation

27:17

do you have any

27:19

memory store implementation that you would like to

27:22

suggest?

27:23

Papers? Any memory storage implementation that you would like us

27:28

Sorry, did you say papers?

27:30

No

27:32

Oh, okay, okay. That we'd like to suggest

27:35

To solve what problem specifically?

27:39

Well

27:39

one thing is put things in files that

27:41

I have on my laptop

27:43

but I think I'm looking into something more enterprise-y

27:47

so what kind of solution do you suggest over there?

27:52

Okay, yeah, so, I mean

27:53

maybe I was kind of coy in the

27:54

talk because we're not allowed to make product call to actions

27:57

but given that you asked

28:02

we have

28:03

a lot of this

28:04

references the architecture that we used in our memory

28:09

our memory infrastructure for our managed agent solutions

28:12

and so when I talk about these things about productionizing

28:16

memory, so everything like versioning

28:19

hashing, et cetera

28:20

that's all available within our memory and dreaming

28:22

API through cloud managed agents

28:25

So if you

28:26

did want an out-of-box solution for this kind of thing

28:28

that is where I would point you to

28:32

hi

28:34

the

28:35

earlier on you talked about guardrails and permissions

28:39

and I think

28:40

I'm sure most of us have probably read

28:41

the code code leak and the memory stuff

28:43

The dreaming stuff was definitely some of the

28:45

most interesting in it

28:47

But how do you scale that

28:49

at enterprise if you've got hundreds of users

28:52

with different permission sets?

28:53

How do you make sure dreaming follows those

28:56

same guardrails?

28:58

If it's

28:59

happening out of band and the context is different compared to

29:02

say

29:02

the context the agent might have when a

29:04

user is using it

29:08

So just to check that I understand

29:11

we have, like, some permissioning about what agents can access

29:14

like, in terms of memory

29:15

and then a separate

29:16

like

29:17

yeah, so, I mean, I think these

29:18

these things compose quite well

29:21

because when you set up a dreaming procedure

29:24

You decide exactly which session transcripts to attach

29:27

and so you could

29:28

build a process which mirrors whatever permissioning you

29:30

have on the agents

29:33

so yeah, I mean, if

29:35

that's to say that it's not the case

29:37

that when you kind of trigger a dreaming job

29:38

it just takes like everything in a certain time period

29:41

you can configure it that way

29:43

but

29:43

you could also just search over whichever transcripts

29:45

have the same permission set as like

29:48

whatever your memory store is

29:49

and then make sure that that matches

29:51

if that makes sense

29:55

Hi, thank you. This was really

29:56

really interesting

29:57

I

29:58

I found it

29:58

fascinating earlier when you were mentioning about like versioning

30:01

concurrency, durability, all of these, all these things

30:03

At what point are we like reinventing

30:06

databases from like first principles again?

30:12

yeah, this is, this is an interesting point

30:15

I still think, like one of the things that I suppose

30:19

and I, I, this is a good reminder for me

30:21

but something I didn't say

30:24

What we're trying to do here is like thread the needle

30:26

like, sorry

30:27

find the right boundary to draw between

30:30

kind of letting these agents autonomously act

30:32

and then also like which things should just

30:34

be kind of programmatic

30:36

things that are baked into the harness

30:38

And so I think what you allude to is like

30:42

first of all

30:42

we were just kind of like letting these

30:43

agents

30:44

write in markdown files and just like commit

30:45

whatever they wanted. And now we're kind of seeing

30:48

having seen which primitives work really well

30:49

we're thinking about like kind of codifying that

30:51

in the harness

30:52

And so when we think about like the

30:53

hashing or the versioning

30:54

yeah

30:54

we are kind of going back to the

30:56

software engineering principles that we've seen work well in the past

30:59

but in a way that kind of autonomous

31:01

agents can act and like

31:04

can interact with those really effectively

31:06

So I think to some extent

31:08

like we sort of are merging back into those practices

31:11

but that's because

31:12

we have enough signal now to know that

31:14

those things should just be done in a very deterministic way

31:17

and there's no need to reinvent the wheel

31:18

I hope that answers your question

31:21

Perfect. We're absolutely out of time

31:23

Thank you once again. Big round of applause

31:25

for Lamix

31:26

Thank you