This is f*cking insane. Anthropic engineer: "You're not supposed to prompt Opus 5.5 You're sup

Transcript

0:02

Hey everyone, great to see you all today

0:05

and hope you're all having a wonderful day

0:07

so far here at AI DevCon

0:09

I certainly am

0:11

I hope that your context windows are not

0:13

too full and you have a bit of

0:14

space for

0:15

a little bit more information about context

0:18

by way of introduction

0:19

my name is Lamis

0:20

I'm a member of technical staff at Anthropic

0:23

I work on our applied AI team

0:26

and this is a team which sits between research

0:28

product, and go-to-market

0:30

So we do a mixture of working on internal projects

0:32

as well as directly with customers

0:34

who are building agents at the frontier

0:38

Me specifically, I work with startups and founders

0:41

many of whom I'm sure are in this room today

0:43

and

0:44

I think I have the best seat in

0:45

the house because

0:47

these are the users that are constantly pushing

0:49

us

0:49

right up against the boundary

0:51

of what is possible with models and products today

0:54

and as such, we just get to

0:56

like, ride the exponential together

1:00

One thing that constantly comes up

1:02

as I'm sure you're all aware

1:04

is what it really takes

1:05

to take the raw

1:07

model intelligence that we have today and translate that into durable

1:11

scalable, useful products

1:14

And one of the main levers

1:15

that we have in order to do this is context engineering

1:18

which will be the focus of my talk today

1:23

So, on this journey

1:25

I want to quickly

1:27

do a recap of where context engineering has

1:29

gone in the past year

1:30

It's a space that's completely blown up

1:33

And through that

1:34

we'll kind of distill the primitives that have

1:36

proven to be really useful

1:37

some stuff that has been a little bit less useful

1:41

Secondly

1:41

we'll talk about what the state of the

1:43

art is for memory management today

1:46

And thirdly, and in particular with that

1:49

we'll talk about not just what nice theoretical principles are

1:52

but what it takes to actually build these

1:53

systems in production

1:56

And then finally

1:56

we'll talk about where this will go on

1:58

the path to continual learning

1:59

and in particular, touching on a paradigm called dreaming

2:06

So, we've said this before

2:08

and models, we release new models all the time

2:12

they are more and more intelligent

2:14

but when it comes to actually deploying these

2:16

models in your agents

2:17

in your environments, in your organization

2:21

the intelligence alone is not going to compound

2:23

because they

2:24

need this context that helps them perform the

2:26

specific tasks that you need them to

2:28

In particular

2:29

a lot of this context is often kind

2:31

of orthogonal to the model intelligence

2:33

right?

2:33

Like, the newest model, we just released one

2:38

isn't going to out of the box

2:39

know exactly what it takes to succeed in

2:41

your organization

2:42

and with the tasks that you want them to

2:44

And so

2:45

it's a really great investment

2:47

to work on the context engineering part

2:49

because this over time has the effect of multiplying the intelligence

2:53

even as models get smarter

2:56

So I'm sure you'll all be familiar with

2:57

these problems. It's like agents not knowing

2:59

their way around a code base or

3:01

knowing enough about your own user preferences

3:05

and then additionally, like

3:06

you don't have the effect where

3:08

agents are better at the task the next

3:10

time they perform it

3:11

so they, they might not learn from their mistakes

3:13

and as such, you don't have this

3:16

continual learning effect. So just to recap

3:22

where we've got so far on this

3:25

journey of context engineering

3:27

At Anthropic, we like to say

3:30

do the simple thing that works

3:32

And this is a timeline that only really

3:33

spans the past year

3:35

and where we started

3:36

was with these ClaudeMD files

3:38

that we launched with Claude Code

3:41

And what we learned from this was that

3:42

it was kind of unreasonably effective

3:44

Like this markdown file that just gives the

3:46

agent a couple of instructions about

3:48

maybe your way around the code base

3:49

the organization, your own user preferences

3:53

That injected into the beginning of the model

3:55

context at the beginning of a session

3:57

was so good at steering it towards the

3:59

things that mattered and helping it navigate

4:01

and align its actions towards your preferences

4:05

However

4:05

we also learned a couple of things about

4:07

what doesn't work here

4:08

So when we're injecting this at the beginning

4:11

at the beginning of the session into context

4:13

we obviously start to run into problems

4:16

where you get problems with context block

4:18

like what happens when this

4:19

file with very important preferences gets very

4:22

very long?

4:23

how do I manage that over time?

4:26

And so we went back to the drawing

4:27

board and thought about like ways that we

4:28

could improve this

4:30

Separately, though

4:31

what was true was that having just a

4:33

very simple markdown file

4:34

which is human readable

4:35

which your agent can write to

4:37

which you can write to

4:39

is really, really effective

4:41

So a second avenue that we investigated was

4:44

these memory tools

4:46

And this is interesting because it leans into this idea of

4:51

okay

4:51

what happens if we

4:53

let agents autonomously manage their own memory systems?

4:56

So we let them decide when they read

4:58

when they write, and when they update memories

5:02

And this is all happening in band

5:04

which means that it's within the context of

5:06

a session. So during a session

5:08

a agent is thinking about

5:10

like, what might be interesting to pull from memory

5:12

what might be interesting to write to memory

5:15

So

5:16

autonomy

5:17

proves to work really well in this case

5:21

And over time

5:22

we've kind of developed that into systems where

5:23

we're even less opinionated about what these tools

5:25

need to look like

5:26

and I'll touch on that in a second

5:30

So the next, the next stop on this journey was skills

5:33

which I'm sure you've heard a bunch about

5:35

today

5:36

And what this solves is this problem of the

5:39

like, ever-growing context. So we have this really clever idea of

5:43

progressive disclosure

5:46

The way I like to think about it is

5:49

actually, first, on skills, what skills are good at

5:51

it's processes where you have

5:53

like

5:53

a procedural workflow. So

5:55

something where you have an opinion about

5:57

how the process works end to end

5:59

that you want the agent to follow

6:01

and what's very clever is that the agent

6:03

only looks at this front matter

6:05

a couple of sentences

6:06

at the top of the file

6:07

before loading the skill

6:09

But

6:09

you can still load as much detail as

6:11

you want into the main body of the

6:12

file. So you're able to

6:14

at the same time

6:15

have very deep levels of detail

6:17

whilst not overloading the model's context

6:20

And the way I like to think about

6:21

it

6:22

is as if I'd had a bookshelf

6:24

in my room And every time someone talks to me

6:27

I can kind of scan and look at

6:29

my

6:29

list of books

6:30

and see if any of the titles might

6:31

be relevant to the conversation

6:33

and kind of pick that off the shelf

6:34

and read it when I need to

6:35

So for example

6:36

if someone walked up to me and started

6:38

speaking to me in French and I noticed

6:39

that I have a French dictionary

6:41

I could pull that out

6:42

and it would give me context kind of

6:43

loaded

6:44

during the conversation that would help me

6:47

without me having

6:48

paid attention in like seven years of French

6:50

classes at school and having that all loaded

6:51

into my context already

6:53

So this was a really

6:55

really great innovation

6:57

But one bottleneck potentially with it

6:59

is that

7:00

it's still kind of driven by humans and

7:02

agents together. So you're still

7:04

even if you're using your agents to write

7:06

the skills with you

7:07

you're still being quite opinionated about like what

7:09

things need skills

7:12

So that takes me to the final step on this path

7:14

which is

7:15

what we perceive to be state of art

7:17

for memory systems today

7:19

And what we have done

7:21

and yeah, what we think is best practice

7:24

is modeling

7:25

these memory systems just as file systems

7:28

So this kind of aggregates a couple of

7:29

the learnings from this path

7:31

So file systems are great

7:32

you can just fill them up with Markdown

7:35

Agents are actually just very good at using

7:37

normal file system tools like Bash and grep

7:40

so just let them search over the file system

7:42

rather than being opinionated about the specific tools

7:45

that they should use to read and write

7:46

to memory

7:48

And then, yeah

7:49

that search kind of

7:50

mirrors this idea of progressive disclosure

7:53

You can index these memory systems really well

7:55

so that agents can intelligently search over them

7:57

And that's where we

7:58

have kind of got to so far

8:01

So just to recap the key learnings from that

8:04

format, Markdown is great

8:07

For reading memories, you know, allow the memories to grow large

8:10

but

8:11

Give agents tools to quickly index and search for what's relevant

8:16

and finally, like

8:17

give agents autonomy when they're writing to memories

8:22

And if you were to go out and build this system

8:23

it would work really well

8:24

You would have the

8:25

feeling of continual learning because your agents would

8:28

get better at the individual

8:30

whatever individual tasks you're doing

8:34

However, as with everything

8:37

this very neat idea

8:38

runs into some problems when you try and

8:41

scale it to production

8:42

So we have an idea

8:44

we have a concept for theoretically what works

8:47

and when we then think about scaling these

8:50

to production

8:51

in environments where we have many agents collaborating

8:54

at the same time

8:55

where they run over very long periods of time

8:59

and where potentially these

9:01

like, code bases get really complicated

9:03

all manner of problems start to arise

9:05

and we've seen these in production time and

9:07

time again

9:08

So one, a couple of things just to like

9:10

yeah, spark your imagination

9:12

Think about multiple agents trying to write to

9:15

a memory file at the same time

9:16

How do you manage that?

9:18

Think about

9:19

one agent running into a problem and deciding

9:21

to like write to

9:22

the organizational

9:24

wide context which every other agent is currently

9:26

reading from. Like if something was incorrect there

9:29

that would scale to all of your agents

9:31

and be pretty disastrous

9:34

and think about when you have humans and agents collaborating on

9:37

on memory context together

9:38

like how do you

9:39

keep track of what's going on?

9:42

The final problem is that memories can go stale

9:44

of course. Something that was relevant in the past

9:47

might not be relevant today

9:48

or maybe it was written incorrectly

9:51

or even maliciously injected by someone trying to

9:55

prompt inject your agents to write

9:56

bad things to memory. So

9:58

you have to have a lot of guardrails

9:59

in place to make sure that these

10:02

nice autonomous memory systems actually work in production

10:08

And so

10:09

I'm going to talk through a couple of

10:11

key principles

10:12

that we

10:13

use when designing memory systems in production

10:15

to make sure that we do get

10:17

to use all of those nice effects that we've talked

10:19

that we've spoken about

10:22

So the very first thing is versioning

10:24

So when you're designing any kind of memory system

10:27

you need to be able to store versions

10:29

to keep track of what's going on

10:31

to allow you to roll back should you need to

10:34

if a new update isn't particularly good

10:37

Additionally, you probably want to think about

10:39

like, what context was this update based on?

10:42

So which agent session

10:43

which transcript

10:45

resulted in me wanting to make this update?

10:49

And additionally, like, you might want to track

10:51

like, who did it, which agent, which human, et cetera, et cetera

10:55

So this is really important

10:56

The second thing is concurrency

10:59

So we've talked about

10:59

okay

11:00

what happens when I deploy thousands of agents

11:01

all working off the same memory system

11:05

And the solution that we've adopted here

11:08

is to have this hashing system

11:10

So

11:11

when an agent decides that it wants to

11:13

write an update to a memory

11:15

it takes a hash

11:17

it then drafts its edit

11:18

and then before it writes the update

11:21

it takes another hash

11:23

If those two things do not match

11:24

then the agent cannot write it because

11:27

it means that some update was made in

11:28

the meantime

11:30

And in order to handle that

11:31

the agent repools the memory

11:33

drafts its new update

11:34

and then tries to commit this again

11:36

So these are the kinds of

11:37

just engineering practices that allow you to scale multiple agent architectures

11:43

scale memory to these kinds of architectures

11:48

Another couple key principles

11:50

so permissioning is really important

11:54

When you have large memory bases

11:56

you probably have a mixture of top-level

11:59

organizational wide knowledge. It might be like your key

12:03

like what your organization is trying to achieve

12:05

or key principles about the code base

12:07

which you've really carefully curated

12:11

all the way down to the level of

12:12

like a scratch pad for an agent

12:14

where it writes down its working memory

12:16

and it's very like individualized

12:18

And all the way in between

12:19

you could have things for specific organizations or

12:21

cross sections

12:23

And so it's really important that you have

12:25

guardrails when it comes to permissioning

12:27

these memories

12:29

So like I said

12:30

you wouldn't want

12:31

one agent to just decide that

12:33

it should update the organization wide context

12:35

probably. You might want that as read only

12:37

However, for its own scratch pad

12:38

you would want it to have write access

12:42

and yeah, yeah, that's, that's permissioning

12:47

final thing, which is kind of peripheral

12:50

but still really important when you design memory

12:52

systems

12:53

Is portability

12:56

So like I mentioned before

12:59

your

12:59

curation of your memory systems

13:01

is going to be so important like

13:04

throughout the future. This is really

13:06

really important organization, user

13:08

or like work task specific context

13:12

And so it's likely that if you're putting

13:13

a lot of effort into curating this

13:16

you want it to be accessible

13:17

across potentially multiple product surfaces and

13:22

accessible by multiple systems

13:24

So designing it in a way with a

13:25

clean API in which it's portable and you

13:27

can access it is really important

13:31

And so when you put all these things together

13:34

we have the kind of

13:36

learnings we have from allowing agents to creatively manage their memory

13:40

and then these production level

13:41

guardrails that allow them to like reasonably

13:44

use all of those principles in practice

13:47

And when you do this

13:48

you get very effective results

13:50

So

13:51

just sharing here a couple of learnings from

13:53

what we've seen deploying

13:54

these large-scale memory systems in production

13:58

And for example, we see you get better accuracy

14:02

so you have this effect

14:04

where the second time the agent does the task

14:06

it actually does it better

14:08

with higher results

14:10

because it's noted all of those memories

14:12

about what went wrong

14:14

Secondly, that then has second-order effects on the speed and latency

14:18

sorry, speed and cost of your agents running

14:21

because they're then spending fewer tokens

14:23

they can

14:24

more easily one-shot these tasks

14:26

because they actually know what they're doing

14:29

And you'll see that across all sorts of different processes

14:32

agents are just able to

14:33

do the task better and faster

14:35

Finally

14:36

having this

14:37

process where your agents are starting to autonomously

14:40

write their own memories

14:42

frees up

14:43

Capacity and context for you

14:44

as

14:45

product developers potentially to focus on product wins

14:48

while you know that the agents are doing

14:49

this kind of self-learning

14:51

continual learning loop in the background

14:53

And yeah, once that infrastructure is set up really well

14:56

this, this works very symbiotically

15:02

as ever

15:03

we do then reach a new bottleneck

15:06

And this specifically is about in-band memory

15:10

So in-band memory, as I mentioned before

15:13

is when agents are writing to and reading

15:16

from memory within a specific session

15:19

So if you think about Claude code

15:20

for example, when you like spin up a new session

15:24

it's

15:24

it's

15:25

largely like focusing on that specific context when it's reading to

15:28

reading and writing from memory

15:31

and this just architecturally or philosophically has limitations in the general

15:36

like agent fleets, continual learning objectives

15:39

There's two, two main reasons why

15:41

First of all

15:42

is that you have this inherent split of

15:44

focus and resources

15:46

So you're asking an agent to complete a task

15:49

but at the same time

15:50

you're also asking it to invest in memory curation

15:53

which would help it perform better in a

15:55

future run

15:56

So when you put these things together

15:58

it's just a very difficult optimization problem

16:00

because how much capacity should an agent put into

16:03

like

16:03

helping future versions of itself

16:04

versus doing the task that you actually asked

16:06

it to do?

16:09

And also, there's, like, other effects, like latency

16:10

for example

16:12

The second thing is that

16:14

the agents just have an inherent visibility limitation

16:18

So

16:19

they

16:20

only have a context of what's going on

16:21

in their session

16:23

As such

16:23

they just won't see patterns that happen across

16:25

sessions. So when you get frustrated that your

16:27

agent keeps making the same mistake over sessions

16:29

it just doesn't understand how frustrating that is

16:32

because it

16:33

has a new context window in each of those

16:35

Secondly, when you're running multiple fleets of agents in different environments

16:39

You

16:40

these single agents just don't have the context

16:42

of what other failures other agents are running

16:44

into

16:45

So for these two reasons

16:46

we introduce this concept of some out-of-band memory curation

16:50

and this

16:52

helps to make these problems go away

16:54

And just to introduce an analogy for

16:56

why this in theory should work

16:59

I'd like you to think about a school

17:01

for example

17:02

where you have lots of students that submit

17:03

a lot of work

17:05

and you also have

17:07

teachers that market and a head teacher that

17:09

reviews everything

17:11

This is

17:11

a system that we have in the real

17:12

world for good reason

17:14

and it's because when you have

17:15

certain individuals that have dedicated capacity for helping people learn

17:19

that's really effective

17:21

And when you also have people that have

17:22

visibility over the whole fleet of agents or learners

17:26

and they're able to spot patterns and then

17:27

kind of steer context or

17:29

let's say in this case

17:30

the curriculum, that's also really effective

17:32

So as always

17:34

we

17:34

kind of look to the real world to

17:36

think about how to build these systems

17:40

sorry, I also didn't touch on a final limitation

17:42

which is that memories go stale

17:44

so you need something

17:45

some pass that checks that

17:47

everything that's written there is still correct

17:51

And so we introduced this concept of dreaming

17:55

which is like a second o-

17:56

second order process over memory

17:59

So if we think about how that's been constructed

18:01

we have the, like, actual context, which agents reference and is

18:05

has useful information, the memory processes

18:08

which allow agents to kind of autonomously manage that context themselves

18:12

and then dreaming

18:14

which is a process that runs in batch and asynchronously

18:18

with its own allocated resources

18:20

to ensure that those memories themselves are effective

18:24

up-to-date, and helping the agents learn over time

18:31

So, what does dreaming look like?

18:35

Essentially, what we do Is we take an existing memory store

18:40

so this is a collection of memories

18:43

We then take

18:45

a bunch of sessions or transcripts from agent

18:47

interactions over a period of time

18:51

and we give these together to an agent

18:53

which reviews all of the transcripts

18:56

looks at the memory store

18:57

and starts to identify patterns

18:59

for where there could be

19:01

uplift in the memories

19:05

It then outputs a new memory store

19:08

where there are

19:09

proposed changes to the existing memory store

19:12

And what the agent is able to do

19:13

as I mentioned

19:14

is

19:15

spend tokens on solving this problem of making agents learn better

19:19

identify patterns for where agents are consistently failing

19:24

and then propose changes for what

19:26

might make a more effective memory store

19:28

such that next day

19:29

when you run these agents again

19:31

they'll actually feel smarter and they're running better

19:35

To go back to my analogy

19:36

just to paint some pictures of like

19:37

what this could look like in practice

19:40

Let's imagine

19:41

that the

19:42

head teacher reviewing all these transcripts notices that

19:45

every geography student has incorrectly answered a question

19:49

They're just all writing like complete garbage to

19:51

this question

19:53

The teacher notices that actually by kind of

19:56

in this case, analyzing the memory store

19:58

that entire topic is missing from the curriculum

20:01

So what the teacher was able to do

20:02

is notice that pattern

20:04

look at the memory store and suggest a

20:06

new change to that curriculum such that the

20:08

next day when these agents run

20:10

they now have that information that they needed

20:13

To give another example

20:14

the teacher might notice that

20:17

in a certain maths exam

20:18

all of the answers are wrong in the

20:19

same way

20:21

all of the students are outputting radians when

20:23

it's meant to be degrees

20:24

I don't know if anyone else in

20:25

like, GCSE maths had that problem too

20:28

but what they're able to do is give an instruction saying

20:31

like

20:31

this is how you should configure your calculators

20:33

And in the case of agents

20:36

that's like

20:37

noticing in the transcripts that there's something wrong

20:39

with a tool configuration

20:40

So you might notice that

20:42

something in the tool calls keeps failing

20:44

And what's important here

20:46

is that when we look at those transcripts

20:49

we're not just looking at kind of the passes of

20:50

like

20:51

responses between agent and

20:54

the system or the user

20:55

We're also really scrutinizing

20:57

like

20:57

those tool calls and all of the other

20:58

metadata that is really central to the agent's performance

21:03

Finally

21:04

you could also notice something that's like fleet-wide

21:06

or organization-wide

21:08

So for example

21:09

like everybody's using too many em dashes and

21:10

you don't like that

21:11

so you want to

21:13

add some organizational-wide

21:16

announcement or

21:17

context change that says not to do that

21:20

And so

21:21

I hope that paints the picture of like

21:22

why this could be really

21:22

really effective

21:24

And now I'll just talk about

21:27

how you

21:29

would go about designing such a system

21:31

in production

21:33

So you have some concept of your memory store

21:36

which is a concept

21:37

which is a collection of memories

21:39

Memories themselves might just be markdown files

21:42

organized in this directory

21:44

You then take a number of the transcripts

21:47

and like I mentioned

21:48

that is a mixture of like

21:51

the back and forth passes between the agents

21:53

as well as metadata on tools

21:56

other skills they used

21:56

et cetera

21:58

And the way that we've designed it

22:00

we have the orchestrator deploy a fleet of

22:02

sub-agents that go and analyze all those transcripts

22:07

And one point worth making here

22:09

is that when you design these systems

22:11

you have the ability to steer how these agents

22:14

which both write memories and coordinate dreaming

22:17

go about the problem

22:19

And by steering, I mean that you're able to tell them

22:21

like, in your specific case

22:23

these are the kinds of things I think

22:24

are important and relevant

22:26

these are the kinds of things that are

22:27

not important and relevant

22:28

So you do have the

22:29

ability there to start to

22:31

curate that memory and dreaming process

22:34

to your organization specifically

22:37

And

22:39

the orchestrator then reviews all of the responses from the sub-agents

22:43

and it then decides

22:45

like

22:45

where there are prevalent enough patterns that it

22:48

thinks this warrants a change in the memories

22:52

From there

22:52

it proposes individual changes to the memory store

22:56

And in our case

22:57

the way that we've designed this in production

22:59

is

23:00

the agent will additionally give you examples

23:02

of transcripts where it's noticed this pattern has happened

23:05

and also some stats on like

23:07

how prevalent this issue is and why this warrants actually updating

23:11

the memory store

23:14

And so all of this works really neatly

23:16

You get this output

23:17

and you as the individual

23:19

can decide where you want to accept changes to the memory

23:24

where you want to reject them

23:25

et cetera

23:27

And this works really effectively

23:31

So together

23:33

we have these two processes that run in

23:35

parallel

23:37

The first is memory

23:39

and these agents are using some of their

23:45

Like in-band contacts and in-band

23:47

resources to write to memory where they think

23:49

it's important

23:50

And this is neat because it means that

23:52

in the actual next run

23:54

the next session, that agent will be better

23:56

So there's a shorter time to kind of

23:58

seeing that change

23:59

But

24:00

inherently these agents have competing resources

24:03

when they think about what to dedicate to memory

24:05

what to dedicate to completing the task

24:08

and additionally a lack of visibility

24:11

So on the other side

24:12

we have dreaming, which is this out-of-band process

24:16

and this allows, again, broader visibility and dedicated capacity

24:21

i.e. token spend

24:22

which is specifically directed towards helping agents

24:25

learn better

24:27

And you might think

24:29

okay, that sounds really expensive

24:31

Why would I want to chuck extra resources

24:32

at this?

24:34

But if we kind of go back to

24:35

the improvements we saw when you build effective memory stores

24:38

actually you can see a bunch of costs

24:40

go down because

24:43

agents are able to one-shot things more effectively

24:45

They have more information that they need

24:48

in order to perform a task well

24:53

So, to summarize

24:58

at the very least

24:59

do the simple thing that works

25:00

Context management

25:02

makes such a huge difference to your agent

25:05

performance

25:07

Implementing things like a ClaudeMD file

25:09

like skills, which I'm sure you've heard about a bunch

25:12

and allowing agents to autonomously manage these systems

25:15

themselves goes a really long way

25:17

Once you think about scaling those things

25:20

into architectures with many agents

25:23

agents that run over a very long time

25:27

situations where you will

25:28

like

25:29

continue to work and develop

25:30

on a workspace or a code base over

25:32

a long period of time

25:33

Or very complex domains

25:36

you should start thinking about

25:37

adding some features or some guardrails that allow

25:40

those agents to manage their memory

25:42

in a way that is safe

25:43

verifiable, auditable

25:47

I should also say here that whilst this kind of

25:51

these kinds of practices are really effective when

25:53

it comes to coding tasks

25:55

for example

25:55

this by no means is just specific to

25:57

coding. Like I use memory all the time

26:00

when I'm

26:00

producing presentations that has context on like

26:03

how I like to write things

26:04

how I like my slides

26:04

et cetera, et cetera

26:05

and that develops over time

26:06

So this is really not coding specific

26:09

The final thing

26:11

is

26:12

if you really want to kind of like

26:13

close the loop here

26:15

think about adding an additional out of band process

26:18

like dreaming, as we call it

26:20

to consolidate your memory and

26:23

cut things that are no longer relevant

26:25

add things that agents are missing

26:27

and clean up and organize memory systems

26:31

So to close

26:33

I want to say that this

26:35

journey that we've been on with Context Engineering

26:37

a lot of this stuff

26:38

has only happened in this past year

26:41

This is very much an open area of research and development

26:45

and one which we see huge value

26:46

in the future

26:48

So we're so excited to see the kinds

26:49

of things and contributions that you'll all make

26:51

to this space

26:52

So I encourage you to keep thinking

26:54

keep learning, and keep dreaming

26:56

Thank you

27:03

Wonderful

27:05

Thank you, Lamis. That was great

27:08

do we have any questions in the room?

27:09

Oh, golly gosh

27:11

I'll dive straight here first

27:14

Thank you

27:15

Thanks for the presentation

27:18

do you have any

27:19

memory store implementation that you would like to suggest?

27:24

Any, any memory storage implementation that you would like us

27:28

Sorry, did you say papers?

27:30

No

27:32

Oh, okay, okay, that we'd like to suggest

27:35

To solve what problem specifically?

27:39

Well

27:39

one thing is put things in files that

27:41

I have on my laptop

27:43

but I think I'm looking into something more enterprise-y

27:47

so what kind of solution do you suggest over there?

27:52

Okay, yeah, so, I mean

27:53

maybe I was kind of coy in the

27:54

talk because we're not allowed to make product call to actions

27:57

but given that you asked

28:02

we have

28:03

a lot of this

28:04

references the architecture that we used in our memory

28:09

our memory infrastructure for our managed agent solutions

28:12

and so when I talk about these things about productionizing

28:16

memory, so everything like versioning

28:20

hashing, et cetera

28:20

that's all available within our memory and dreaming

28:23

API through cloud managed agents

28:25

So if you

28:26

did want an out-of-box solution for this kind of thing

28:28

that is where I would point you to

28:33

hi

28:34

the early on you talked about guardrails and permissions

28:39

and I think

28:40

I'm sure most of us have probably read

28:41

the Claude code leak and the memory stuff

28:44

The dreaming stuff was definitely some of the

28:45

most interesting in it

28:47

But how do you scale that

28:49

at enterprise if you've got hundreds of users

28:52

with different permission sets

28:54

how do you make sure dreaming follows those

28:56

same guardrails?

28:58

If it's

28:59

happening out of band and the context is

29:01

different compared to say the context the agent

29:04

might have when the user is using it

29:08

So just to check that I understand

29:11

we have like some permissioning

29:12

about what agents can access like in terms

29:14

of memory and then a separate like

29:17

yeah, so I mean I think these

29:18

these things compose quite well

29:21

because when you

29:22

set up a dreaming procedure

29:24

You decide exactly which session transcripts to attach

29:28

and so you could

29:28

build a process which mirrors whatever permissioning you

29:31

have on the agents

29:33

so yeah, I mean, if

29:35

that's to say that

29:36

it's not the case that when you kind

29:37

of trigger a dreaming job

29:39

it just takes like everything in a certain

29:40

time period. You can configure it that way

29:43

but

29:43

you could also just search over whichever transcripts

29:46

have the same permission set as like

29:48

whatever your memory store is

29:50

and then make sure that that matches

29:51

if that makes sense

29:55

Hi, thank you. This was really

29:57

really interesting

29:58

I

29:58

I found it

29:58

fascinating earlier when you were mentioning about like versioning

30:01

concurrency, durability, all of these, all these things

30:04

At what point are we like reinventing

30:06

databases from like first principles again?

30:12

yeah, this is, this is an interesting point

30:15

I still think, like one of the things that I suppose

30:20

and I, I, this is a good reminder for me

30:21

but something I didn't say

30:24

What we're trying to do here is

30:25

like, thread the needle, like, sorry

30:27

find the right boundary to draw between

30:30

kind of letting these agents autonomously act

30:32

and then also, like

30:33

which things should just be kind of programmatic

30:37

things that are baked into the harness

30:39

And so I think what you allude to is

30:40

like, first of all, we were just kind of

30:43

like, letting these agents write in Markdown files and just

30:45

like, commit whatever they wanted

30:46

and now we're kind of seeing

30:48

having seen which primitives work really well

30:49

we're thinking about, like

30:50

kind of codifying that in the harness

30:52

And so when we think about

30:53

like, the hashing or the versioning

30:54

yeah

30:55

we are kind of going back to the

30:56

software engineering principles that we've seen work well in the past

30:59

but in a way that kind of autonomous

31:01

agents can act and

31:02

like

31:05

can interact with those really effectively

31:06

So I think to some extent

31:08

like, we sort of are merging back into those practices

31:12

but that's because

31:12

we have enough signal now to know that

31:14

those things should just be done in a very deterministic way

31:17

and there's no need to reinvent the wheel

31:18

I hope that answers your question

31:21

Perfect. We're absolutely out of time

31:23

Thank you once again. Big round of applause

31:25

for Lamis

31:26

Thank you