Write, Change, Recall, Forget: MongoDB's Pete Johnson on How Retrieval Drives Agent Performance

Thanks so much for having me, Nathan. I appreciate it.

Sure. I like to think of this in terms of the history, in terms of the different kinds of problems we've had to try to solve with software over the last sixty years or so. And where it really starts for me is in June 1970 is when EF Cod, who's an IBM researcher, wrote the original white paper that gave birth to SQL.

So it was June of 1970. I happen to have been born in February 1970. So I am as old as SQL, Nathan.

And if you think about the kinds of applications we were building at the time as an industry and the kinds of hardware we had at our disposal at the time. The applications were very departmental in their nature. Right?

Not every not every company had a computer, and those that did, the people that were using it were often on the same floor or certainly in the same building. They were working nine to five hours. It was okay to have downtime on the weekend.

And if you think about the big three piece of hardware at the time, memory, compute, and storage, storage was by far the most expensive of those three things in 1970. So if you think about how you might organize data, keeping in mind that the scarce resource was storage, you would of course do it in such a way that's now known as normalization. So you may have in university or in coding school, you may have learned things like first and third normal form, all about trying to lay out data on disk in the most efficient way so that you aren't storing any piece of data more than once.

The simplest example I can give with that might be my wife and I share an address and maybe with a retailer might wanna know what our address is. There's two people at that address. You might have in one table the list of people.

In another table, you might have the addresses. You might have a third table that links the people to the address so that you're only storing that address one time. And that's the most disk efficient way to do that with, like I said, some kind of normalization.

Well, forward to 2007, which in October 2007 is when the first commit was made to MongoDB and the world is a very different place in 2007. The world has internet. The world has cloud.

The world has mobile devices. IPhone came out in The US for the first time that year. So it's a very different world and the scarce resource in that world because we had forty seven years of Moore's law between those two dates.

The scarce resource in that ecosystem became time because the kinds of applications that we would write that were in the cloud or that we would write for mobile devices, you could not have downtime on the weekends. It was not people who were confined to the same building. It was not nine to five use cases.

So it was very different world that required a very different kind of database, and that's what gave rise to NoSQL. Now our flavor of NoSQL, which like I said, that first commit was in October 2007, uses it as its core JSON. So you've mentioned the stuff with JavaScript.

That's why that was so popular fifteen years ago or so is because what you get back from your API call is JSON. We're storing it in transit as JSON. We're storing it on disk and a binary form of JSON we call BSON and that's part of why we're so fast is because we're keeping that format the whole time so that when you serialize it off a disk, when it goes through our servers, when it ends up in your client, it's essentially staying in the same format that whole time.

So if you think about the kinds of applications that we've been building since 2007, like I said, disk space is no longer the scarce resource. If you take that address example, if you denormalize that data just a little bit, if you instead of storing it across three tables, if you'd have a JSON structure that has my name and my address and then a second one that's got my wife's name and the same address, you're duplicating the address but what do you get in return? What you get in return is one disk read.

So that's why it's faster. So instead of having to do three disk reads to get that same data, you can now do it in one disk read. Does that mean that every problem is solved by a no SQL database or by MongoDB in particular?

No. But it also doesn't mean that every problem has to be solved with SQL. And that's how that sort of evolved out of SQL was invented at a time where the problems were slightly different and a lot of those problems still exist.

SQL is certainly a valid solution to lots of different problems that we still have. But when you need speed, when you need scale, and increasingly, when you need better retrieval quality, that's really where MongoDB shines.

Sure. It it's a mix. Like you just said, we're in like 75% of the Fortune 500.

But if you look at the overall market, you mentioned our market cap, we're about 2,500,000,000 is what we ended our revenue that was our revenue number for our fiscal twenty six. We're currently in the middle of our fiscal twenty seven, and that's about 3% of the 100 to $110,000,000,000 database market. Where most enterprises choose us is those first two things that I mentioned.

When you need it fast or when you need it at higher scale. Because you're not storing disk, you're not storing that data across multiple places of disk, that's how you get the speed, that's how you can get better scale if you're if you're storing. If you're gonna retrieve JSON because every modern programming language can serialize JSON into some object format so that you could use it immediately in your business logic.

And that's the other aspect of time. It's not just wall time for the transaction, but it's also the developer time of what do I have to do with this WAD that I get back from my API call and then make some kind of useful business decision out of it. But what we see is right solution for the right problem.

And increasingly, what AI has done is it's opened up the number of problems that were a better foot fit for than a traditional sequel approach, especially those at higher scale. And that's why I mentioned the three things, the speed, the scale, the retrieval quality has really become an important factor. It continues to especially later here in 2026 as you start to see people strategize around token maxing to evolve different kinds of agentic architectures in ways that we didn't see in the first half of twenty twenty six.

Retrieval quality has become more important for AI, but it it it does end up being a mix. It depends on the corporation. Some of them will take it workload by workload and for this workload, we'll use Mongo and for a different workload, we'll use SQL.

Others are taking an all in strategy. It really varies based on, in some cases, how much greenfield versus brownfield do you have in your portfolio? In other cases, it's where's my talents already?

I was just at for a major automotive company here in North America, I was just at a hackathon where we trained up, like, 250 engineers and turned them loose for three days on a hackathon and they've got some new product ideas that fell out of that. So there there's there's no single right answer. Right?

There's lots of ways to sort of walk through that forest. But at the end of the day, it does come down to right problem for a resolution for the right problem. And where we typically help people is with those modern applications that need that speed at scale and those gigantic use cases that increasingly need that better retrieval quality.

Some of it was customer experience kind of applications. Others was how can we make the dealers more efficient? How can we make the mechanics more efficient?

Some of it was how can we make the in car experience more efficient? It really spanned the gambit of different types of use cases that a big three automaker might have. But really, it's I mean, I'll go as far as to say, like, our education system has a bias, a historical bias that thou shalt always normalize.

And for a lot of use cases, that's valid, but there are an increasing number of use cases where these big three that I just mentioned matter. And increasingly, that retrieval quality one is becoming a much bigger deal for agentic architectures as you start to try to take better advantage of the context window than just stuffing it with a million tokens every time. And we can get into some of the details as to why we're a good fit for that and why that's emerged, like, in the wake of what happened to Uber earlier in the year with them burning through their token budget for twenty twenty six and thirteen weeks.

That grabbed a lot of headlines, and there's been similar stories about where, like, maybe token maxing wasn't such a great idea to begin with.

Sure. So let me so first, let me push back on one thing you just said. Most people think that we're schemaless and that's not actually true.

It's not that we're schemaless, it's that we're schema flexible. That you can change the schema more easily over time because more of the data is denormalized and therefore centralized in a way. It's not a strict requirement unless you specifically say so that every document in a collection, so we don't have tables and rows.

Right? We have a think of a JSON WAD as a document and you have multiple JSON WADs put together into a collection. Not every document in a collection necessarily has to have the same shape.

You can have different shaped documents in the same collection. So because the schemas are flexible and because you can have different doc different document shapes in the same collection, that's why people assume that we're schemaless. But what it does is it gives us the ability to be malleable in a way that a traditional SQL schema is less malleable.

I won't say not malleable at all because it depends on how you're laying out the data. But anybody who's ever had to change a SQL schema that's already in production and the cascading effect that has on multiple tables knows the pain of what I'm talking about that if you had some of that data denormalized, it's far easier to add attributes to a document that's already there and do so selectively in a way that isn't possible nearly the same extent in the sequel world. And this is where how we've implemented like the history of how we implemented vector search and the impact that has on application architectures for agents ends up like the details matter here.

So if you'll allow me, I'd like to talk a little bit about that history. Does that sound okay? Yeah, please.

Okay. So for us, it really started with an unusual place and that is with lexical search. So in 2020, we noticed that a common use case for MongoDB was for people to stand up their own Apache Lucene servers co located with wherever their their MongoDB clusters might be.

And the reason they were doing is they wanted to be able to point that Lucene cluster to different text fields in a document and be able to do keyword retrieval off of them. Perfectly reasonable things. We thought, well, as you might know, there's there's three different versions of MongoDB.

There's community which is you're responsible for the support, you're responsible for the operations. There's enterprise advanced, which most customers are using on prem, where you're responsible for the operations, but we're responsible for the support. And then there's Atlas, which is our managed service version of it, which you can deploy your instances on any cloud hyperscaler data center you'd like amongst Amazon, Google, and Azure.

And in that form factor, we will do the support and we'll do the operations. So those are the different choices you have when you deploy MongoDB. And regardless of which one you choose, like I said, we noticed people standing up Lucene servers so that they could get keyword retrieval on what they have.

So in 2020, we introduced what's now known as Atlas Search which is on the managed version the managed version of MongoDB. You automatically just you just get it as part of that instance that that there's a lower priced tier where you just point it to attributes, some text attributes that you already have in your data will automatically index them. You can do keyword retrieval search on those.

Then the next logical thing was a couple years later, well, if you're gonna have lexical search, you might as well also have vector search. So think about this, what's a vector? At the end of the day, a vector is an array of floats.

Right? You take some piece of data and whether that piece of data is text or an image or audio or video, you pass it to an embedding model of your choosing and what you get back is an array of floats. So to us, an array of floats is just an additional attribute to a document you already have.

So for us to implement vector search was just add an additional attribute to the flexible schemas that we already have and now build a vector index on top of that array of floats and that's your vector search. And what it created was this notion of being able to do these powerful hybrid searches as well. So if you think about the sort of the trivial example that most people learn MongoDB off of is suppose you have a book and a book might have a text field for the title.

And it might have a text field that is URL that points to the cover of that book. You might have an integer that's number of pages, you might have an integer that's the year of publish, and you might have a text field that's the synopsis. So if that's a standard record for a book in if that's a standard document for a book in MongoDB and that's put in a collection, suppose now I want to create a vector index off of the synopsis.

I could take that text synopsis text, I could pass it through an embedding model of my choice and I then store the array of floats that come back from that embedding model. And if I wanted to, suppose I had 10,000 records, I had 10,000 documents in my collection, maybe I only wanna do a vector search on the top thousand. Maybe I don't wanna do it on all of them.

Because of the flexibility of the schemas, because you can have shapes, MongoDB not only allows that but thrives on it really well. So we had lexical search, we had vector search, and then that enabled us to do this notion of hybrid search. So suppose I wanted to do combine a lexical search of the title of the book with this now vector search that I have on the synopsis.

And what if I also wanted to pre filter because I've got some integer fields in here. Suppose I wanna do a search there where I do a lexical search on the subject and a lex and a vector search on the synopsis but I want to eliminate any documents, any books that weren't published after the that were public. Oh, I only wanna see books in my dataset that were published after the year 2000.

So now I have these three levers of query power that together are giving me lots of interesting ways now query my data that I combine this sort of pre filtering based on the metadata with the lexical search with the with the vector search. Once So we had that in play by like 2023, we thought, well, this works with any embedding model you want, but could we make this easier to develop if we had an embedding model that was part of it? And that's why we purchased Voyage in 2025, which should create this better together story.

We can get into all kinds of things with the better together, but the way that we go to market with Voyage is you can use Voyage on whatever vector database you want. You can use any embedding model and use MongoDB as your vector database or you can use them together and there's a better together story that's part of that. So that's kind of our history.

Like I said, kind of unusual that it started with lexical search, but it kind of went lexical search, vector search, hybrid search with pre filtering, and now this better together storage with the Voyager bedding models. That's how that built over time for us.

That is. So one of the advantages that we have in the way that we implemented vector search is it gets to stand on the shoulders of the base product that we already had. So we've been running Atlas in production for a decade now.

So we already knew know how to replicate across multiple data centers, even across multiple clouds. We already know how to do sharding, whether you wanna keep data close to specific users for business reasons or for legal reasons. We already know how to do security, not only query at rest or not only encrypted at rest and encrypted in transit, but we can also do during the query.

We get to stand on the shoulders of all of that for the vector search. And one of the things that we get to do on top of that is MongoDB has always had this notion of aggregation pipelines and the point of aggregation pipelines is instead of forcing you to do a query, get some JSON wad back of your result set and then have to massage that before you use it. Instead, with the aggregation pipelines, you can give extra you can give extra instructions to the back end so that that data gets sorted, filtered, massaged in different ways so that what you get back from that API call, you can now use immediately and you don't have to do any other further massaging.

So because we already had that mechanic in the core product, what you're referring to is we just announced a pair of features, one's called rank fusion and one's called score fusion. And what that allows you to do is in absence of an aggregation pipeline in these new aggregation pipeline stages, you would have to make the vector search in the and then independently, you would have to make the lexical search and then you would have to combine them yourself. Now with those two aggregation pipeline stages, you can do it in one API call.

So you only have one round trip call to our servers and then on the back end, we'll do both of those calls for you and then we'll combine the results either based on their rank with rank fusion or based on their score with score fusion. So that what you get back is an already sorted list based on which one of those functions in the aggregation pipeline you chose so that you could then use that JSON WAD, that result set immediately.

So for now, what most people are doing is and this is the reason why we have the pipeline stages that we do. It's it's usually based on a rank or by score, and if you also throw in the pre filtering that lets you that lets you reduce the dataset before you do the queries, and then you get those queries back and you can interleave them based on either score or rank. For now, that's what most people are trying to do.

But my joke for this, Nathan, is like there's two kinds of developers. There's those that love MongoDB and there's ones that haven't tried us. Like we're we're we're always in a pretty tight loop of feedback where we the whole reason we came up with Rank Fusion and ScoreFusion is because the community told us they wanted it.

And very much the same way that why did we start doing Atlas search? Why didn't we make lexical search part of the platforms? Because we saw people were using it.

We found a way we saw a way to make their lives easier. If it's so that you don't have to spend the operational cycles managing your own Lucene servers, what could you do with the cycles that we give back to you? And it's the same thing with StrainFusion and ScoreFusion so that you don't have to manage that code base yourself so that if you could just throw it in as a verb in the query that you already have.

Okay. Now, what are you gonna do with that with that time that we're giving back to you?

So there's a couple of more broadly speaking, and I promise I'll answer your question. More broadly speaking, the use case for this level of search, for this kind of semantic or vector searching is typically rag pipelines for agents or for a more sophisticated agentic memory for agents. That's when you typically see vector search start to enter the conversation when it comes into an agentic architecture in some way.

Specifically, what you're talking about here is chunking and chunk size tends to be more in that rag pipeline use case. I've got some external data, I've got some proprietary data, I wanna inject that proprietary data into my LLM of choice without having to go through an expensive fine tuning or training process because the LLMs are trained on public data. They're not trained on my proprietary data.

So how do I inject that in a safe way that still gives me some data fidelity so that I'm not I'm not giving away my proprietary data, but I can still get it to focus on my business problems instead of every LLM knows who the last 30 Oscar winners were, and that's not super helpful when I'm trying to get an agent to help me figure out what my knock should do next or my my IT ticketing should do next. So that's the use case. So if you've got a bunch of documentation that someone who is a human professional in that world, you now need to figure out how big of the how big that should the chunks be that I place in the individual documents and what's the fidelity of that.

And when you're the application developer trying to figure this out, there's this trade off that you're looking at. So if I make my chunks too small, let's say to the fidelity of a sentence. If I make my I potentially lose the context in which that sentence appears.

Just if you just read one sentence from a novel, doesn't tell you a whole lot. But if you then also read the paragraph or the page, it tells you a lot more about what the context of that sentence is. So that's an argument for, well, maybe I should make my chunks bigger.

There comes a point though where if my chunks are too big, my storage costs start to go up and my retrieval quality starts to go down because if I've got three pages, let's say, I'm trying to get to the fidelity of a sentence, I now have too much information and I'd lose the fidelity of that sentence. So what's the right answer? I I've been doing this a long time, Nathan.

The answer is always it depends. Right? You have to go through as a developer, you have to go through this iterative cycle like I chunk to a certain size, I run some tests, what's my retrieval quality?

Okay. Now I try it again. I try a different chunk size.

I do some tests. I see what my retrieval quality is and I have to do that three or four times that for my use case to try to find out what the right balance is between chunk size, storage costs, and retrieval quality. So one of the reasons we bought Voyage is because like those guys, like their side hustle is teaching people at Stanford how to how to build LLMs.

Those guys are all geniuses and they came up with this new concept last summer called contextualized chunking. And we've got a blog article on this that we can potentially link through here. But the idea here is that if left to your own devices and figuring out your own iterations of chunk sizes, there comes a point in a graph where if you've got retrieval quality on your x on your y axis and you've got chunk size on your x axis.

With a traditional embedding model, there comes a point where at lower chunk sizes, you have zero context. So the retrieval quality is low and it it builds. But then at some point, it flattens and then it starts to degrade if your chunk sizes become too big and that's why you have to go through these iterations to find the right combination.

But what if you didn't? What if instead of sending that all as one big text blob? What if instead you send it as two?

What if you sent the fidelity of the sentence you actually want and then as a second string, you send the size you send the other contextual informations where the name contextualized chunking comes from. And we will figure out for you what the right chunk size is for that combination and we will give you a vector array that comes back that balances those things for you. When you flip that script, it turns out you can get better retrieval quality with a smaller chunk size which otherwise is not possible.

If you're doing it the traditional way, you the way to get better chunk side way to get better retrieval quality was with higher chunk size. But if you use this contextualized chunking, we released version three of that last summer. We just released four of that in the last six weeks.

You can flip that script. Now, there still are use cases where you need to control the chunk size for different things. But if that's something that you wanted to not worry about or have to learn about, and the way I think about this is more developers will build AI agents in the next three years than did in the last three years.

And in order to make that possible, we have to lower the learning curve. We have to make it easier for the developer ecosystem to learn how to do this. So if you don't wanna have to learn how to go through the iterations of figuring out what the right balance of chunk size and retrieval quality is.

Instead, if you could use contextualized chunky, we'll figure it out for you and you still get good retrieval quality out of it. So that's that's one of the benefits that Voyage offers that no other embedding model out on the market offers, that ease of use feature that gets you better retrieval quality.

Yes. That's exactly how it works. So you get back one array of floats exactly how you did before.

It's just now you don't have to go through the iterations of figuring out the size of the chunks you need. And similarly, there's another voyage feature that all the voyage all the voyage embedding models have called Retroska Riesling because the other place you have to make a decision about storage cost versus retrieval quality is with number of dimensions. So what do I mean by that?

Everybody knows what two dimensions is if you've taken high school level algebra. Right? X y.

Right? But in an embedding space, you tend to have at least two fifty six dimensions and sometimes as high as twenty forty eight. And the more dimensions you have, so each dimension is represented by one of those floats in that array of floats.

The more dimensions you have, the richer your embedding space is and the better retrieval quality you get. But at higher dimensions, that doesn't come for free. You've got a storing 256 floats, takes up less space on disk and in the index and memory than it would for twenty forty eight.

So you again have to go through this iteration of what's the right number of dimensions for my use case given my what my storage costs might be. All the voyage models have a feature in them called Retroska reasoning. It comes from the Russian nesting dolls.

If you think about how Russian nesting dolls work, right? You've got you've got one doll and you open it up and there's another one exactly the same but smaller, a smaller Fidelity inside and then you keep doing that in over and over. So what what the how the how the voyage models work is when we generate let's suppose you did it 10/24.

Suppose you wanted ten twenty four dimensions. You run some tests, now you wanna try five twelve. With a traditional model, you have to run your entire corpus of data through a second time at five twelve.

But with voyage models, you don't have to do that. When you've run it through at ten twenty four, vectors, those floats, they're ordered. So if you wanna try five twelve, you just lop off the last five twelve and that's now you immediately can begin testing on the remaining five twelve.

So again, it doesn't completely solve the problem of figuring out what the right combination of retrieval quality and storage space is, But it helps you get to the answer faster. So these are there when you take some of these things in sum, there would be three or four of these kind of features that make it easier to use and help you get to your final answer more quickly And the idea there is to give you time back in your day that you can work on your business logic instead of figuring out the plumbing.

Well, I'd return to the three things I mentioned in our first twenty minutes or so. It's it's speed, scale, and retrieval quality are the main three things. What most people do, and you're not gonna offend me if this is what you did, I mean, people start with Postgres and PG vector, then they choose they choose their embedding model with whatever cloud they're using.

Like Gemini is prominent if you're gonna use if you're gonna be on Google just like OpenAI's embedding model is is pretty popular over on Azure because of the historic relationship that those two companies have. But there comes a point in when you're when you're doing a demo, when you're doing a POC, it doesn't always show itself. But there comes a point where when do milliseconds matter to your use case?

When does scale matter to your use case? And that typically comes depending upon chunk size that typically comes at about a 100,000 vectors is when is what I mean by scale. And when does retrieval quality matter?

If you look at Voyage AI model, Hugging Face has a benchmark out there called RTEB that Voyage AI models are typically at the top of and we can get as much as 14% improvement compared to some of those other embedding models that we just mentioned. So are there use cases for which a 14% difference in embedding model quality is the difference between a hallucination and a correct answer. And that's before you even start putting re rankers on it, which is another way that you can boost retrieval quality without having to to do anything special to your data.

So like I said, it's it's speed, it's scale, and by scale, I typically mean in the neighborhood of a 100,000 vectors and retrieval quality out of your embedding model. Most people think embedding models are commoditized and that is not true. There is a very big difference that you can get in retrieval quality based on what embedding model you choose.

Anthropic does not have an embedding model in market. They recommend us.

these production environments? Yeah. I think there's two places that I've seen that recently, and one of them is with the reranking that we were just talking about.

So in this free, we released dollar re rank, which is a companion to the ScoreFusion and rank fusion that we already talked about. So those two relate to doing a hybrid search. The dollar re ranked, again, you would in a historical way, if you would have to do your vector search once, send your results to a re ranker to get them reordered in a way that is most optimal for rag use case to then pop them into the context window.

Just like we did with the score fusion and the rank fusion. We've now got a stage where if you do dollar re rank, you call the API once. We'll do both of them for you on the back end so that you only have to make one round trip to the server for that.

So that's one place that we've seen some additional ease of use. The other one is a feature that we released recently called auto embeddings, which is you tell us this is a better together story. You tell us what which collection, you tell us which attribute on documents in that collection and you tell us which voyage model and how many dimensions you want and we'll take care of the rest of it.

Anytime an existing document with that attribute in it changes, we will automatically take that new, let's say it's text, put it through the embedding model, update the vector in the document, update the index in memory, and we'll do all that for you. If a new document shows up into that collection that has that attribute, we'll go through the same cycle for you. So again, to remove some of the plumbing so that you get some time cycles back as a developer.

So you don't have to you don't have to go craft your own and maintain over time your own embedding pipeline. You just pass us one JSON WAD, and we'll do it for you.

Sure. So I again, history matters here. Right?

When ChatGPT came out in 2022, in the end of twenty twenty two, it had a very simple application architecture. You took you took the query, you put it in a context window, the LLM would process it, and give you a result. End of story.

That's how it worked. And by spring of that year, by the next spring, you would see headlines like ChatGPT passes the bar exam and it would have these very cool things it could do. And we've already talked about why you need RAG because those LLMs even today are only trained on public data.

If you wanna inject proprietary data into it, you need to to specifically do that with a RAG pipeline. What we saw in 2025 though was the second problem that we see with with the core LLMs and that gave rise to the need of memory and that is like in spring of twenty three, if you had asked ChatGPT what's MongoDB stock price today, it couldn't tell you. And the reason it couldn't tell you was the LLMs have a knowledge cutoff date.

There's a date after which they don't know anything about the world because the model vendors at some point have to stop training and put them into inferencing mode. And the way that we overcame that in 2025 or so was with tools and with MCP and being able to have to grant to the LLM, hey, here's this API or here's this tool we could call to go get additional information. That's why whether you're a Claude person or a Gemini person or a ChatGPT person, that's why today if you ask, hey, what's MongoDB stock price?

It could tell you because it can find a tool in its arsenal, go make a web search, get that information, and report it back to you. The other thing we saw in 2025 though is this notion of looping where the result of one loop of that agent then became the input of the last one. The context window is still very dung.

Right? The context window is still you have a fresh context window with every time you pass it to the LLM and that's when we needed agentic memory. And the early passes we saw of agentic memory was short term memory was typically, okay, take every response that we got within this session and cram it into the context window so that we can see what happened earlier in this conversation.

And then long term memory was, okay, now do the same thing for every session we've had for the last three days and cram that in there as well. And then a two negative side effects that I now see people starting to push against. Negative side effect one, we already talked about is token maxing.

Right? If you if every agentic loop puts a million tokens in the context window, even con even context you don't necessarily need, your cost starts to add up. But there's also a functional cost to that.

There's a couple academic studies that show that like the first seven k and like the last seven k tokens are the most important and the stuff that's in between can end up just muddying and confusing the LLM as it's trying to come up with an answer. So instead of thinking of how could I maximize, how could I cram a 100 a million tokens into the context window every time? What if I could choose just the right 200,000 for this agentic loop?

So we're starting to see more sophisticated short term memory that doesn't take everything in the session, but just the things in the session that are relevant for this past. And the same thing with long term, and we're starting to see some new memory types, especially in fortune five hundreds. There's a new memory type called taxonomic memory.

So think about either in a vertical, so like automotive we were talking about before or in a specific company. There's some terminology that is specific to verticals or specific to a particular company that a lay person who is not in that vertical, those words may mean different things. So you might have a list of, let's say, a 100 of those terms.

But for this agentic loop, what are the five that are relevant for this loop of the agent? And then do that again for the next loop. What are the five that are relevant for that loop of the agent?

So again, instead of taking all 100 terms and putting them in the context window every time when not all of them are gonna be relevant, what if you could instead have a more sophisticated memory type memory system that instead of just taking everything, use vector search or semantic search or some combination of vector and lexical with some pre filtering and just got the just the right just there's some perfect you never get there, but there's some perfect context window for this pass, and that perfect window doesn't use all of it. It uses just the stuff that it needs to. Those are the kinds of things that we're starting to see in enterprises both to combat the cost of the tokens and to combat the relevance of the tokens so you can get the best answer out of this call to the LLM.

This is the hardest problem with agents right now, I think. And the way that I put this to people is we, as an industry, we've been building databases for sixty years. We've been building agents for like eighteen months, man.

Like nobody knows all the answers yet. We're all sort of collectively figuring it out. But what I do see with these more sophisticated memory types is that as the application author, that you have two responsibilities with these memory systems.

Where with the prior one, you only had one really. So not only do you call this memory system, hey, here's my query. Give me the best maybe you give it a token budget is the way some of these work so that you don't you don't have to figure out how many memory types there are and how to query them.

You're just like, give me the best 50 k tokens or give me the best 100 k tokens. You give it a token budget and then it gets it back from the mix of different memory types it might have. And then you put that in the context window, you get your answer, and then your second responsibility is to send that answer back to the memory system.

So the memory system can then curate it and then write it into the memory system. And when it writes it into the memory system, the more sophisticated ones also have this notion of RBAC. So that if you and I are in the same job type that we get to share the memories so that if you come up with a really good memory and then I get to reuse it later, then that ends up being beneficial for both of us.

So that's the second responsibility you have. But hardest part is what you're talking about here. The way that one coworker just wrote this to me yesterday, I wanna quote him.

Write, change, recall, forget. Because these things have a a half life to them. Right?

Like, things that are more recent are more important than ones that that took place weeks, if not months ago.

Well, there's there's a couple different ways to tackle this. Like I said, the forget part is the hard part of it. But what we see our customers doing is this is where like the retrieval quality of those memory types, that's where that that non commoditized embedding model can make a difference and where a re ranker can make a difference.

You you can get depending upon the use case, you can get like a five to 10 boost at your retrieval quality just by using a re ranker on top of whatever your embedding model of choice is. And that's true of architecturally, that's why we made it so easy to implement the re ranker on top of the voyage embedding models when you're doing that vector search. But there are use cases where maybe you have so much data that you might have to take a hybrid approach where maybe you use maybe use a graph structure for, I don't know, two to six levels of data and then you get down to a leaf and you do a vector retrieval inside that leaf.

We see people doing that as well. There's all kinds of information you can find on our website about how you can because we're JSON based, you can use MongoDB to build graph structures into your data so that you only have to go one place for a graph database, a core database, a vector database, the embedding, re ranking all in one place. So you don't have to try to stitch together multiple tools yourself and have to maintain that over time.

But we do see people doing that with graph structures. It's typically not as deep as like you you would traditionally think of as a as a graph database need. But like I said, it it's pretty common to go, I don't know, half a dozen or maybe maybe a dozen layers that maybe for bigger corpuses of data that are more heavily categorized like some of our retail customers do that with product databases.

If you think about how a hierarchy of products might appear on a website or on a mobile app, they might segment that first by product category using a using a more of a graph style. And then once you get to a particular product category, then do some vector searching within that individual category node.

Well, our biggest customer right now, they haven't been very public about how they did it. There's a company called Eleven Labs out there. They have multiple agents per customer.

This sort of micro agent approach where you get multiple smaller agents at the disposal for individual customers and given the number of customers they have. So they Eleven Lives started life as a model provider that was doing way sophisticated speech to text and text to speech and they built a platform on top of that that is more of like an audio editing suite that they do for that and as part of that, they they have a bunch of agents doing all kinds of editing and transformation kinds of things for their customers. And if you think about the kind of context memory that they have there, an important way to to do that as well, I would worry about any memory architecture that instead of relying on lower costs, embedders, and re rankers is relying on multiple passes of the LLM to help you categorize and shrink the the the corpus of data that you might wanna then put into the context window for the larger sort of more functional LLM call.

Because then that's just adding up tokens as well. I mean, why things like embedders and re rankers have a lower token cost is, again, right tool for the right problem.

They can. And so all those things add up over time. And it's it's not just tokens, but you're you're right.

The overall latency for the decision loop that you're into makes a big difference.

Yeah. So I I have the good fortune in my job. I've been to seven countries this year to talk to probably a 100 different customers about where they are in their AI journey, and they tend to fall into sort of three camps.

Camp one is I bought a license for this one tool and I'm done. Right? Like my AI strategy is done.

I bought one thing. And while that could be a good starting point, it typically is not specific enough to help solve the problems of your overall enterprise. Then you have a group of people who have done some POCs and maybe a couple of production deployments with varying degrees of success on ROI that had struggled with ROI.

I always argue probably picked the wrong problem. Picking the right problem is really important in this space and the way that I encourage customers to think about this is like, what are the top 10 to 15 problems that you have going on in your business right now? And of those, what do you have good data for?

I mean, you made a comment earlier in conversation like data quality is a big deal. Things like bad data quality and bad security posture don't get solved by AI. They get amplified by AI.

So what's your what's your biggest problems? What do you have good data for? And then what do you already have metrics for around the problem?

And that's the biggest difference that people tend to skip over when it comes to problem selection. Because if you don't already have metrics for how something is performing, you won't know if it got better. So call center use cases are very popular low hanging fruit in enterprises because I already know what cost per call is.

I already know what call volume is based on how I'm already bonusing people in those jobs. So if I introduce AI into their workflow and I see those numbers jump, I can attribute that change to the AI and I can do a back of the envelope ROI. We see the same thing with software delivery life cycle.

Early in the year, we saw all kinds of bragging about, well, now because I've got Claude code or I've got g gravity or if I've codex, now I can produce five times as much code as I did before. Anybody who's been doing it any length of time knows that lines of code is a terrible metric to judge the productivity of a set of developers off of. Instead, how how fast are you getting from idea to correction deployment?

That's the metric that matters there, not not necessarily lines of code. So the metrics there matter and that's the difference between the folks that are in that middle state either stuck in sort of POC purgatory and haven't quite got to production deployments. The biggest difference isn't in how they're applying the tech, it's what problems they chose to try to solve.

And then you've got people that are sort of more advanced that are looking at some of these more sophisticated memory types and things where they're trying to optimize the systems that they have. And in some cases, they're they're making purchases of larger platforms to help them do that. Other times, they're doing homegrown.

And like I said, we're we're like eighteen, twenty four months into this. There's there's no there's no one way to do this yet. There's there's no lamp stack for agents yet in the way that we have with web development.

We will get there. Having lived through that having lived through that life cycle, we'll eventually get there. But like, there's there's no react in Angular.

There's no LAMP stack for agents right now.

Most of Fortune five hundreds that I talk to are Dewey employee facing use cases with human in the loop for exactly the reasons that you just laid out and for some of the ROI reasons that I laid out before. So if you think about the employee facing use case, you already know what you're bonusing your different employees are and what KPIs, what key performance indicators you're using to judge whether or not they're doing a good job. So like I said, I used the example of call centers before but you can think of any job whether you're an insurance adjuster or a line worker in a manufacturing plant or whatever your job might be.

Everybody's got some key performance indicators and if you introduce AI into whatever that workflow is and you see those jump, that means that you're you're you figured out some ROI. But there's also security a data security issue for that as well. If you and I are employees for the same company, is it great if my salary accidentally leaks to you?

No. But it's way better to have that than if we are two customers of the same company and I get to see your data. That is a VP has to get on blame for that kind of scenario.

So like the risk reward is higher for customer facing use cases than it is for employee facing use cases. So at least where we are now ecosystem where I mostly see the Fortune 500, you get exceptions of course, at but I see a lot more work going into those employee facing use cases with human in the loop than I see going full autonomous AI with customer facing use cases.

So all the things you just mentioned are certainly things that not just us, but every software company is involved in one way or another. The two that I would shine spotlight on are the MCP tools to make it easier for agents to talk to your data in your MongoDB instances. We had an MCP server for a while and we were now kind of self hosting some of those as part of Atlas as part part of the ecosystem.

And then the other one is the agent skills. So think of that is kind of curated system prompts for doing things like data modeling, doing things like optimizing operations, things that we have well organized playbooks for that previously we would have as documentation, but now we have them as a series of doc of markdown files that you can feed to an agent of your choice to give it sort of a lowdown on different ways that it should approach different tasks. And there's maybe the I think it's agent skills we just dropped a couple of months ago has like the top eight.

It's like it's six to eight different agent skills that we have for things like operations and data modeling to help you do those sorts of things. So, yeah, that's become pretty commonplace these days, and we're certainly part of that ecosystem.

there was an acquisition of a VectorDB startup,

Mean, fundamentally, what we've always been about has been how can we make the day in a life of a developer easier so we can make it easier for them to build their business logic and spend less time worrying about the plumbing lower in the stack. I went through a description earlier in this conversation about how we implemented vector search. For us, because we already had the JSON based, it was it was relatively straightforward for us to just, okay, you add an additional attribute.

That attribute is your array of floats. You generate an index based on that array of floats. So that was pretty straightforward for us to add to the existing product and take advantage of the things like the charting, the security, and the data replication that we already had in the base product that some of the net new vector database companies out there have to catch up and struggle with a little bit.

So that's why we chose to sort of do vector search the way that we did is because of the flexibility based into built into the document model from the very beginning, it was it was pretty straightforward for us to do that. But where it was not as straightforward and why we made the Voyage acquisition was today, can still use whatever embedding model you want as long as it generates that array of floats. You put that array of float to your Document, you go create your index, you're off and running.

But we saw an opportunity to to a, that the market was seeing embeddings and re rankers as a commodity whereas we we saw Voyage really standing out and like I said before, Anthropic agrees with us. And we could create these better together stories like the auto embeddings that I mentioned before. Another one I didn't mention is being able in Atlas to manage all your API keys in one place so that you don't have to go to one console for your core data, another console for your vector data, a third console for your embedding models, and a fourth console for your re rankers.

You can get all that in one place. The management of it is easier over time. So if you look at the history of MongoDB features and really what our focus has been, we're all developers at heart and it's it's about making it easier for the developer ecosystem to learn and operate this these new techniques that we see in application architectures around agents, whether it's a rag pipeline or whether it's the agentic memory.

So how do you make how do make vector search more approachable? How do you reduce that lower lower that learning curve so that more people can learn it more quickly and start to participate in this ecosystem?

We are see we are seeing this broader ecosystem of data that that wasn't indexed before because it didn't lend itself to a traditional lexical search. The the cool thing about the vector search is, I mean, vector search at its core is taking some piece of data and mapping it into in space, into some geographical geometric space. And then really all vector search is similarity.

Like where what are the closest vectors to this new thing that I'm searching on? And video is a great example. Audio is a great example.

Unstructured data, you just take a bunch of PDFs that you have sitting around in a SharePoint. I mean, those are good examples as well. So yeah, I think that there's there's some truth to that that we're it's not just how do I take the data and put it into a database, but now how do I find it?

And how do I find in a way that is fast, is scalable, that's a good quality? And that's why the way that we implemented vector search and being able to within the same platform have the combination of the pre filtering, the vector search, and the lexical search. We feel like that gives us advantage and gives our developer community more levers to pull from than what some of the alternatives are.

And like I said before, be because it's it stands on the shoulders of the core product with the with the Atlas version, you can already deploy that in any hyperstellar data center you want with good data resiliency, with good security, and if you need to to shard that data so that it doesn't meet particular geographies. We got all that for for essentially for free because of how we implemented vector search on top of the core product. And, yeah, we're we're seeing all kinds of different kinds of data get put into those documents in a way that we didn't before.

I think biggest thing if I turn question on its head just a little bit, there's a presumption in other countries that The US is ahead and doing things that other people are not. And I found the opposite to be true. So, I mean, I live and work in out of Cincinnati.

So US is one of those countries. I've spent some time in both Amsterdam and London earlier in the year, but I just did a tour that included stops in Toronto, Bengaluru, Mexico City, and Sao Paulo. And the two most sophisticated customers I talked to this year were in Mexico City and Sao Paulo.

And they assumed when we started their conversation that they had US competitors that were doing things that they weren't. And like I said, the opposite was true. I think we've reached a point with some of these technologies that, like, geographic barriers don't matter nearly as much as they did during like the web app era or or even during the cloud era.

Because for the cloud era, like if one of the hyper stealers didn't have a data center in your country yet, you were kinda out of luck. That's not true anymore. Like pretty much every country has at least one hyper stealer data center in it and by extension, access to models and vector databases and embeddings and re rankers.

Like access for these things is far better than what I've seen with previous technology revolutions that we've seen. So I think understand why there would be an assumption that US companies would be ahead since so many of the bigger AI companies are US based. But like I said, the two coolest things I've seen this year were in Mexico City and in Sao Paulo.

So I think that those geographic barriers to being ahead in the market are starting to disappear.

Well, I think it has more to do with the lack of barriers outside The US than it does with any of the behaviors of the companies within The US. It's just access is democratized now in a way that it wasn't that whether it wasn't for the cloud and for the mobile waves. Right?

Like, if you didn't have the radio towers yet, an iPhone's not gonna do you much good. But now we have all this infrastructure that's laid worldwide in a way that we didn't fifteen years ago that that makes access to these things far easier than they were before. So there's just there's just a wider set of people in a wider set of countries that have access now that they didn't before.

So I I don't think it's any it's not an indictment of the way that I think US companies are thinking about it. I just think that there that greater access erodes those geographic barriers that we've seen with previous technology waves.

Well, there's one technology thing that we have I didn't get a chance to talk about and that's shared embedding spaces. And then there's there's also this notion of just like kinda where we are on this journey together. The shared embedding spaces real quick is one of the other ease of use things that the voyage models provide that nobody else does is in January, we released four versions of the of the version four of the text model.

Everybody typically releases small, medium, and large and at those different price each one has a different price point and a different retrieval quality that comes around with that. But we also introduced something called the Nano which is open weight and free for anybody to download in off of Huggiespace and what shared embedding spaces does is each one of those four models shares an embedding space. Meaning that embeddings that are generated with one of those models, they're compatible with the other three And what they'd let you set up is you could take your corpus of data and you could embed it using say the large and developments, every other embedding model is gonna force you to pay tokens during your development cycle to hit against their embedding model.

So as you're query as you're taking your queries and you're you're passing them through the embedding model to do your searches. But with us, you can use the Nano there. You can run that on your laptop if you wanted to.

So you could if you wanted to have zero token costs aside from the embedding of the corpus of data, but for your queries, you could eliminate your your token costs during development in a way that wasn't possible before January. So that's not for everybody. You do take a little bit of a retrieval hit when you use a different retrieval quality hit when you use a different model, but it for some customers that don't wanna have to that wanna lower their token cost during development, this is one way that they can potentially do that.

So that's that's shared embedding spaces which we introduced in January. And more broadly, I wanna come back to the idea that we've been building databases for a really long time. We have not been building agents for a very long time.

We're we're gonna continue to see iterative improvement and it's gonna happen fast. Like, MC published MCP on like Monday before Thanksgiving two years ago and by March, all of their competitors had embraced it as a protocol. Token Maxine first got mentioned in like early March and by April, it was over as like a conversation piece.

Like, the quickness, the speed at which we have these conversations and we go through these cycles of these different features, it just it's faster than it ever has been before. So the fun part is there's still a lot we all have collectively to learn. We're gonna all collectively figure out how to make this easier for everybody and you you'll see more and more agents in everyday life as a result.