Praveen explains why his team moved off GraphRAG in favor of a tree-structured memory for faster incremental updates and how they balance retrieval freshness, latency budgets, and access control at LinkedIn's scale.
Connect with Praveen on LinkedIn. Congrats to Populist badge winner Luka Ganić, who answered SVG image as button in Flutter so well it outscored the accepted answer!
TRANSCRIPT
Ryan Donovan (00:00) Hello, everyone, and welcome to the Stack Overflow Podcast, a place to talk all things software and technology. I'm your host, Ryan Donovan, and today we are talking about a very, very large agentic memory operation there they put together at LinkedIn. And my guest for that is Praveen Badagutla, who is a principal AI
He's a principal AI researcher over at LinkedIn. So welcome to the show, Praveen.
Praveen Bodigutla (00:35) Thank you, very nice to meet you, Ryan.
Ryan Donovan (00:37) Yeah. Glad you could be here. So before we get into the the topic today, tell us a little bit about how you got into software and technology.
Praveen Bodigutla (00:47) as I said, I'm a principal AI researcher at LinkedIn and I lead a few foundation efforts, including the memory agent as well as like being the founding engineer and led some of the AI product development, both for the enterprise as well as the consumer side. the journey to where I'm right now and how I landed here is pretty nonlinear. I actually
Ryan Donovan (01:08) Mm-hmm.
Praveen Bodigutla (01:09) started as a platform engineer at Yahoo a long time ago after.
Finishing my undergrad in like computer science and economics. Then I did like a couple of masters in financial mathematics as well as data sciences from Stanford and NYU, where I got the opportunity to work with some of the renowned professors such as like Andrew Ng, Kyun Kyoon Chou, and Jeffrey Ullman. In my past life, I was a quant developer at the Investment Bank in New York. And after that I joined Alexa, where I worked on dialogue model research.
So the central theme of the career has been innovating on the side of AI and building platforms
Ryan Donovan (01:51) Uhhuh.
Praveen Bodigutla (01:52) so that the innovation translates into useful products for the end users.
Ryan Donovan (01:58) Yeah. and I think, you know, we're we're talking about something today that that feels like a a big i innovation in a space that a lot of people are talking about, the agentic memory and context layers. now you all built a whole cognitive memory agent that that serves something. Can you give us a little little overview of what the project was? Praise.
Praveen Bodigutla (02:25)
Right,
so let's start with like why we actually build the cognitive memory agent.
Ryan Donovan (02:30) Yeah, that's a good start. Yeah.
Praveen Bodigutla (02:32) So LinkedIn has developed and successfully launched this hiring agent or hiring assistance for the recruiters to manage their hiring workflows. And what we observed was through these interactions that the recruiter had with these agents, they expressed some of their hiring preferences and also they refined the role that they're hiring for.
Not only do they mention like where they are hiring for and also what are the skills that they're interested in, they also give direct feedback on the candidates that were shown to them. So what we observed was there is stickiness to these preferences, which actually translate to how they define a role on similar titles and other roles that they're hiring for. So in order to provide this really true agentic experience for recruiters.
We wanted to provide this personalization layer, and that is the genesis of the cognitive memory agent, which gives this concept of state for our
Ryan Donovan (03:32) Mm-hmm.
Praveen Bodigutla (03:33)
for our agents, which the users, which in this case are recruiters who were interacting with that. And why a memory agent is because we wanted to not just fetch context, but also manage the entire life cycle of memory or the whole memory flywheel, starting from
Understanding what's ingested, what's retrieved,
Ryan Donovan (03:54) Mm-hmm.
Praveen Bodigutla (03:55) how it is contextually relevant, and how is it even organized and updated. So the memory agent basically manages this entire flywheel and provides this deep personalization for our end users. And that was the reason why we developed the memory agent.
Ryan Donovan (04:13) Yeah. And I I think it's it's interesting you mentioned state and it's not just a sort of session conversational state. It's not just a sort of preferences state. Like it's a a whole it's a three layer sort of burrito of state here. can you talk about the what those three layers are and and and why you needed three layers?
Praveen Bodigutla (04:37) Right, so the three layers or like the four layers or rather the memory that we use within our memory agent are one of them is the conversation memory, which is let's say you and I are interacting right now and we are expressing our preferences in terms of like different the tasks that we are trying to achieve. So that's most recent and that's up to date, and that's the information we have. Then we have on the other end our semantic memory layer, which is
the aggregated information of the user based on the interactions and preferences that they have expressed across sessions. And as you know, on LinkedIn we have different product offerings and different surfaces where the users interact with, and recruiters not only use the hiring assistant agent but also use a search platform to look for candidates. So they express those preferences there. So we want to make sure that.
The experience that the users have is we know what they're looking for and we are personalizing the whole experience for them. So the the semantic layer aggregates information not just across different interactions that the user has with the agent, but also they're based on their activities that they have on related product services as well. Now, in between, we also have the episodic store, which gives this temporal querying layer where
we can identify that hey what are the most recent relevant activities that this user performed. So it not only provides like the specificity of the signal and also provides provenance. Let's say we aggregate information, we can now trace it back to, what were the activities that went in that helped us to come to this conclusion that this is the preference of the end user. And the last but not least is the procedural memory layer. Every user, every recruiter if they're actually interacting with the
agent, even if they're hiring for similar roles, the way they interact, the trade-offs they make and the preferences they express are very different. Some of them might
Ryan Donovan (06:35) Mm-hmm.
Praveen Bodigutla (06:35) be actually have put more weight on the location and workplace type as opposed to others might be putting more information on seniority and specific aspects of the role. So the how of how the user actually achieves these tasks, in this case the hiring job, that is captured in the procedural memory. So
The memory is this layered cake of like conversational, episodic, procedural, and semantic memory which captures information at different granularity and specificity.
Ryan Donovan (07:09) Hmm. and I you know, there's the the sort of three end buckets for state, but or or four. but I the it all comes from basically a single data stream, right? Sort of the the discussions, interactions on the site. what's the challenge of extracting these separate sort of behavioral state markers?
Praveen Bodigutla (07:38) Right, so if you look at the stream of conversations, so there are two different sets of data sources. One, let's say the interaction that the recruiter or the user has with the agents. And if we talk specifically about that, your context can bloat up as you accumulate these interactions over a period of time. Like think about the hiring process involving multiple steps. First, you have to start with the job description, then refine the job description.
Then you are shown certain candidates and then you give certain feedback on that, and then you reach out to certain candidates and you prefer some and archive others. So all of this information gets accumulated. So on runtime, we want to make sure that this is compacted correctly. So we have this ingestion service which, as it's actually looking at this interaction in real-time and near real time, tries to consolidate these. In fixed workflow agents, it's kind of like relatively easier because you know the
boundaries of each and every step in the workflow. But as we are moving towards more deep agents and advanced interactions, the the steps can be interlinked. User can first, like the recruiter can first start with, let's say, calibrating a candidate and then go back and refine it and then go back and forth between these. So identifying the right session boundaries, identifying the right interaction subtopics, and then organizing the memory and making sure that it is
discoverable. So that stream is challenging in its own way because you have to make sure that you have compressed it, you have not lost information, and then you have accurately represented that information that is persisted. And last but not least we have to retrieve that information too. Now
Ryan Donovan (09:24) Right.
Praveen Bodigutla (09:24) as as that interaction is going on, I could change my preferences. I could say, you know what,
Ryan Donovan (09:29) Mm.
Praveen Bodigutla (09:29) I don't want to hire in this particular location. Maybe let's actually look for a different skill.
Maybe look for people with complementary skills. So when certain preferences are expressed during the course of conversation, then that brings this concept of like, hey, how do we make sure that the most fresh and recent information gets priority? And if there
Ryan Donovan (09:50) Yeah.
Praveen Bodigutla (09:50) are any conflicts while we are retrieving the memory and answering the queries that are that the user is posing via the application agent, their accurate, fresh.
recent and the conflicts are handled correctly and most importantly in low latency as well.
Ryan Donovan (10:07) Yeah. I imagine with with something like that, a recruiter could be hiring for multiple roles. is there a difficulty, complication with sort of like, you know, hiring in multiple with multiple preference sets?
Praveen Bodigutla (10:25) there is difficulty, but at the same time, there is this nice carryover effect of preferences from one similar set of roles that the recruiter is hiring for. now, when we talk about the challenge here is we want to first of all understand how is our data or this preference or memory in this case organized. Now, in the hiring assistant case, we we have a natural tree like structure.
For some of these preferences that are hired. For example, a recruiter has multiple projects that they're hiring for, and each of these projects can have their individual preferences. So that's the granular leaf level node. And then you can aggregate these preferences at the recruiter level. And then you have this cohort, like multiple recruiters. Let's say they belong to the same company, they're hiring for similar roles. They're sharing of information across these different recruiters belonging to the same cohort. So When we mine this information and organize it, which is aligned with the inherent structure that is present in the data and also in terms of the interactions, then it can actually become a superpower for you to like bootstrap some of these conversations for let's say new recruiters who are coming in, and now they don't have to start from scratch, they already have a blueprint of how their company is actually prioritizing in terms of the roles and the preferences that are expressed.
through their hiring process and workflows across the different members of the same cohort.
Ryan Donovan (11:55) Mm-hmm. that's a good segue. You you talked about the this being stored as sort of like a tree structure. how are the the other layers treated? Is there a sort of one size fits all storage structure or are they all kind of a different engineering problem?
Praveen Bodigutla (12:14) Right, so let's look at what is the basic construct that fits everyone because we're building a platform. We're not trying to redesign and reinvent the whole platform, memory platform for every different application. And what makes every application unique? So if the memory design itself, our memory agent is the land graph agent, and it uses certain tools to access these memory layers. Now these tools are very generic.
Like in the sense which we are actually reading and writing to episodic stores, and we are the signature or the APIs that are used for accessing these memory layers is is fixed. However, because we are using tools, you can override some of the descriptions and customize that for specific applications, preferences and logic and how they actually want to store the data. Now, so that is the memory orchestration layer in terms of how you plan, how you reason.
So that is standardized and it's a fixed like workflow in terms of how the memory is fetched and reasoned over before responding. Now, in terms of like the memory structure itself, now in the case of recruit recruiters, we had a clean, nice hierarchy, tree hierarchy of preferences for the recruiters based on the projects and also the cohort. However, the same may not work for a different application where they want to organize.
something more like as a graph. Let's say for example I'm connected to an individuals and it's not like a clear hierarchy. And then they design their own long term memory structure and preference structure and make it available. But now that you have actually exposed that structure using a standard interface or tool, now that becomes queryable. Now it's discoverable because we kind of know how to query it. Now the orchestration layer and the reasoning layer and as well as the synthesis layer remains the same.
But individual representation, let's say the long-term memory is by typically that changes. The episode boundaries could be different. For example, in certain cases, like in the recruiter case, it's single activity because it has high signal. We
Ryan Donovan (14:23) Mm-hmm.
Praveen Bodigutla (14:24) want to reduce the noise, we want to increase the signal to noise ratio. But in certain cases, that activity is spawned over multiple terms. So we that becomes the episode boundary and what gets persisted. and the conservation memory more or less remains
The same in terms of how it is structured because this it's the interaction that we are synthesizing. and the procedural memory is inferred memory. So some of these are mined structures and based on what's relevant for the application that are extracted. So it's a combination of how much customizability that you want to provide at the application level, but at the same time at a platform level, you're providing the basic tenets in terms of the whole design, which is very
stable and also generalizable and scales to different applications and solutions at the same time.
Ryan Donovan (15:13) Right. And and if I understand correctly you're saying all of the sort of all of the memory layers f first go through a sort of memory orchestration piece before it gets to the end application, is that right?
Praveen Bodigutla (15:24) Right. So we have like specifically an ingestion service, and you have the retrieval service and then the offline consolidation job, which consolidates all this information, removes duplicates, joins with external sources and addition, augments it with addition information, removes stale and like duplicated information and conflicting pieces of preferences. however, like the whole memory piece goes the managed memory piece goes through this ingestion.
organization and then the retrieval.
Ryan Donovan (15:57) so for for a long time, obviously the the way people got context and and referential information from for for AI and agents was the the RAG, the retrieval augmented generation or you know, some kind of vector database. This seems like it's it's doing a lot of, you know, pulling from multiple sources, consolidating, aggregating, determining what's what's good.
what's the the idea the how do you determine what is relevant both from each individual memory store and then from this sort of aggregated memory blob or whatever it is?
Praveen Bodigutla (16:42) Right. So I think when we talk about relevance, we are talking about two s separate aspects of relevance. What one is where do you sync all information from? How do you know this information is relevant that you need to ingest and ingest and like make it part of your memory? And the second one is how do you retrieve relevant information? How is that typically di different from the traditional RAG where
I extract everything and then just provide it as context for the application agent to manage. From the ingestion point of view, there is domain expertise that comes into picture because this application agent is tailored to solve certain tough. In this case, the recruiters are hiring and looking for candidates to fill their roles. Now, the points that we are looking at is what are the different signals that we need to consume where the recruiter has expressed preferences.
And what is the end goal? How do we measure that like we're actually providing the right memory abstraction and making sure that it's an efficient stack and also reduces the friction and makes them much more effective and productive when they're hiring? So that is one of the key principles that we use. Like where do I find what are the different product surfaces the recruiter interacting with? What are the kind of preferences that they're providing there?
And how do we actually synthesize that information? Now we want to make this transparent as well. We don't want to just like aggregate it because they might have expressed preferences and then they're like, I didn't actually say this. So
Ryan Donovan (18:17) Right.
Praveen Bodigutla (18:18) that's where the prominence space comes into picture, the traceability
Ryan Donovan (18:21) Mm.
Praveen Bodigutla (18:21) aspect. So as we are ingesting the specific granular information in our episodic store, we whatever is synthesized, whatever is like provided. This is your preference profile, this is your graph of.
Preferences as well as this your domain intelligence, personalized domain intelligence. Each of this is type citations, which is like actual records where you say this is what you've done and this is why we think this is your particular preference. And in the future, we're also working towards providing more control to the users when they can explicitly say that, hey, I no longer think that this is the right preference for me, like forget this. And also maybe remember more additional preferences than I actually came to know and I can just provide you the help.
they can add those key value pairs or information here. Now that's from the ingestion point of view. And we have evaluation scene as well like we have a three-tier evaluation. So when we persist information into memory, we make sure that the information is not lost. Like whatever entities are present your input information is actually preserved when it is persisted and has provens and citations accordingly.
So making sure that we have good quality evaluators for the memory that's persisted as well gives us confidence that this is actually the right information that we need to aggregate and accumulate and synthesize to help the recruiter in their tasks that they're trying to achieve. And and the transparency aspect of it also helps them see like how and why we are making certain decisions. Now, from the retrieval standpoint, it becomes interesting.
There are some standard patterns that the application agents use to query memory. For example, whenever a recruiter logs in, the first step, go fetch all the information. We are aware of that. And that we can pre-aggregate, create a cache, and make it available. But there are patterns where non-standard, which is more dynamic. For example, the recruiter is like, as I said earlier, will be moving from one aspect of
or the workflow to a different step and then interleaving, expressing preferences, overriding the previous ones. So from the retrieval standpoint, now based on the preferences that they have expressed, we have to extract the right information. For example, if they are like, hey, did I give similar activity similar feedback? And like were there candidates where I have actually given similar feedback, what was the preference that I was expressing in terms of skills in those aspects?
Those queries we cannot anticipate. But because we have this layered memory and also
Ryan Donovan (21:05) Yeah.
Praveen Bodigutla (21:06) way to like do some kind of like temporal query, EBR based retrieval. Now we know that in the workflow the recruiter is at a particular step. This is the context that they've provided. Based on the context, fetch the most relevant piece of information, reduce the noise and increase the signal, and give that back so that the application agent can make.
much more calculated decisions and provide much more valuable information to the recruiter, as opposed to just extracting all of their activity, which can cause a context bloat.
Ryan Donovan (21:38) Sure, yeah. do you do you do any sort of you know, like like double counting if if somebody expresses a preference multiple times, are you like, this is a much more reliable preference that they've extract s expressed? Or is it just like yeah, that matches what we have on file?
Praveen Bodigutla (21:57) So as of now, there is immense value that we observe even recalling that information, even with without internal prioritization of one preference and by having more value than the other one in terms of like the exact values of those preferences themselves. we have like a prioritization hue and conflict resolution policy to prioritize one layer of memory over the other.
And however, like what we are looking into is how does a weighting scheme like dynamically evolve based on their preferences that they have expressed multiple times. I mean, first of all, we don't want them to like express again and again the same preference there. That that means we have introduced friction. Like if we don't synthesize
Ryan Donovan (22:43) Right, right, right.
Praveen Bodigutla (22:47) this correctly and provide like incorrect information, they're like, no, no, not in that location.
Ryan Donovan (22:52) Right.
Praveen Bodigutla (22:52) so one of the downstream metrics, like
We call the tier three metrics, which measures this helpfulness and impact on the product, also measures friction. It's like, are we actually reducing the number of turns? Are we increasing the number of terms? so that's a trade-off we have to make. Like, how much of recall and without waiting and equally weighting the information is actually helpful over prioritizing and providing the additional layer of cognition and weighting, which might put more cognitive load on the
a member to also manage at the end of the day and like say that I think this is higher priority this is
Ryan Donovan (23:26) Mm-hmm. I mean the the the sort of layers and and you know you talked about extracts and transforms, it's it's starting to sound like a sort of an ETL pipeline for the agentic era. Is that is that accurate or is there is there extra complications here?
Praveen Bodigutla (23:48) I think the extra complication is what makes it an ETL for agentic error, where
Ryan Donovan (23:53) Ha ha ha.
Praveen Bodigutla (23:54) instead of like just storing everything, we have to be very parsimonious and make sure that like I said, like high we are capturing signal and we're not capturing noise.
Ryan Donovan (24:06) Mm.
Praveen Bodigutla (24:07) and that's what makes some of these aspects agentic, both from the ingestion as well as from the
retrieval standpoint, that we're capturing the right signal and organizing it in such a way so that it's easily discoverable, maintainable, provides all the governance as well as the access controls that are necessary, and then to provide safe retrieval and management of this memory so that there's no context pollution and also there's no accident leakage of information from different
Ryan Donovan (24:39) Right.
Praveen Bodigutla (24:40) places where it should not be leaked from. Yeah.
Ryan Donovan (24:44) Yeah.
Yeah. So that's a another excellent segue you just setting me up, Raveen. I love it. so the you know it LinkedIn is a is a pretty big scale. It's a lot of people and a lot of personal information. You talked about the the sort of access controls and safety and preventing leakages. what sort of things did you build in to make sure that the the data only goes to the the right places?
Praveen Bodigutla (25:11) Definitely. So there is multi-tenancy and isolation of data stores. every time when an application is actually if we set up the memory for an application, we want to make sure they have their own isolated data storage. There is no reusing or using like other applications data store. So there is clear isolation. And the other thing that we make sure is that the user who's signed in, who identify the user based on this.
Signing credentials are accessing the memory that they should be accessing and not beyond that. And when we purchase this memory in this hierarchical structure that I was talking about, we also tag it with the owners who have access to it. So that we we we are also like in the every step of the workflow, make sure that the right credentials are passed through so that we are accessing the right information and also it's restricted to the information that this user has access to as well.
Ryan Donovan (26:07) So it it there's maybe not a a whole, you know, memory system for each user, but it sounds like it's close, right?
Praveen Bodigutla (26:17) It's kind of provides this nice personalized memory for each user, but at the same time also helps them share with their cohort when it's allowed, and create this domain intelligence, which is specific personalized domain intelligence for that cohort. So you can
Ryan Donovan (26:36) Mm-hmm.
Praveen Bodigutla (26:37) manage both in the in the setting that we have with this whole nice, beautiful tree like structure.
Ryan Donovan (26:43) Yeah. I wanna sort of talk about the, you know, the kind of engineering trade offs here because it seems like this is a could be a lot of extra data to to store and retrieve and to to manage. are you have you been thinking about sort of like optimization strategies, how to, you know, prune data that isn't needed anymore? how to scale, you know.
reduced amount of contact spend, all that sort of good stuff.
Praveen Bodigutla (27:14) Absolutely. to be honest, like when we started with some of the popular technologies then to even create memory, the long-term memory that I was talking about and index it, we were using GraphRag. And what we realized was it's not it's slow and it's also not cost efficient because it invokes
Ryan Donovan (27:34) Yeah.
Praveen Bodigutla (27:34) a lot of LLM calls to identify the linkages between different nodes and every time to
rebuild this index and recreate this memory with not allowing us to scale for the scale at which we're had handling at LinkedIn. so this tree like hierarchical memory organization helped us provide this incremental update optimization where we know which leaf node, which branch and what is a particular node that needs to be updated. And we just percolate that preference along that
branch of the tree so that you're not recomputing and re-indexing the entire memory at the same time. And as you rightly pointed out, freshness and consistency are very important for us. So how we handle that is some of that I already touched upon is like when your retrieval layer is intelligent, it knows that a this is memory was updated at a particular point of time and this is the priority queue or the priority order in which I resolve
Some of the conflicts that are inherent in the data that was persisted, and use that intelligence to make sure that the right and resent and fresh memory is provided to the end user as well as the application agent, which is interfacing with the user. From the ingestion side, we touched upon that earlier when we talked about the memory compaction, right? We were like looking
Ryan Donovan (29:03) Yeah.
Praveen Bodigutla (29:03) at these the queue of
incoming terms and when we are extracting these activities and if we are making decisions based on the most recent activities of the user, you have that latest information there. And sometimes using simple urge policies such as if it is longer than certain time horizon like six months or one year it's still because you're already
Ryan Donovan (29:30) Mm-hmm.
Praveen Bodigutla (29:31) synthesizing this information into your long-term memory. It's available. So using those policies also help in making sure that
the memory is fresh and it's
Ryan Donovan (29:40) Mm-hmm.
Praveen Bodigutla (29:40) representative and it's relevant to what the user is doing right now. And and from the consistency point of view, I think we touched upon that when we talked about syncing data from different sources, identifying those relevant data sources and keeping that as part of your long-term memory.
Ryan Donovan (29:57) Yeah. y you mentioned the sort of move away from from graph rag to a more tree structure, to save on the L L calls. Are there other bits of engineering that you've done to sort of reduce the amount of AI here to to increase the determinism to save on costs?
Praveen Bodigutla (30:17) Absolutely. So our retrieval layer or orchestration layer in itself is a planner synthesizer LLM layer, which basically reasons over the memory when once it's retrieved to answer these complex queries such as or insights and examples such as like what are the trade-offs that I'm making. So you have to fetch all the relevant information into that. What we noticed was flattening the planning layer.
instead of doing sequential planning, but doing one-step parallel planning, which of identifying the right set of memory tools to invoke for solving this particular query for this user based on historical information about like where do we persist and like what is the what is the most active surface for these different users. And then applying selective LLM calls for response synthesis.
If the query is just fetching records, the query can be answered just based on information that's already in the conversation. We don't have to again invoke a complex reasoning process to reason through all the memory. So carefully choosing this orchestration, building in a way so that like there is less time spent planning and what the information is actually retrieved correctly and with low latency. To the application agent. So bear in mind the latency budget we have as a memory agent is probably 10 to 20% of the entire response latency budget that the application has, because memory is just one part of the context. And the
Ryan Donovan (31:59) Right.
Praveen Bodigutla (32:00) application has to fetch information from different sources to synthesize this response and present it to the user because their goal is to solve the task.
Ryan Donovan (32:09) Mm-hmm.
Praveen Bodigutla (32:10) so we have to be extremely careful about how we design this stack so that it's providing the right contextually relevant memory but at low latency. And some of the optimization inference optimizations at the VLLM serving engine level, such as using prefix caches, chunk prefills, all all of these go help a lot in reducing the overall latency.
And last but
Ryan Donovan (32:35) Mm-hmm.
Praveen Bodigutla (32:35) not least, being like very opinionated about some of the structures that you use in terms of how you respond. Like LLMs,
Ryan Donovan (32:41) Yeah.
Praveen Bodigutla (32:42)
if you actually let it just respond with any number of arbitrary reasoning tokens, then it can go on and can increase the latency. So
Ryan Donovan (32:48)
Right. Right.
Praveen Bodigutla (32:51) providing that clear API where you have a structured output which follows a structured format. Also guiding the LLM to actually generate in that format and limit the number of tokens.
that also helps in reducing the overall latency.
Ryan Donovan (33:06) Mm-hmm. we are at thirty three minutes here. I wanna give you a chance if there's something we haven't talked about that you wanna talk about, we can cover that. Otherwise I can ask a sort of future looking question.
Praveen Bodigutla (33:19) So I mean I think the the future looking question or the direction is definitely interesting. so we Yeah.
Ryan Donovan (33:25) Okay. Let me let me let me let me set set you up for that. Yeah, yeah.
S so you you mentioned early a little bit about the sort of problems you're looking at solving in the future. What what are the the other you know, what are the big things on your mind to to solve next?
Praveen Bodigutla (33:43) Right, so we are actually investing or exploring different options from even the memory persistence layer and
Ryan Donovan (33:53) Mm-hmm.
Praveen Bodigutla (33:53) the abstractions that we want to provide to memory to access memory. And why that is important is because as the LLMs get powerful, we are we are going into this era where L I mean LLMs are getting incre increasingly powerful and they
Ryan Donovan (34:07) Mm-hmm.
Praveen Bodigutla (34:08) have they can manage the
Memory as well. But at the same time, you you want to provide the entire memory lifecycle using the memory agent. So given this dynamic nature of interactions that these LLMs now or deep agents allow the users to have, how does the write abstraction for memory beyond just like the tools for each and every layer, so that the memory is easily discoverable and you can
Provide additional layer layers of optimization by changing the underlying storage layer to maybe virtual file systems. So that is one of the active areas of investments that we are currently looking at. And also the evaluation, one of the hardest problems that we have today is attribution. Like and as I said, like application agents not only fetch memory, but they're fetching information from different sources of information. So
Investing in robust evaluation and representative evaluation strategies to capture changes in the interaction patterns. so that is another area of like active investment that we're looking at and and further optimizations from the level of like dynamically identifying these session boundaries, compacting them, and optimizing the whole end-to-end as opposed to like optimizing individual layers separately.
So these are all excit exciting areas of applied research and innovation that we're really looking forward to.
Ryan Donovan (35:47) Cool. all right. well, I'm gonna take us to the outro. I'm gonna shout out somebody who earned a badge on Stack Overflow, and then I will say my name, my title, call to action, and then where I can be found on the internet, and you can do the same. Okay. Well, it is that time of the show where we shout out somebody who came on to Stack Overflow, dropped some knowledge, shared some curiosity, and earned themselves a badge. Today we're shouting out a populist badge winner.
Somebody who dropped an answer that was so good it outscored the accepted answer. So congrats to Luca Ganich for answering SVG image as button in Flutter. If you're curious about that, we'll have the answer for you in the show notes. My name is Ryan Donvin. I edit the blog, host the podcast here at Stack Overflow. If you have questions, concerns, topics to cover, if you want to say hi, email me at podcast at stackoverflow.com.
And if you want to reach out to me directly, you can find me on LinkedIn.
And probably.
Praveen Bodigutla (36:53) Well, congratulations as well to Luca
Ryan Donovan (36:55) Mm-hmm.
Praveen Bodigutla (36:56)
for providing really valuable insights and input. it was a pleasure talking to you, Ryan. and this was wonderful experience sharing the technology that we have built to solve some of the memory problems for different agents. And as you rightly identified, that this is this might be the new ETL or the second brain that we are building to solve some of these
Ryan Donovan (37:19)
Yeah.
Praveen Bodigutla (37:19) agentic interactions and
really looking forward to see how the technology pans out. This is just the beginning and there is a lot more research that we are seeing is coming our way. really happy to be sharing what we have built so far and thank you for giving the opportunity and you can definitely reach me out on LinkedIn.
Ryan Donovan (37:37) Okay. And what can they learn more about the project?
Praveen Bodigutla (37:42) so some of the like we have papers that we have one of the papers was actually accepted at KDD. So I would encourage checking out the publications as well as the blog more blog posts that are coming that will come through as we develop and embark on this journey as as we explore and try to incorporate as well as develop and build the state of the art solutions for handling memory for personalization.
Ryan Donovan (38:07) Wonderful. Well, thank you for listening, everyone, and we'll talk to you next time.