Distinguished Engineer and AI-assisted delivery expert at Thoughtworks.
And then talk about memory banks. Yeah, I recognize that from work where "AI has taken off" as well.
Guess what: As memory banks grow or accumulate the AI gets confused and doesn't quite deliver.
So far, a human that actually knows their product still prevails and is necessary to actually guide any AI effort. AIs have been trying to bullshit me so much it's not even funny any longer. Of course they all apologize and figure out reality when I guide them but that doesn't change the facts. And I simply can't read all the documents the AIs write for themselves to correct all of them and even if I did I wouldn't be sure enough that they'd improve significantly enough for me to try and spend this mind bogglingly boring amount of time to help this thing that's supposed to take my job ....
Memory is a bad idea right now, because it requires well tuned retrieval to deliver value, and one sized retrieval systems don't work, full stop. Most memory systems are designed around a homogenous chat paradigm and produce negative results in heterogenous chat environments or non-chat based agentic workflows.
The right way to do "memory" is to feed it to a "metacognition/default mode" network that builds a theory of mind / task ideation structure async from the main agent, then injects context relevant steering into the agent for each prompt based on this metamodel. So, "agentic memory" basically.
Guess what: As memory banks grow or accumulate the AI gets confused and doesn't quite deliver.
So far, a human that actually knows their product still prevails and is necessary to actually guide any AI effort. AIs have been trying to bullshit me so much it's not even funny any longer. Of course they all apologize and figure out reality when I guide them but that doesn't change the facts. And I simply can't read all the documents the AIs write for themselves to correct all of them and even if I did I wouldn't be sure enough that they'd improve significantly enough for me to try and spend this mind bogglingly boring amount of time to help this thing that's supposed to take my job ....