AI Agent Memory Architecture: Structured SQL, Semantic Vector Search & Memory Bank
From Google Cloud Tech · · 12 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
Code for this episode → https://g.dev/ai/fashion-mind
In this episode, you'll learn the critical difference between Structured Facts and Semantic Understanding, why they should never live in the same database, what to do with raw audio/video perception, and how to read and write memory asynchronously without blocking latency-sensitive voice streams.
Watch along and learn: * Short-Term (RAM) vs. Long-Term Agent Memory * Facts (Exact Lookup) vs. Understanding (Semantic Recall) * Why dumping exact facts into a vector store breaks your agent * Where to store Structured Memory (PostgreSQL, MySQL, Google Cloud SQL) vs. Semantic Memory (pgvector, AlloyDB, Vertex AI Agent Engine Memory Bank) * How to handle raw multimodal perception (camera frames & audio waveforms) * he Two-Track Read/Write Loop: Synchronous tool calls for facts vs. asynchronous off-the-hot-path extraction for understanding
Chapters: 00:00 - Intro: What should a multimodal agent remember? 00:25 - Demo: Personal styling agent with long-term memory 00:53 - Short-term (RAM) vs. Long-term memory 01:28 - The 2 types of memory: Facts vs. understanding 02:29 - Where memory lives: SQL databases vs. vector stores 03:40 - Vector store options & managed memory bank 04:30 - Common mistake: Putting everything in a vector store 05:24 - What about raw perception? (Audio & video streams) 06:40 - How agents read memory at session start 07:38 - Saving memory: Fast sync writes vs. async extraction 08:44 - Keeping memory extraction off the live hot path 10:42 - Architecture recap & next episode: Real time vision
💬 Question of the day: What is the first thing you would want your AI agent to remember about you? Drop your answer in the comments below!
🔔 Subscribe and stay tuned for the next episode, where we give the agent eyes with real-time vision!
Resources: Gemini Live API docs → https://g.dev/cloud/gemini-live Google Cloud SQL docs → https://g.dev/cloud/sql-docs AlloyDB docs → https://g.dev/cloud/alloydb-docs Vertex AI Agent Engine Memory Bank → https://g.dev/ai/scale-your-agents
Follow Annie online: LinkedIn → https://goo.gle/annie-linkedin X → https://goo.gle/annie-x
Watch more The Omni App → https://g.dev/cloud/modern-ai-app 🔔 Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
#AIAgents #MultimodalAI #SoftwareArchitecture #VectorDatabase #GoogleCloud #LLM #MachineLearning #AgenticAI #CloudSQL #VertexAI
Speaker: Annie Wang Products Mentioned: Google Cloud SQL, AlloyDB, Vertex AI Agent Engine Memory Bank, Gemini Live API