AICO: How an AI remembers you

Why memory matters for a companion

If an AI is supposed to be a **companion**, not a one‑off tool, memory stops being a feature and becomes the substrate of the relationship.

AICO is designed to:

• pick up threads from previous weeks and months,  

• understand how people, projects and events relate,  

• and adapt its behaviour over time.

This requires more than "store all messages in a database" or "throw everything into a vector store". The memory stack is explicitly grounded in ideas from cognitive science, in particular Complementary Learning Systems (CLS) and multi‑store memory models (working vs long‑term memory, semantic vs episodic).

Design principles

Three principles shaped AICO’s memory system:

• Separation of timescales – fast, fragile learning and slow, stable learning live in different subsystems.

• Separation of forms – some knowledge is best stored as text, some as vectors, some as a graph.

• Adaptivity over time – the system should change *what it remembers and how it retrieves* based on the person using it.

This leads to a layered architecture.

Scientific background

The design leans on a few well-established ideas from cognitive science and computational neuroscience:

• Working vs. long-term memory – short-lived, capacity-limited buffers for active processing vs. more durable storage.  

• Semantic vs. episodic memory – general knowledge about the world vs. memory of particular events and episodes.  

• Cognitive maps – internal graph-like representations of environments, people and relationships.  

• Complementary Learning Systems (CLS) – the hypothesis that fast, pattern‑separating learning in the hippocampus works together with slower, more statistical learning in cortical systems.

CLS in particular argues that no single memory mechanism can simultaneously be fast, stable and generalising. Instead, you need:

• a system that can very quickly encode new, possibly one‑off experiences without overwriting older ones, and  

• a system that integrates over many such experiences to build robust structure and abstractions.

In AICO’s implementation:

• Working memory plays the role of a fast, transient buffer, closer to hippocampal encoding.  

• Semantic memory approximates a cortical long‑term store for facts, themes and skills.  

• The knowledge graph functions as a cognitive map of a person’s world: people, entities, projects, events and their relations.  

• The Adaptive Memory System (AMS) orchestrates consolidation and forgetting, deciding what should move from “fast” to “stable” stores and how retrieval strategies should adapt.

The goal is not biological fidelity, but a memory system whose dynamics roughly match the constraints that make long‑term human learning possible: multiple stores, different stability regimes, and explicit mechanisms for consolidation and pruning.

Layer 1 – Working memory (LMDB): fast, contextual, disposable

Working memory holds what is **immediately relevant** to the ongoing interaction:

• recent turns in the conversation,  

• short‑term flags (current task, mode, focus),  

• transient facts that don’t need to be kept forever.

Implementation details:

• Stored in LMDB for fast key‑value access on local disk.  

• Time‑boxed: items have explicit horizons or decay functions.  

• Indexed by conversation ID and time, not just by user ID.

Analogy:

• Functionally similar to the hippocampal role in CLS: rapid encoding of new experiences, later used to teach slower systems.

Why it matters:

• Keeps context windows tight and relevant.  

• Allows experimentation with different conversation strategies without polluting long‑term memory.

Layer 2 – Semantic memory (ChromaDB + BM25): knowledge over time

Semantic memory is where longer‑lived knowledge goes:

• notes you write with AICO,  

• recurring topics and ideas,  

• distilled summaries of conversations or documents.

Technically, it combines:

• ChromaDB as a vector store (embeddings over chunks of text), and  

• BM25 / keyword statistics for sparse retrieval.

Retrieval uses RRF‑style hybrid ranking: both semantic similarity and keyword overlap matter. This makes it robust to:

• vocabulary drift (you change how you describe something),  

• and rare but important keywords (names, IDs).

Why it matters:

• AICO can pull in relevant material from your history for a new question, even if you phrase it differently.  

• Knowledge survives well beyond a single conversation or device.

Layer 3 – Knowledge Graph (libSQL + graph engine): a map of your world

Some information is relational by nature:

• who knows whom,  

• which projects involve which organisations,  

• which events followed from which decisions.

For this, AICO maintains a property graph:

• Nodes: people, organisations, projects, documents, events, places, abstract concepts.  

• Edges: relationships with type and temporal attributes (e.g. *works_on*, *introduced_by*, *followed_by*).  

• Backed by libSQL (for persistence and indexing) plus a light graph query layer (GrandCypher‑style).

Usage patterns:

• Build and update the graph incrementally from conversations, notes and imported data.  

• Query paths ("how is Alex connected to this project?"), neighbourhoods ("who is involved with X?"), and temporal slices ("what changed since our last planning session?").

Why it matters:

• Gives AICO a structural model of your world, not just bags of text.  

• Supports explanations ("I’m bringing this up because it connects to…") and better disambiguation ("this ‘Alex’ is your colleague, not your cousin").

Adaptive Memory System (AMS): coordinating the layers

The three layers above are the substrate. The Adaptive Memory System (AMS) decides:

• what gets promoted from working memory into semantic memory,  

• which entities and edges to add or update in the graph,  

• which retrieval strategies to favour for a given user.

Conceptually, AMS has three roles:

1. Consolidation scheduler

• Runs background jobs that scan recent interactions.  

• Identifies candidates for summarisation or promotion (e.g. recurring topics, explicit "this is important" cues).  

• Writes condensed summaries to semantic memory and updates graph edges.

2. Behavioural learner

• Tracks how you respond to different retrieval strategies:  

  • Did you use the suggested note?  

  • Did you accept or correct an inferred relationship?  

• Uses bandit‑style selection to prefer what works better for you over time (e.g. more graph‑based recall vs more note‑based suggestions).

3. Policy engine for forgetting

• Applies retention policies: not everything should live forever.  

• Allows different regimes (e.g. "private journal", "work assistant", "team space") with different defaults for what is archived vs. pruned.

The goal is not perfect biological fidelity, but a pragmatic translation of CLS into an operable system: multiple stores, different stability, feedback‑driven consolidation.

Memory Album: explicit, human‑driven memory

Not all important things can be inferred from patterns. AICO therefore exposes a Memory Album:

• You (or, in a team setting, selected members) can mark conversations, messages or events as album entries.  

• These entries carry:  

  • textual content and summaries,  

  • tags and references to people/projects,  

  • an emotional tone or significance label.

• Technically, album entries:

  • live alongside the semantic and graph layers,  

  • are preferentially retrieved for reflective prompts ("what were my big wins this month?"),  

  • and are never deleted automatically unless you explicitly choose so.

This gives you a manual override over what "matters", instead of leaving everything to implicit signals.

How memory connects to emotion and agency

Memory in AICO is not purely cognitive. It is designed to feed:

• the emotion system (inspired by C‑CPM style appraisal chains: relevance → implication → coping → normative), and  

• the emerging agency framework (longer‑horizon tasks, commitments, and follow‑ups).

Examples:

• A stressful episode stored as an album entry with high emotional weight can change how future, similar situations are framed.  

• Knowledge graph edges between people and recurring problems can alter which coping strategies are proposed ("loop in this person", "set a boundary", etc.).

The architecture in the first article (message bus, domains, modules) ensures these systems remain separate but connected: emotion reads from memory; agency schedules actions; memory in turn is shaped by what you and AICO actually do together.

Local-first and long-lived by design

Because AICO is local‑first, all of this runs on:

• your laptop,  

• your home lab,  

• or your own cloud tenant.

The memory stack is:

• encrypted (libSQL/LMDB with key management),  

• observable (CLI tooling for inspecting LMDB, ChromaDB, KG),  

• and maintained (scheduler tasks for consolidation, compaction, health checks).

The result is a system you can, in principle, run for years, with:

• a growing, structured history,  

• a companion that gets better at remembering and supporting you,  

• and an architectural path to keep evolving models and algorithms without discarding what you have already built together.

If AICO is going to be more than a temporary app, its memory had to be designed that way from the start.