How AI assistants remember: memory, context, and provenance
AI assistants remember in two layers: a context window that holds the current conversation and resets when it ends, and a durable memory store that saves facts outside the model and retrieves them later. A trustworthy memory store records provenance for every entry, so the assistant can show where a memory came from instead of presenting a guess as a fact.
Two kinds of remembering
When an assistant seems to remember, one of two different mechanisms is at work. The first is the context window: the bounded amount of text a language model can attend to in a single exchange. Everything in the window is visible to the model at once; everything outside it does not exist for the model. The second is a durable memory store: a database of saved records that lives outside the model entirely. The assistant writes facts into it during conversations and reads relevant ones back later, inserting them into the context window of a future exchange. The model never learns your life; the system around it files and retrieves your life. Understanding this split explains most memory behavior you will see, including the failures.
The context window is not memory
The context window behaves like working memory. It is real while the conversation lasts and gone when it ends. It is also finite: when a conversation outgrows the window, something must be dropped or summarized, and summaries lose detail without announcing which detail they lost. A larger window postpones this problem; it does not remove it, and it does nothing across sessions. An assistant that knows your name tomorrow is not using a bigger window. It wrote your name down somewhere. The interesting questions are all about that somewhere: what gets written, who approved the write, where it is stored, and what it looks like when it comes back.
Durable memory stores
A durable memory store saves discrete records: facts about you, preferences, events, commitments. At retrieval time the system ranks stored records against the current conversation and injects the most relevant few into the model's context. Done well, this is what makes an assistant feel like a continuing presence instead of a stranger with your file. Done carelessly, it is a machine for confidently misremembering: wrong facts persist, stale facts outlive their truth, and retrieved text arrives in the model's context indistinguishable from things you actually said today. The design decisions that separate the two are provenance, decay, and conflict handling.
Provenance: where a memory comes from
Provenance is the recorded origin of a memory: the source it came from, when it was captured, and its epistemic standing, meaning whether it was stated by you, inferred from patterns, assumed as a default, or has since been contradicted. Provenance is what lets an assistant say 'you told me this on Tuesday' truthfully, and it is what lets you audit and correct the store. Without it, memory becomes a fabrication engine with extra steps. A language model completes text plausibly by design; hand it an unsourced store and it will present retrieved guesses, stale entries, and its own confabulations in the same confident voice, because nothing in the system marks the difference. The failure is structural, not a matter of model quality. The rule that prevents it is simple to state: never assert a memory you cannot source.
Decay and conflicts
Lives change, so stores must age. The blunt approach, deleting old records, destroys history you may want back. The better approach is decay of accessibility: stale, rarely useful memories rank lower at retrieval time and stop surfacing, while the underlying record survives until you delete it. Conflict is the other constant. You said you prefer mornings; months later you say you prefer evenings. A system that silently keeps one version is guessing about your life. The honest design flags both records as conflicting and asks you to resolve them, then records how the conflict was resolved. Both mechanisms serve the same principle: the store may reorder what it volunteers, but it does not quietly rewrite what it holds.
Syncing memory across devices with end-to-end encryption
A memory store is the most sensitive dataset an assistant accumulates, which makes its sync path the most important one to get right. Under end-to-end encryption, each record is sealed on your device into an envelope of nonce plus ciphertext under a key derived from your passphrase, and the server stores that envelope alongside a small set of ordering fields: a version number, a timestamp, and a deletion flag. When two devices edit the same record, the server picks the winner purely from those fields, last writer wins, without ever opening the envelope. Deletions travel the same way, as tombstone markers with the envelope removed. The result is that your memories follow you across devices while the machine in the middle handles only sealed containers it cannot read.
How Violet's memory model applies this
Violet is a personal AI partner, currently pre-launch, and its published memory model is built around these rules. Every memory record carries provenance fields distinguishing known, inferred, assumed, outdated, unverified, and contradicted claims, and a hard truthfulness rule forbids saying 'you told me' unless a source can be cited. Memory capture is consent-shaped: candidates start as suggestions, and sensitive information is not stored long term without explicit confirmation. Retrieval is relevance-gated and decay lowers accessibility without deleting source records; conflicting memories are flagged for the owner to resolve rather than silently merged. The owner can review, edit, export, and delete all of it. And when records sync through Violet's cloud, they travel as sealed envelopes under a device-derived key, merged by version metadata the server can read around an envelope it cannot open.
Common mistakes when evaluating memory features
- Assuming a big context window replaces durable memory. It cannot persist anything across sessions.
- Assuming the model was trained on your data because it remembers you. Assistant memory is retrieval, not training, in these systems.
- Accepting memories with no visible source. If the assistant cannot show where a memory came from, you cannot correct it and the assistant cannot stand behind it.
- Ignoring the write path. A store you cannot review, edit, or delete from will eventually hold something wrong about you permanently.
- Ignoring the sync path. Memory synced in provider-readable form hands your accumulated life to whoever holds the server keys.
Questions
Does a bigger context window remove the need for a memory store?
No. A context window, at any size, holds only the current exchange and resets when it ends. Anything an assistant knows about you tomorrow was written to a store outside the model and retrieved back. Larger windows reduce how often detail is dropped mid-conversation; they do not create persistence.
Can an assistant remember something I never told it?
Yes, in the sense that systems can store inferences drawn from patterns in what you did say. That is exactly why provenance matters: a well-designed store labels such entries as inferred rather than stated, so the assistant does not later present its own inference back to you as your words.
Is Violet's memory available to use today?
No. Violet is pre-launch: the apps are not in stores and there is a waitlist on the home page. The memory rules described here, provenance labels, consent-gated capture, decay without deletion, and owner controls, come from its published design documents and code, not from a live service.