Local-first AI: what it means and when it is right
Local-first AI keeps your data, and as much processing as practical, on your own device, treating the cloud as optional rather than required. That buys speed, offline capability, and privacy by default, at the cost of device resources and harder multi-device sync. A hybrid design adds end-to-end encrypted sync so the cloud helps without reading anything.
Two architectures, two defaults
A cloud-first AI assistant stores your conversations and memory on the provider's servers; your device is a window onto data that lives elsewhere, readable by the provider unless it is end-to-end encrypted. A local-first AI flips the default: your data's home is your device, everything useful works there, and the network is an enhancement rather than a requirement. The term local-first comes from a 2019 research essay by Ink and Switch, which framed it around a simple test: you should keep working, at full speed, when the network goes away, and the primary copy of your data should be yours.
For a personal AI the stakes are higher than for a document editor, because the data involved is not a spreadsheet. It is what you said at midnight, what you are afraid of, and what you are planning. Where that lives by default is the single most consequential architectural decision a personal AI product makes.
The tradeoffs, honestly
| Dimension | Local-first | Cloud-first |
|---|---|---|
| Latency | Reading and writing your own data is instant; no round trip to a server for your interface state | Interactions can wait on the network; a slow connection makes the whole product slow |
| Privacy | Content stays home by default; there is no third party to trust for data that never leaves | The provider can read whatever it stores, unless the content is end-to-end encrypted |
| Offline | Whatever runs locally keeps working with no connection | Little or nothing works without connectivity |
| Multi-device | Hard: you must build sync and resolve conflicts between devices | Simple: one authoritative copy on the server, every device reads it |
| Durability | The device is a single point of failure unless you add sync or backups | The provider handles replication and backups for you |
| Model power | On-device models are limited by your hardware's memory and compute | Hosted models can be far larger than anything a phone or laptop runs |
Note what the table does not say: it does not say local models are bad or hosted models are dangerous. Both columns are legitimate engineering positions. The question is which costs you can live with for this particular kind of data.
Data locality and model locality are separate choices
A common confusion is treating local-first AI as identical to running a language model on your own machine. They are independent axes. You can run a local model over cloud-stored data, and you can keep every byte of stored data on your device while sending an individual question to a hosted model for a better answer. The second combination matters because it separates two very different exposures: a stored archive of your whole life, versus one question in flight. A message you send to a hosted model is plaintext to that model at request time, in any product; that is how asking works. Your accumulated history never needs to be exposed for that to happen.
When local-only is the right call
- The data is so sensitive that you want no third party involved at all, and you accept responsibility for your own backups.
- You work offline or on unreliable connections and need the assistant to function there.
- You only use one device, so sync buys you nothing.
- You have hardware capable of running a local model well, and its quality is enough for your use.
When local-only falls short
- You live across a phone, a laptop, and a browser, and re-creating context on each one defeats the point of a companion that knows you.
- Losing or breaking the device must not mean losing the relationship and its memory.
- You want answers from models larger than your hardware can run.
- You will not reliably maintain your own backups, and you know it.
The hybrid: local-first data with end-to-end encrypted sync
There is a third design that keeps the local-first default and fixes its two weakest points, sync and durability, without handing content to the provider. Data lives on the device. When sync is on, each record is encrypted on the device under a key derived from your passphrase, and the server stores only a nonce and ciphertext per record: an envelope it can hold, return, and merge by metadata, but never open. Your other devices derive the same key from the same passphrase and decrypt locally. The cloud provides availability and multi-device convergence while remaining unable to read a single record. Hosted inference, if you use it, stays a per-question choice rather than a bulk transfer of your history.
The hybrid has honest costs too. You must remember a passphrase, because a provider that cannot read your data also cannot recover it for you. The provider still sees metadata: account identity, sync timing, usage counts. And the client software has to actually do what it claims, which is why the claims should be specific and checkable rather than atmospheric.
How Violet applies this
Violet, currently pre-launch, is built local-first with the hybrid as an option. The companion app keeps its state on the device, and its Trust panel lists the actual stored keys, by name, read from real state when you open it. While cloud sync is off, the panel says plainly that nothing leaves the device, because nothing does. When sync is on, the server stores only sealed envelopes, using PBKDF2-derived keys and AES-GCM as described in the deep dive linked below. Voice input follows the same posture: the microphone is live only while you hold the button, with no wake word and no always-listening mode.
Questions
Is local-first AI the same as running a local model?
No. Local-first describes where your data lives: on your device, with the cloud optional. Running a model locally describes where inference happens. They combine freely: you can keep all stored data local while asking a hosted model individual questions, or run a local model over synced data. Conflating them hides the real decision, which is about your data's home.
Does adding encrypted cloud sync break the local-first promise?
Not for content, if the sync is end-to-end encrypted. The device remains the place where data lives in plaintext, and the server holds only sealed envelopes it cannot open. What changes is metadata: the provider now sees that an account syncs, when, and how much. An honest hybrid names that cost instead of pretending the cloud sees nothing at all.
Is a local-first AI slower because my computer is weaker than a data center?
For your stored data, it is faster: reading and writing local state involves no network at all. For model inference, it depends on the model. Small on-device models respond quickly but are less capable; large hosted models are more capable but add a network round trip and a third party. A hybrid lets you choose per question.