From private computation to private memory

Google has described how its Private AI Compute platform could remember information across sessions and devices without leaving readable records in ordinary cloud storage. The existing system was designed for stateless work: process a request inside protected hardware, return the result and wipe the temporary state. Persistent assistants need something different.

The new design gives each user a hardware-isolated memory database. Records are encrypted with a per-user data key, and that key is wrapped by a second key derived from a secret held on the user's device. The encrypted database can be stored and backed up by Google's surrounding infrastructure, but Google says it can be decrypted only inside an approved confidential environment during an authenticated request.

What the enclave does

When an assistant needs context, the device establishes an encrypted channel to a confidential server and checks evidence about the software it is running. Inside that protected boundary, the service can unwrap the data key, search or update the memory and run the model. Temporary prompts, tokens and activations are meant to be erased after the response. The long-term memory is re-encrypted and retained by design.

This is not the same as keeping everything on a phone. It moves trust into a chain of hardware isolation, attestation, key management, reproducible builds and a public record of approved software. The architecture can reduce who is able to see plaintext. It cannot make the surrounding service disappear.

The audit found safeguards and limits

Google commissioned Trail of Bits to threat-model and review the secure-memory code. Four consultants spent five engineer-weeks on the assessment, with access to both public and internal components. The final report says the reviewed code contained no mechanism for a Google employee acting alone to read user data. It also found ten security issues across cryptography, data exposure, validation, logging and memory safety.

Eight issues were resolved before the final report. Two remained open. One means deletion is not backed by a cryptographic freshness guarantee, so an old encrypted memory database could theoretically be restored by the storage layer. The other sends truncated hashes derived from model responses outside the trusted boundary for recitation checks. Trail of Bits rated that second issue high severity, while noting that practical exploitation would require privileged internal access and be technically difficult. Google says raw prompts and plaintext outputs do not leave the boundary and is adding an in-enclave filter to reduce the exposed hashes.

Privacy claims need continuing evidence

Persistent state introduces a basic trade-off. The out-of-enclave service needs a stable user identifier to find the right encrypted records. Google's own technical brief therefore says the memory system cannot make the same network-level non-targetability claim as a stateless, anonymously routed request. The protection shifts to keeping stored content opaque without the user's keys and restricting plaintext work to approved enclaves.

The audit is useful evidence, but it is a point-in-time review, not a permanent guarantee. Trail of Bits says critical parts of the system remain closed source and some infrastructure was outside its scope. Google's brief lists client-side attestation checks, independently witnessed transparency logs and broader reproducible-build coverage as next steps. Those changes would let outsiders verify more of the chain directly. Until then, private AI memory should be judged as a layered engineering claim with public evidence and visible gaps, not as a simple promise that the cloud cannot see.

Sources

  1. Google DeepMind: Advancing Private AI Compute with secure, server-side memoryPrimary September 23 announcement describing the intended memory capability, device-held secrets and company privacy claims.
  2. Google: Private AI Compute Technical BriefPrimary updated 15-page architecture brief covering data flow, key handling, threat boundaries, lifecycle, external verifiability and future work.
  3. Trail of Bits: Google PAIC Secure Server-Side Memory Security AssessmentCommissioned independent code and threat-model review with scope limits, ten findings, fix status and recommendations.
  4. Project Oak: private memory implementationGoogle's open-source implementation of the memory server and runtime components referenced by the technical brief and audit.