Learn · Intermediate

Encrypted inference: can a model answer a question it cannot read?

Encrypted inference is the attempt to let a cloud AI model answer your question without its operator ever seeing what you asked. Two very different technologies compete for that job. Hardware enclaves decrypt your data inside a sealed region of a chip that the operator is designed to be unable to inspect, and they run at close to normal speed today. Homomorphic encryption computes directly on encrypted data so nothing is ever decrypted, and it is mathematically stronger and still far too slow for real models.

Understanding which one a product uses is the difference between a strong privacy claim and a moderate one, and companies are not always precise about it.

The problem

Using a hosted model means shipping your input to somebody else's computer. For a question about pasta recipes, nobody cares. For a patient's chart, a client's contract, an unreleased earnings statement, or a company's private source code, that transfer is often flatly prohibited, and the standard reassurances do not help. "We encrypt data in transit and at rest" covers the network and the disk, but the model has to actually read the text to answer it, and at that moment the data sits in the operator's memory in plaintext.

Running the model locally solves this and creates a new problem: the best models are too large for consumer hardware, and keeping them updated on every user's machine is impractical. So the interesting question is whether a remote server can compute an answer it cannot read.

Approach one: sealed hardware

The shipping answer is confidential computing, sometimes called a trusted execution environment or enclave. Modern processors can carve out a protected region of memory and computation that even the machine's operating system and administrators cannot look into. Your device encrypts data to that specific enclave, the enclave decrypts and processes it, and the result comes back encrypted.

The piece that makes this more than a promise is remote attestation. Before sending anything, your device asks the enclave to prove, cryptographically, exactly what code it is running. If the hash does not match what you expect, you refuse to send. This means the operator cannot silently swap in a version that logs your prompts.

Google's Private AI Compute is the largest deployment of this idea for AI, running Gemini models inside what Google calls Titan Intelligence Enclaves on its own tensor processing chips, with named consumer workloads on the Pixel 10.

Think of it as a bank vault with a glass ceiling. Your valuables really are unwrapped inside, and the bank has built the room so that nobody, including its own staff, can reach in. You are genuinely protected, but you are protected by the room's construction and by the bank's honesty about having built it correctly. Enclaves have been broken before by side-channel attacks that infer secrets from timing, power draw and cache behavior without ever breaking in directly.

Approach two: encrypted math

Homomorphic encryption makes a much stronger promise. Certain encryption schemes let you perform arithmetic on ciphertexts such that decrypting the result gives the same answer as doing the arithmetic on the original values. Multiply two encrypted numbers, decrypt, and you get the product. The server computes a correct answer while remaining completely blind to the inputs, the intermediate values, and the output.

Craig Gentry's 2009 doctoral work showed for the first time that a scheme supporting arbitrarily many additions and multiplications was possible, which is what makes it "fully" homomorphic. The obstacle he had to overcome, and which still shapes the field, is noise. These schemes hide data by burying it in random noise, and every operation makes the noise grow. Past a threshold, decryption fails. Gentry's insight, bootstrapping, lets the system refresh a ciphertext to reduce its noise, at substantial cost.

Picture a locked glovebox. You seal your materials inside, hand it over, and the other person works through gloves built into the walls, never touching or seeing the contents. They hand it back and only you can open it. That is the guarantee, and it is why cryptographers find it so appealing: nothing about it depends on trusting the person holding the box.

Why it is not shipping

Speed. Homomorphic operations are orders of magnitude more expensive than plain arithmetic, and a language model performs an enormous number of them per token. Worse, these schemes are naturally good at addition and multiplication but awkward with everything else, and neural networks are full of everything else: comparisons, the nonlinear activation functions that make deep learning work, and the softmax at the heart of attention. Early work such as Microsoft's CryptoNets handled this by replacing activations with polynomial approximations, which works for small networks and degrades badly at scale.

The second barrier is that writing these programs requires deep cryptographic expertise. That is the gap Google's HEIR targets: a compiler where a developer annotates which values are secret and the toolchain lowers the program to backends like OpenFHE and Lattigo. HEIR's repository is explicit that it is not an officially supported Google product, which is a fair summary of where the whole field sits.

The number nobody publishes is the honest scoreboard: an end-to-end slowdown figure for homomorphic inference against plaintext on a realistic workload. Its absence is informative.

What to take away

When a company says your AI usage is private, ask which mechanism. If the answer involves enclaves, attestation or confidential computing, the guarantee rests on hardware and on the vendor's implementation, and it runs fast today. If the answer is homomorphic encryption, the guarantee rests on mathematics, and you should ask what it costs and on what size of model, because for full-scale language models the honest answer is currently that nobody ships it.

Both differ from differential privacy, which is a promise about what a trained model can leak about individuals in its training data, not about who can see your prompt. All three get marketed with the same word.

Ground Truth. (2026, August 14). Encrypted inference: can a model answer a question it cannot read?. Ground Truth. https://groundtruth.day/learn/encrypted-inference.html

@misc{groundtruth:encrypted-inference,
  title  = {Encrypted inference: can a model answer a question it cannot read?},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/learn/encrypted-inference.html}
}

Topics: privacy · encryption · security · infrastructure · fundamentals