Residency says where your data sits. Confidential computing says who can read it while the model runs. Inference on a rented or colocated GPU in a UAE data centre gives you the first and not the second. Here is what a GPU TEE protects, what it costs, and what to demand from a vendor.
If a vendor tells you "your data stays in-country", they are answering a question about geography. They are not answering the question your risk register actually asks: who can see the data while it is being used? Confidential computing for AI inference in the UAE is the answer to that second question. It runs the model inside a hardware trusted execution environment (TEE) on the GPU itself, so the data centre operator, their engineers and anyone with hypervisor or management-plane access cannot read your prompts, your retrieval context or your model weights while they are in use. Between the prompt arriving and the answer leaving, that data otherwise sits in plaintext in server RAM and GPU memory. On a rented or colocated GPU in a UAE data centre, it is all in principle readable by the host.
The GPU TEE is the technology that closes this gap, and it is now practical: NVIDIA has shipped it since the H100, the tooling is public, and the measured overhead for inference sits in the low single digits on a well-configured stack. This article sets out what a GPU TEE protects, the stack to demand from a vendor, the performance numbers to budget for, and the answers that should already be in your procurement and due-diligence packs.
Residency answers where. Confidentiality answers who.
Data residency is about where data is stored and processed. Confidentiality is about which people and processes can observe data in use. They are different controls, and in the UAE the second is now the harder of the two.
The regulatory backdrop pushes regulated organisations towards in-country processing but says much less about in-use confidentiality:
None of this prevents the operator of your in-country GPU from reading your prompts. An AI DPIA or a Central Bank outsourcing assessment that stops at "the data centre is in Dubai" has answered the easy half of the question. When an auditor or counterparty asks "who inside the hosting arrangement can access this data, and how do you know?", residency alone has no answer. That is the gap a TEE fills.
The NVIDIA H100 was the first GPU with hardware confidential computing, and the feature now spans the Hopper and Blackwell generations. When the GPU boots in CC-On mode, a chain of trust is established: the GPU firmware performs a secure, measured boot from an on-die hardware root of trust; the driver running inside a CPU confidential VM establishes an SPDM session with the GPU; and the GPU produces a cryptographically signed attestation report that you verify before trusting it. All data crossing the PCIe bus between the CPU TEE and the GPU passes through encrypted bounce buffers, and command buffers and kernels are encrypted and signed in transit. CUDA applications run unchanged, as NVIDIA documents in its H100 confidential computing technical blog.
Two details decide whether the control is real:
The CPU TEE is not optional. A GPU in CC mode is only meaningful when the driver runs inside a CPU confidential VM: Intel TDX (Emerald Rapids or Granite Rapids) or AMD SEV-SNP (Milan, Genoa or Turin). The NVIDIA Confidential Computing Deployment Guide (v7.1, April 2026) lists the supported combinations, including Blackwell single and multi-GPU modes and Hopper multi-GPU Protected PCIe mode. A vendor offering "GPU confidential computing" without a CPU TEE is offering the shell without the contents: prompts are still plaintext in host memory.
CC mode is a state you can check. Inside the confidential VM it is queryable:
nvidia-smi conf-compute -f # CC status: ON
nvidia-smi conf-compute -e # CC Environment: PRODUCTION
nvidia-smi conf-compute -q # CPU CC Capabilities: AMD SEV-SNP or INTEL TDX
Protected: prompts, retrieval context, KV cache and model weights while in use, against the host, the hypervisor and anyone with physical or administrative access outside the measured TCB. Not protected: the GPU's HBM, which NVIDIA does not encrypt because it is deemed secure against everyday physical attack tools, and network transit, which remains your own TLS concern. The GPU also refuses work until attestation has passed and the ready state is set, which is the enforcement hook that makes the architecture auditable.
Attestation is where most deployments quietly fail. The pattern that actually delivers confidentiality has two gates, and both must be verified, not assumed.
Gate 1: composite attestation of CPU and GPU. The CPU TEE proves it is a genuine SEV-SNP or TDX confidential VM with the expected firmware. The GPU proves it is genuine NVIDIA silicon in CC mode with approved driver and VBIOS versions. NVIDIA's Remote Attestation Service (NRAS) is the primary verifier for the GPU side, with a local verifier for air-gapped deployments. The attestation quick start shows how little code this takes:
from nv_attestation_sdk import attestation
client = attestation.Attestation()
client.add_verifier(attestation.Devices.GPU,
attestation.Environment.REMOTE,
"https://nras.attestation.nvidia.com/v4/attest/gpu", "")
result = client.attest(client.get_evidence())
token = client.get_token() # JWT with per-GPU claims
The token's claims are concrete and pin-able: driver version, VBIOS version, secure boot state, certificate revocation status. Your release policy should pin the ones you care about, so a driver upgrade changes the measurement and fails closed until you approve it.
Gate 2: attestation-gated key release. Model weights ship encrypted at rest (AES-256-GCM is the standard pattern), and the data encryption key is wrapped by a key encryption key held in an HSM-backed KMS. The KMS releases the key only to a workload that presents a passing attestation. Microsoft's confidential LLM inferencing tutorial documents the full build on an AMD SEV-SNP VM with an H100, including the Key Vault release policy:
{
"version": "1.0.0",
"anyOf": [
{
"authority": "https://sharedeus2.eus2.attest.azure.net",
"allOf": [
{ "claim": "x-ms-isolation-tee.x-ms-attestation-type", "equals": "sevsnpvm" },
{ "claim": "x-ms-isolation-tee.x-ms-compliance-status", "equals": "azure-compliant-cvm" }
]
}
]
}
In that pattern the CPU evidence gates the KMS, and the GPU is verified in-guest before any key is used. The composition matters: no verified CPU measurements, no key; no verified GPU state, no model. A vendor that cannot walk you through both gates, with the evidence artefacts, is selling a feature flag, not a control.
The honest performance story is two-sided. Inside the GPU nothing changes: raw compute and HBM bandwidth are at par with non-confidential mode. The cost sits on the interconnect. NVIDIA measures the encrypted CPU to GPU path at roughly 4 GB/s, limited by CPU encryption performance, so workloads that are compute-heavy relative to data movement amortise well, while workloads that hammer host-to-device transfers or serve short, prefill-heavy exchanges pay more. NVIDIA's own published CC-mode overhead for LLM inference is in the low single digits, roughly 2 to 5%. Independent H100 measurements give you the range to plan against:
| Study | Platform | Models | Headline CC-mode result |
|---|---|---|---|
| Zhu et al. (2024) | AMD SEV-SNP (EPYC 9V84) with H100 NVL, vLLM 0.5.4 | Llama-3.1-8B, Phi-3-14B, Llama-3.1-70B (4-bit) | Throughput overhead under 7%, below 5% for typical queries, near zero for the 70B model; TTFT overhead up to 19% on the 8B model |
| H100 under Intel TDX benchmark (2026) | Intel TDX confidential VM, 1x H100 80GB | Mistral-7B, Qwen3-30B-A3B | TTFT up 21.8% and 27.8%; global token throughput down 17.7% and 21.1%; closed-loop throughput gaps of 11.5 to 20.2% |
Three operational readings fall out of these numbers:
Take a Dubai-based insurer deploying a claims-summarisation assistant: a Qwen3-30B-class model on a colocated 8-GPU H100 node, prompts of a few thousand tokens of policy and claims text, outputs around a thousand tokens. The colocation provider's staff hold host access, which is precisely the threat a TEE addresses, and the model weights are commercially licensed, so they are encrypted at rest with the data encryption key wrapped in the insurer's HSM.
Sizing, using the TDX benchmark's single-GPU figures scaled across the node:
The insurer has three options, all arithmetic on the same 21% gap: run a second node during peak windows (more GPU-hours, but no queueing), accept that peak concurrency caps earlier (the knee at 32 instead of 40), or trim context length. None of them is a reason to skip confidentiality. All of them belong in the capacity plan before the pilot, not after the first month of queueing complaints.
Latency is the smaller problem: at the study's 5 requests/s, TTFT rises from roughly 395 ms to roughly 505 ms, which no internal user will notice. The operational surprises are elsewhere. The attestation handshake is a one-time startup event, so it adds nothing to per-request latency, but a confidential VM cannot simply reboot: the deployment guide notes that rebooting terminates the VM under CPU CC limitations, and the GPU scrubs its state on reset. Restarts are measured in minutes, so autoscaling and maintenance windows need designing around that. Driver persistence mode must be enabled or the SPDM session tears down and the GPU needs a full reset before it will work again. And every driver or VBIOS upgrade changes the attested measurements, so your release policy must be re-pinned deliberately, in a change window, rather than drifting with apt upgrade.
Finally, confidentiality only holds end to end: the application inside the enclave must not be quietly calling a third-party moderation, guardrail or embedding API, because that exports every prompt out of the TEE in plaintext. The enclave boundary is where your architecture review starts.
| Anti-pattern | Why it fails |
|---|---|
| GPU CC mode without a CPU TEE | Prompts and driver state sit in plaintext host memory; the host can read them |
| CC-DevTools left enabled | Matches CC workflows with security protections disabled and performance counters on; it is a debug mode, not a deployment mode |
| Attestation performed but not enforced | If the KMS hands the model key to any booted VM, the measurement exercise is theatre |
| Verifying the CPU but not the GPU | Half a composite proof is no proof; the Microsoft pattern verifies both before it unwraps keys |
| Third-party API calls from inside the enclave | Confidential computing does nothing for prompts that leave the TEE to an external service |
| Unpinned TCB | Without version pinning in the release policy, a routine driver update silently changes the trust boundary |
| Assuming HBM is encrypted | It is not; it is deemed secure against everyday physical attack tools, which is a different claim |
Two further limits deserve plain statement. First, model weight licensing is a contract question a TEE does not touch: running a model inside an enclave does not create rights the licence does not grant, and some provider licences restrict deployment context, tenancy or geography regardless of the hardware. Second, attestation introduces a dependency you must own: NRAS is an online NVIDIA service, with local verification available for air-gapped estates, and the revocation and reference-manifest data behind local verification can go stale. If your threat model includes a fully offline deployment, test that path specifically.
When a provider claims "confidential AI" or "your data never leaves your control", these questions separate the two tiers:
| Question | What a good answer looks like |
|---|---|
| Which CPU TEE, which GPU, which modes? | Named SKUs: Intel TDX or AMD SEV-SNP with Hopper or Blackwell GPUs in CC mode, per the supported combinations in the deployment guide |
| Show me the attestation flow | Composite evidence: CPU TEE report and GPU attestation token, plus the policy that pins driver, VBIOS and firmware versions |
| What gates the model key? | HSM-backed KMS with a release policy bound to the attested measurements; no passing proof, no key |
| What is the measured overhead on our workload shapes? | Numbers for TTFT, throughput and saturation concurrency on representative prefill and decode mixes, not a vendor datasheet percentage |
| What leaves the enclave? | An explicit list of outbound calls from inside the TEE, ideally empty |
| How do reboots, driver upgrades and node replacement work? | A documented cold-start and TCB-repinning procedure, with expected timings |
At Azrty we design, build and run this kind of AI infrastructure for organisations in the UAE: the GPU platform, the gateway in front of every model, and the access and control layer around both (AI infrastructure). For teams that need agents and models running on their own GPU servers, each workload in its own isolated space and all managed from one console, our Kuvryn AI platform is built for that shape of deployment. We are based in Dubai, and our own team does the deployment and the support.
If you take one thing from this article: next time a vendor says your data stays in-country, ask them who can see it while it is in use, and ask for the attestation evidence. The hardware can now answer that question. Whether your vendor's stack does is a different matter, and it is entirely checkable.
Originally published on Azrty.