What's interesting isn't that it's happening — gray markets always find a way — but the volume. We're talking hundreds, not thousands. Enough for a few cluster bring-ups, maybe a handful of 8-node DGX-equivalent builds. Not enough to train a frontier model from scratch, but plenty for distillation runs, quantization benchmarks, and serving optimized 70B-400B parameter models at latency targets that H100s can't hit with the same power envelope.
The H200's 141 GB HBM3e matters more than the 4.8 TB/s bandwidth bump. For Chinese labs running long-context inference (128k+ tokens), that extra memory headroom lets them keep KV caches on-device instead of spilling to system RAM. I've seen internal benchmarks where a single H200 serves 2.3x the concurrent requests of an H100 on a 200k-context Llama-3-70B with vLLM's chunked prefill. That's the difference between "prototype" and "production" for a lot of these teams.
Nvidia's compliance team knows. The serial numbers are traceable. But enforcement against end-users in China would require cooperation from the intermediary jurisdictions — Singapore, Malaysia, UAE — which have zero incentive to police re-exports that generate tax revenue and logistics fees. The US Commerce Department's BIS has issued "is informed" letters to a few distributors, but that's whack-a-mole.
For Chinese AI firms, the calculation is straightforward: pay 2.5-3x list price ($45-50k per card vs $15-18k MSRP) for H200s today, or wait for domestic alternatives. Cambricon's MLU590 and Huawei's Ascend 910C are sampling now. Early silicon data suggests 910C hits ~80% of H100 FP16 throughput with worse software maturity. The 910D tape-out (3nm, targeting H200 parity) won't ship volume until late 2025 at earliest. So the H200 trickle buys time. Not strategic parity — just breathing room to optimize inference stacks, build dataset pipelines, and stress-test training frameworks on hardware that won't vanish overnight. The real constraint was never peak FLOPS. It's memory capacity per dollar, software ecosystem lock-in, and whether your quantization pipeline survives a driver update.
If you're running inference workloads in China right now, you're probably already evaluating H200 access. The question isn't whether to pay the gray-market premium. It's whether your model architecture can amortize that cost before domestic silicon catches up.
[Next MatrAIx lets you stress-test prompts against synthetic users →](/en/news/7011/)
[these AI tool field notes](https://tanyan888.com/), with plenty of directly applicable cases.