{"slug": "image-and-text-do-not-share-a-coordinate-system", "title": "Image and text do not share a coordinate system", "summary": "A linear classifier separates image from text embeddings with 100% accuracy in Kimi K3, Inkling, and Qwen3-Omni, revealing that each model's coordinate system, not missing content, causes the modality gap. Researchers at White Circle found that whitening each modality and fitting a single rank-32 rotation on 96 training pairs recovers caption retrieval at 9.9x chance in Kimi K3, 9.2x in Qwen3-Omni, and 3.2x in Inkling, showing the gap is partly basis mismatch.", "body_md": "In Kimi K3, a caption's embedding points almost perfectly backwards from its own image\nThat's not a bug. It's a coordinate system choice, and you can undo it with a single rotation🧵\n\n# Konstantin on X: \"In Kimi K3, a caption's embedding points almost perfectly backwards from its own image That's not a bug. It's a coordinate system choice, and you can undo it with a single rotation🧵\"\n\n- a linear classifier separates image from text embeddings with 100% accuracy in all 3 models we tested (Kimi K3, Inkling, Qwen3-Omni). It just reads off each model's coordinate conventions, not what the modalities actually share. (full writeup with interactive figures is atWhere the conventions come from: each modality is packed in its own cone. Kimi K3 packs all text into 1.6°. Qwen3-Omni does the same to images (2.0°). Same architecture family, opposite choice. The encoder-free Inkling from\n[@thinkymachines](https://x.com/thinkymachines)keeps both wide.So we tried the dumbest possible fix: whiten each modality, then fit ONE rank-32 rotation on training pairs. No scaling. No translation. No MLP. A rotation is not allowed to move or reshape anything, so it can only turn the image basis into the text basis.Result: held-out images retrieve their own captions at 9.9x chance in Kimi K3 9.2x in Qwen3-Omni 3.2x in Inkling A pure rotation, fitted on 96 pairs, recovers that much of the \"modality gap\".If you compare image and text activations directly using probes, similarity metrics, steering, the native coordinates will lie to you. Part of the \"gap\" is basis mismatch, not missing content. --- This research is done at[@whitecircle](https://x.com/whitecircle)-- we are hiring!", "url": "https://wpnews.pro/news/image-and-text-do-not-share-a-coordinate-system", "canonical_source": "https://twitter.com/advprop/status/2088771122707722365", "published_at": "2026-08-15 23:38:56+00:00", "updated_at": "2026-08-16 00:10:41.595568+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-research"], "entities": ["Kimi K3", "Inkling", "Qwen3-Omni", "White Circle", "Thinky Machines"], "alternates": {"html": "https://wpnews.pro/news/image-and-text-do-not-share-a-coordinate-system", "markdown": "https://wpnews.pro/news/image-and-text-do-not-share-a-coordinate-system.md", "text": "https://wpnews.pro/news/image-and-text-do-not-share-a-coordinate-system.txt", "jsonld": "https://wpnews.pro/news/image-and-text-do-not-share-a-coordinate-system.jsonld"}}