{"slug": "jose-hernandez-orallo-proposes-llm-referenced-c-test-for-supraintelligence", "title": "Jose Hernandez-Orallo proposes LLM-referenced C-test for supraintelligence", "summary": "José Hernández-Orallo proposed an LLM-referenced C-test that generates inductive-inference items whose difficulty is measured in bits via Levin's Kt complexity, using an LLM as the reference machine, according to a paper posted to arXiv. The approach produced certified items harder than current human or frontier-model solvability and was administered as a first certified battery to twelve LLMs, allowing capability to be tracked as a continuous ratio-scale quantity without re-baselining each time a stronger model ships. The twelve-model agent-characteristic curves remain a research artifact rather than a drop-in production evaluation tool.", "body_md": "[arXiv](https://arxiv.org/abs/2610.14005)\n\n### Jose Hernandez-Orallo proposes LLM-referenced C-test for supraintelligence\n\nWhich summary reads better? Pick one — models revealed after.Both summaries are AI-generated.\n\nA new C-test generated and certified items with difficulty measured in bits using LLMs as the reference machine, then administered a first certified battery to twelve LLMs. The practical shift is that benchmark items can be produced beyond current human/frontier-model solvability instead of saturating, giving production teams a way to track capability headroom and model-to-model progress where today’s evals flatten out.\n\nThis proposes a benchmark that generates inductive-inference items whose difficulty is measured on an absolute bit scale (via Levin's Kt complexity) using an LLM as the reference machine, so item difficulty no longer saturates against human or frontier-model ability — you can certify problems harder than any current model can solve before they can solve them. Practically, it offers a non-circular, forward-compatible way to rank and track model capability as a continuous ratio-scale quantity, meaning your eval harness wouldn't need re-baselining each time a stronger model ships, though the twelve-model agent-characteristic curves here are a research artifact, not yet a drop-in production evaluation tool.", "url": "https://wpnews.pro/news/jose-hernandez-orallo-proposes-llm-referenced-c-test-for-supraintelligence", "canonical_source": "https://www.snipvote.com/story/cmv7sct0d000az6czvqoiqdki", "published_at": "2026-10-14 12:00:00+00:00", "updated_at": "2026-10-15 00:18:52.845576+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-safety"], "entities": ["José Hernández-Orallo", "arXiv", "C-test", "Levin's Kt complexity"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/jose-hernandez-orallo-proposes-llm-referenced-c-test-for-supraintelligence", "markdown": "https://wpnews.pro/news/jose-hernandez-orallo-proposes-llm-referenced-c-test-for-supraintelligence.md", "text": "https://wpnews.pro/news/jose-hernandez-orallo-proposes-llm-referenced-c-test-for-supraintelligence.txt", "jsonld": "https://wpnews.pro/news/jose-hernandez-orallo-proposes-llm-referenced-c-test-for-supraintelligence.jsonld"}}