cd /news/artificial-intelligence/an-irt-shaped-practice-score-is-not-… · home topics artificial-intelligence article
[ARTICLE · art-59685] src=dev.to ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

An IRT-shaped practice score is not an IQ test

IntelligenceMax uses an item-response-theory-shaped formula to update a learner's practice score after each answer, displayed on the IQ scale. However, the developer warns that the score is not a normed IQ test because the AI-generated item parameters are not calibrated from a norming sample, limiting the estimate to an adaptive practice tool rather than a clinical measure.

read2 min views1 publishedJul 14, 2026

IntelligenceMax updates a learner estimate after every answer and displays it on the familiar IQ scale. The arithmetic resembles item response theory. That resemblance is useful, but it is easy to ask the formula to carry more meaning than it can bear.

A formula can be internally consistent while its inputs remain uncertain. Here, the distinction begins with the item parameters.

For ability θ, item difficulty b, and discrimination a, the model assumes a logistic P(correct). The first estimate has a normal prior centred at 100 with SD 15. After an answer, the implementation takes one capped local update using score and Fisher information. Per-answer change is capped at 1.5 points. The calculation is the easy part. The inputs are where the uncertainty lives.

The AI that generates each question also assigns difficulty and discrimination. Those values are checked for shape and range, but they are not fitted from a norming sample. Calling the update IRT-style describes the logistic form. It does not establish that 115 means the same thing as a score of 115 on a validated instrument.

Within those limits, the estimate can support an adaptive practice loop. It cannot establish a clinical Full Scale IQ, a permanent change in general intelligence, transfer to school or work, or treatment of a cognitive condition.

Reviews of commercial brain training find the strongest evidence on trained tasks and close relatives. Broad gains are less convincing against active controls (Simons et al., 2016; Melby-Lervåg et al., 2016).

Until empirical calibration exists:

This is a logistic, IRT-shaped practice estimate that uses model-assigned item parameters. It is not a normed IQ score.

Public notes: [https://intelligencemax.ai/science](https://intelligencemax.ai/science)

Evidence map: [https://intelligencemax.ai/guide](https://intelligencemax.ai/guide)

OSF: [https://osf.io/kja9b/](https://osf.io/kja9b/)

Disclosure: I built IntelligenceMax. The technical question I most want readers to challenge is whether useful empirical calibration is possible for generated, mostly one-off items without quietly turning the system into a fixed bank.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @intelligencemax 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/an-irt-shaped-practi…] indexed:0 read:2min 2026-07-14 ·