Typesafe says RLHF bakes overconfidence into AI models. Its RLCD method and Jeff model claim calibrated confidence instead. Here's what that means.
What is RLCD and how is it different from RLHF? #
RLCD stands for reinforcement learning for calibrated decisions, a training approach proposed by Typesafe as an alternative to RLHF (reinforcement learning from human feedback). Where RLHF trains a model to produce answers that human raters approve of, RLCD trains a model to produce a decision paired with a confidence score that reflects how likely that decision is to actually be correct. The pitch, made publicly by Typesafe’s Diogo Almeida, is that RLHF optimizes for sounding right to a person, while RLCD optimizes for being right, with a number attached that software can act on.
TL;DR #
- Diogo Almeida , who says he worked on the technique behind ChatGPT, is now publicly arguing that RLHF has structural flaws and is promoting a different approach through his company, Typesafe.
- RLHF trains on human approval , ranking responses by what people prefer, which Almeida claims quietly encourages overconfidence, mode dropping, and unreliable outputs even when the model sounds convincing.
- RLCD trains on outcome accuracy instead , rewarding the model for producing a confidence score that actually matches how often it’s right, rather than rewarding whatever answer a human rater likes best.
- Typesafe’s model, called Jeff , outputs a typed decision plus a calibrated confidence number instead of free-form conversational text, so code can automate above a chosen confidence threshold and escalate to a human below it.
- Typesafe claims Jeff is faster and cheaper for decision-style tasks , citing rough figures of about a tenth of a second and a fraction of a cent per call versus multi-second, fractions-of-a-cent-or-more costs for comparable large language model calls.
- The benchmarks are self-reported and unverified , including a claimed 0% structured output error rate versus low single-digit error rates for GPT and Claude models, and an accuracy-per-dollar chart favoring Jeff. Nothing has been independently tested, and the model is not publicly available yet.
- The real debate is about use case , not raw model quality: whether calibrated confidence scores are a better foundation for AI that acts autonomously, or whether RLHF’s tendency to produce confident, human-pleasing answers is actually appropriate when a person is still reviewing the output.
Remy is new. The platform isn't. #
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
Why is RLHF being questioned now? #
RLHF became the default way to turn a raw language model into something like ChatGPT. Humans rank multiple model outputs, and the model is fine-tuned to produce more of what raters preferred. That process is a big part of why chatbots feel helpful, polite, and confident rather than reading like raw next-token predictions scraped from the internet. Nearly every major consumer-facing model uses some version of this pipeline.
The criticism Almeida raises is that this setup rewards the wrong thing. A human rater is judging how convincing an answer looks in the moment, not whether it holds up when acted on later. Almeida’s argument, as described in his public comments, is that this creates specific failure patterns: models that drop valid but less-common answers (mode dropping), models that express confidence disconnected from actual accuracy, and models that behave inconsistently across similar inputs. None of that is fatal in a conversational assistant, where a person reads the answer and applies judgment. It becomes a real liability the moment a model’s output triggers an action in software with no human checking it first.
How does Jeff’s calibrated confidence approach work? #
Jeff, the model Typesafe built using RLCD, is designed to output a typed decision along with a probability score, rather than a block of conversational text. The idea is that instead of asking a model a question and getting a paragraph back, software calls the model and receives something structured, like a decision plus a confidence percentage. That number is meant to be calibrated, meaning a decision tagged 90% confidence should actually be correct around 90% of the time, based on how the model was trained to score outcomes rather than please a rater.
That structure lets an application set rules around it. Above a chosen confidence threshold, the system acts automatically. Below it, the decision gets routed to a human for review. This is a meaningfully different design goal than a chat model. A chatbot is built to be talked to. Jeff, as described by Typesafe, is built to be called by other software as part of a pipeline.
Because it’s producing a typed decision and a score instead of generating text token by token, Typesafe claims Jeff runs substantially faster and cheaper for this kind of task. The numbers floated publicly are rough: around a tenth of a second and a fraction of a cent per call, compared to multiple seconds and higher per-call costs for a comparable large language model doing the same decision-style task. Typesafe has also cited a claimed 0% structured output error rate against low single-digit error rates for GPT and Claude models on similar tasks, plus an accuracy-per-dollar comparison where Jeff reportedly comes out ahead.
Should you trust the benchmarks yet? #
Remy doesn't write the code. It manages the agents who do. #
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
Not without caution. Every specific number mentioned so far, the latency figures, the cost-per-call comparison, the 0% error rate claim, and the accuracy-per-dollar chart, comes from Typesafe’s own reporting. There’s no independent verification, no third-party benchmark suite, and no public access to the model at the time these claims surfaced. Typesafe has a waitlist rather than an open release, which means outside researchers and developers haven’t been able to run their own tests yet.
That doesn’t mean the claims are false. It means they’re unverified, and unverified benchmarks from the company selling the product deserve the same skepticism you’d apply to any vendor’s self-reported numbers. The concept behind RLCD, training a model to produce calibrated confidence rather than to maximize rater approval, is a coherent and reasonable idea worth taking seriously on its own terms. Whether Jeff actually delivers 0% structured output errors or the specific cost savings claimed is a separate question that only gets answered once outside people can run it themselves.
Is RLCD actually a better foundation for AI, or just a different tool? #
It depends heavily on what the model is being used for. RLHF’s tendency to produce confident, agreeable, well-formed answers is a genuine strength in a conversational assistant where a human is the last line of judgment. People generally want a chatbot that answers clearly and directly, not one that hedges every sentence with probability estimates.
The problem RLCD is trying to solve shows up specifically in autonomous or semi-autonomous systems, agents, pipelines, or backend automation where a model’s output triggers an action without a person reading it first. In that context, a model that’s confidently wrong is more dangerous than a model that says “I’m not sure” and hands the decision to a human. Calibrated confidence scores are, at least conceptually, a more honest signal for that kind of system to build automation logic on top of.
So the more useful framing isn’t “RLCD replaces RLHF.” It’s that they’re optimized for different jobs. RLHF for the front-end interfaces people talk to directly. RLCD, if Typesafe’s claims hold up under independent testing, for the back-end decision layers where software needs a trustworthy number to act on rather than a persuasive paragraph to read.
Frequently Asked Questions #
What does RLCD stand for?
RLCD stands for reinforcement learning for calibrated decisions, the training method Typesafe built as an alternative to RLHF. It trains a model to output a decision with a confidence score that reflects real-world accuracy rather than human rater preference.
Who created RLCD and the Jeff model?
Diogo Almeida, who says he worked on the technique behind ChatGPT, developed RLCD through his company Typesafe. He built it after publicly arguing that RLHF has structural flaws for use cases beyond conversational chat.
Is the Jeff model publicly available?
Not at the time these claims were made public. Typesafe has a waitlist rather than open access, and no outside party has independently tested the model or verified its benchmark claims.
Does RLCD replace RLHF entirely?
Not necessarily. RLHF remains well suited for conversational assistants where a human reads and judges the output. RLCD is aimed at automated or agent-style systems where software acts on a model’s output without a person checking it first.
Are Typesafe’s benchmark numbers verified?
No. Figures like the claimed 0% structured output error rate, the latency and cost comparisons, and the accuracy-per-dollar chart all come from Typesafe’s own reporting and have not been independently confirmed.