Linear probing for confidence estimation of simulations

Simile, a company that trains models to simulate human behavior, has developed a confidence model that predicts the error of its simulations using linear probing, achieving a Pearson correlation of 0.85 between predicted and actual Total Variation Distance (TVD) on a held-out set of ~8600 questions. The confidence model enables users to gauge the reliability of each simulation output without needing ground truth data at inference time.

Back /blog Building confidence in Simile Introduction At Simile, we train a model that simulates how an individual or population will act in a particular situation. Our model takes a prompt in natural language that describes the population, situation, and set of valid actions, and it returns a distribution over the actions that the population will take. We train on a variety of data: transactions from grocery stores and delivery apps that show us what people do, and AI-led voice interviews and surveys on our in-house platform that help us understand why they do it. For example, we can ask our model what brand of chips elderly people in Brazil tend to buy, what a randomly-sampled American thinks about the ChatGPT landing page, or, as a running example, whether a Floridian will tip on a delivery order. Even though we can ask all of these interesting questions, we can’t guarantee that our model will get them all right A model can be highly accurate on average while still not being accurate on every individual case. Therefore, it is useful to assign a measure of confidence to each model output. Judging a simulation We evaluate a simulation by running the same experiment on real human subjects and measuring the error between the predicted and observed distributions. For our tipping example, we work with a data provider that aggregates food-delivery app events to construct the following statistic: that 87% of people in Florida who order DoorDash, leave a tip. We compare the simulation to the ground truth using Total Variation Distance TVD , a metric of distance between two distributions. TVD sums the absolute difference on each answer option and divides by two: A TVD of zero means that the predicted distribution exactly matches the true distribution. A TVD of one means that the two distributions are disjoint. We can compare the model prediction we made earlier to the ground truth that we have now. Among the Floridians in that cohort, 87% leave a tip, and our model predicted 93%: a TVD of 0.06. Extracting a confidence signal Problem statement Our simulation model predicts a distribution . This distribution will have some error defined by the TVD between the distribution and the ground truth. Calculating the error requires the ground truth, which is unknown at inference time. To solve this problem, we train a model to predict the error of a given simulation. We surface the predicted error in the user interface for each question they ask our model. We call the model that predicts error a “confidence model.” We calculate performance metrics for the confidence model as follows: we judge its average error for an individual question using root mean squared error, or RMSE; we judge its ability to distinguish between errors above and below a certain threshold using the Area Under the Receiver Operating Characteristic curve AUROC ; we judge whether predicted error and actual error trend together with Pearson correlation. Performance metrics in this blog post are calculated on a set of questions that are held out from the training data of the simulation model and confidence model. We evaluate on a set of ~8600 questions that are roughly representative of our production queries. We evaluate the confidence model with 5-fold cross-validation, so every question is scored by a confidence model that never saw it in training. We pool those held-out predictions to compute the metrics below. Our splits are deliberately strict: a question's text and its entire experimental sample always land in the same fold. In practice, experimental samples tend to be grouped by topic, since people will run experiments with focused research questions in mind. We have developed several ways to predict the error of a query to the simulation model. We will present the methods we came up with and their relative performance on the benchmark. Baselines are presented first, then we introduce higher-performing methods. Basic features We start with a baseline: a regression on basic features of the question and predicted distribution. This includes the number of options in the question, the entropy, maximum, and kurtosis of the predicted distribution, the prompt length, and the gap between the top two options. Unfortunately, there is very little signal in these features. The AUROC over our evaluation set was 0.57. For context, AUROC judges the ability distinguish between two examples: one above a threshold, and one below, meaning that random guessing would achieve 0.5 AUROC. The correlation between predicted and observed error was also very weak r=0.12 . Semantic embedding Our next thesis was that there might be topics that our model is good or bad at e.g., perhaps we can answer questions about healthcare, but maybe not about space tourism, if our training data covers certain topics more or less . Bars are pooled K-fold estimates; whiskers are 95% bootstrap confidence intervals. Stars are paired bootstraps 10,000 samples against the neighboring method to the left; ns = not significant. Value axes are truncated. There is much more signal there: AUROC increased to 0.69, and the correlation between predicted error and observed error was 0.45. Probing Probing a neural network is the practice of taking internal activations and training a head to predict a property of interest. The most classical instance of probing is to train a head to predict the output of the model; this allows researchers to see how the representations get more rich https://arxiv.org/abs/1610.01644 as you get to the deeper layers of the network. In recent language modeling literature, probes have been used to detect deception https://arxiv.org/abs/2502.03407 and uncertainty in forecasting https://arxiv.org/abs/2607.08046 . This is a natural fit for our problem; perhaps the model we train has some representation of its uncertainty in the hidden state. We chose which layer to use for each fold by splitting the training set further into a train and validation set. For each fold, we train a ridge regression using each layer's activations as the input features, and select the layer to use for the held-out set based on performance on the validation set. The probe reads activations from the same forward pass that produced the answer, so it costs no extra model call at test time: one pass through the simulation model yields both the predicted distribution and the vector the regression consumes. Bars are pooled K-fold estimates; whiskers are 95% bootstrap confidence intervals. Stars are paired bootstraps 10,000 samples against the neighboring method to the left; ns = not significant. Value axes are truncated. Probing performs better than a linear regression on a general-purpose embedding. AUROC increased to 0.73, and correlation increased too to 0.54. Probing is simple, requires no additional training on GPUs, and adds almost no inference overhead. Fine-tuning the simulation model Combining the insight that our model has some representation of its uncertainty, and the fact that, typically, larger models can perform better on tasks, we decided to continue fine-tuning CFT the simulation model itself to predict its own error. In particular, we replace the language modeling head with a linear head that outputs a scalar prediction of error. Learning rate was tuned with the same method that we used to select layers: testing performance of different settings on a validation set. We then train the model with an RMSE loss. Bars are pooled K-fold estimates; whiskers are 95% bootstrap confidence intervals. Stars are paired bootstraps 10,000 samples against the neighboring method to the left; ns = not significant. Value axes are truncated. This edged out the probe on RMSE and correlation, though its AUROC of 0.74 is statistically indistinguishable from the probe’s 0.73. Fine-tuning a general-purpose language model Finally, to test whether the gain from the CFT approach was actually unique to CFT, or just a consequence of having more parameters, we try fine-tuning straight from the Qwen3.5-27B stock model. Bars are pooled K-fold estimates; whiskers are 95% bootstrap confidence intervals. Stars are paired bootstraps 10,000 samples against the neighboring method to the left; ns = not significant. Value axes are truncated. This did not perform as well as the CFT approach. This is consistent with the hypothesis that the process of training a simulation model happens to form useful representations around uncertainty. For the Florida tipping question, the CFT model predicts a TVD of 0.054. Recall that the observed error was 0.062, meaning that the predicted error was off by 0.008. This was a case where both our simulation model and confidence model performed well. From a confidence signal to trustworthy decisions After we have a performant confidence model, we want to use it to help Simile users make better decisions. How can we communicate what different confidence metrics mean to users? First, we have to define a notion of “decision-quality” simulation. We performed an internal rating exercise where members of the Simile team rated whether certain distributions led to the same conclusion as other distributions. We then picked a threshold for TVD with the best F1 score in classifying “Good” from “Bad” distributions 0.856 . This number is 0.16 TVD, based on ~2750 ratings from fourteen raters, with a Fleiss’s kappa of 0.75 over 410 multi-rated questions. We expect this definition to change over time as our definition of “decision-grade” becomes more precise with more ratings particularly from external raters and more diverse evaluation sets. To make our confidence estimates immediately legible, we define thresholds of predicted error which map to predictions being "High," "Moderately High," "Medium," or "Low" confidence. These thresholds are calibrated on our evaluation set to ensure that they effectively distinguish very trustworthy simulations to those that should be extensively verified. We now can tell the user when we are highly confident that a prediction is decision-grade, and distinguish this from a prediction that is not likely to be decision-grade. When we rate something “High”, it is 95% likely to meet the 0.16 TVD decision-quality threshold, compared to “Low”, which is only 38% likely to meet the threshold. The Florida tipping question’s predicted error of 0.054 places it in the High bucket. If we are to arrange the scores into ten buckets such that the same number of simulations fall in each bucket, we can observe a similar split of good-and-bad predictions as in the product-facing buckets. The average error of our highest-predicted-error decile is much higher than that of our lowest-predicted-error decile. The mean predicted TVD also tracks the mean observed TVD quite closely. Conclusion Confidence estimates can help customers understand when to trust Simile and make better decisions. They can also help our team identify weaknesses in the model without first collecting ground-truth data for every candidate question. They could also enable rapid advancement in social science. In principle, we could generate thousands of hypotheses about human behavior, test them with our simulation model, evaluate the likelihood of a test to be accurate with the confidence model, and prioritize real-world experiments that are likely to yield interesting and correct results about the human condition. Additionally, while TVD is a reasonable metric to judge error on a diverse set of questions, our confidence framework can generalize to an arbitrary metric. For any set of prompts and error metrics, we can learn to predict the error given the prompt. This will help the diverse set of researchers who use Simile ensure that their data is decision-grade, no matter where and how an individual sets their quality bar. The core mission of Simile is to represent humanity wherever decisions are made, accurately. Providing a question-level confidence is how we can guarantee that our simulations are worth informing the most consequential decisions. Acknowledgements We thank Chinmay Kulkarni, Tiffany Ding, and many others at Simile for insightful comments on early drafts, Keeyen Yeo and Natasha Tengorro for help with the visuals, Akaash Kolluri for early investigations into error prediction, and Isabelle Levent, Sam Catania, Lucas Haarmann for helping us get this technology into the hands of our users.