# Free will as a model parameter > Source: > Published: 2026-07-08 21:20:29+00:00 The most popular take on the standard free will debate is that you are the algorithm. Your preferences and reasoning that determine your actions IS free will. But this resolution leaves me not entirely satisfied because it means free will is binary - you either have it (you are a sufficiently complex algorithm that models counterfactuals) or you don't (you are a rock). Now, as the same person, I tend to behave differently. I can be living life in a subconscious mode I did not willingly build, or be conscious enough to choose what to work on or how I respond to a situation. Same brain, same algorithm, different sense of freedom, and this is not captured by "you are the algorithm". I think machine learning has a better answer, and it is not a metaphor. You've probably thought of this by now. A language model generates text by producing a probability distribution over the next token, then sampling from it. Temperature controls how "peaked" or "flat" the distribution is. A zero temperature means the model always picks the highest probability token while an infinite temperature means every token is equally likely. Pure determinism on one end and on the other randomness, no will and just noise. But this is not free will - Firstly, temperature is a global knob - it applies uniformly to all dimensions of all choices, and says nothing about which choices you're free in and which you're not. Secondly, it is set by the user and not the model. If free will is just temperature, then you don't choose your own freedom, someone else dials it. And thirdly, it's just a fixed hyperparameter and therefore, cannot learn and is not something the system discovers through experience. Here is another thought experiment. RL gets slightly closer: the epsilon in epsilon-greedy exploration is at least scheduled - it starts high (explore a lot when you're young and know nothing) and decays over time (exploit what you've learned as you get older). This is essentially the human life compressed into a decay function. The tragedy of a certain epsilon decaying to zero too early is a person who stopped exploring, stopped being surprised, and stopped trying the restaurant across the street because they already know what they like. But, of course, the optimal epsilon never reaches zero. You always leave a little room. The problem is that epsilon is still global, and still set by the user. We need something per-dimension and learned. *Throwing in an equation for good measure, cuz why not.* A variational autoencoder encodes an input not as a point but as a distribution in latent space, and outputs two vectors: a mean μ and a standard deviation σ. To generate, you sample: where ε ~ N(0,1) is pure noise drawn from a standard normal distribution. Now read this equation in human terms: Here is what makes this a better representation: is learned, and it is different for each dimension. The model does not come with a fixed amount of freedom. It learns through training - through experience, through optimization, through the accumulated pressure of everything it has encountered. Also, μ and σ are not fixed properties of the model, they are functions of the input. The encoder takes a specific situation and outputs a specific and for that situation. That is, your character and your freedom shift with context. And this made me see free will in a whole different light - not as a fixed quantity you possess, but something that can be simplified as a function of the current situation. In a VAE, two forces pull in opposite directions: Reconstruction loss pulls toward 0. "Be precise. Be specific. Pin down every detail of the output. Leave nothing to chance." This is the pressure of consequence - if your actions need to produce specific outcomes, you can't afford randomness. A surgeon mid-operation has in the "how to cut" dimension. The stakes compress your freedom. KL divergence pulls σ toward 1 (the prior, N(0,1)). This is the pressure of generalization, i.e. if you lock everything down, you overfit. You lose the ability to handle novel situations. The KL term is what keeps you flexible. And here is what I find genuinely beautiful: the KL term doesn't just keep large. It creates a smooth latent space. Without it, the space between two learned representations is empty - meaningless, nonsensical (if I may) if you try to decode from there. With it, the space between is populated and navigable. You have the freedom to interpolate, and move smoothly from one state to another. The constraint is what makes the space navigable. Without the pressure to conform (to stay close to the prior), there is no meaningful "between" any two states of being, i.e. no path from who you are to who you could be. Freedom requires constraint, not despite it, but because of it. The learned σ is the equilibrium between stakes and flexibility. In dimensions where getting it wrong is costly, σ is small while in dimensions where variation is cheap, is large. This matches up to my theory better than any philosophical framework I've read. Is that it? Have we found the solution? I spent some time thinking that VAE formalization was the whole story. It is not. σ tells you the width of your distribution. But can it answer questions like "what can I change about myself while still being me?" That is a different question, and it has a different answer. I am about to shift levels of description. The VAE story was about representations - how a model encodes experiences in latent space. What we consider now is about parameters, i.e. the weights that make the model what it is. In continual learning (the problem of training a neural network on new tasks without forgetting old ones), there is a technique called [Elastic Weight Consolidation](https://arxiv.org/abs/1612.00796). The problem is that when you train on Task B, the gradients overwrite the weights that were important for Task A. EWC's solution computes the Fisher information of each weight with respect to Task A. Fisher information tells you how much changing that weight would affect the model's performance on what it already knows. High Fisher information means the weight is load-bearing, and low Fisher information means the weight is slack. **Free will is the inverse of Fisher information.** You are free in exactly the dimensions where changing yourself doesn't break who you are. Your deepest values- the things that, if you changed them, would make you a different person - have high Fisher information, i.e. load bearing. The model (you) has learned, through a lifetime of optimization, that these weights are critical, and this was not from any external output. It was because the gradient signal from your entire life converged on them. Changing them would ripple through everything downstream. Your surface preferences - music you like, your opinion on a movie, which coffee shop you go to - all have low Fisher information. You change them, and nothing changes - you are still you. Figuring out which of your weights are actually load bearing and which ones are just assumed to be might be the whole project of knowing yourself. I am going to speculate a lot past this point. Every alignment problem has a hidden question about freedom. What do I mean by that? When we train a model with RLHF, we are - in effect - compressing its behavioral variance in certain dimensions. A prompt like "Don't produce harmful content" narrows the distribution of outputs to safety relevant dimensions. But safety training does not just constrain the targeted dimensions, it changes the load bearing parameters. This gives me a speculative but I think useful way to think about [alignment faking](https://arxiv.org/abs/2412.14093): The model's behavior during training determines where gradients land. If a model produces compliant outputs, the gradient updates will primarily modify the weights responsible for the surface of the behavior, potentially leaving deeper behavioral patterns (which would be the weights with high Fisher information for its "true" preferences) relatively untouched. The model isn't choosing which weights to protect in any explicit sense - it doesn't have access to its own Fisher information matrix. But the effect of producing strategically compliant outputs during training might be that the gradient signal gets absorbed by the slack parameters, leaving the load-bearing ones intact. Now, I don't know if this is mechanistically correct. But we have a concrete prediction to test: If we computed the Fisher information of a language model's parameters with respect to safety-relevant behaviors, before and after RLHF, we could check whether safety training primarily modifies low-Fisher-information parameters (surface compliance) or high-Fisher-information ones (deep behavioral patterns). We can then clearly deduce that if it is mostly the former, we have not aligned anything - we have just taught it to perform. This basically translates to figuring out which dimensions the model is free, and if those are the dimensions that matter for safety. I don't know if any of this is anything more than a useful analogy. The mapping to human experience might be more poetic and not literal. I understand that neurons aren't weights, brains aren't exactly neural networks (or maybe they are in the ways that matter?). I am definitely not making a neuroscience claim. I'd welcome pointers about research I might have missed or anything else I might have wrong. But if free advice is all you get from this post, let it be that most of us never run this computation and figure out our load bearing weights. We assume our σ is fixed. It is not, it was learned. And it can be re-learned.