I’m increasingly preoccupied by the question of how universities will cope with the impending reality of inference rationing. These firms are facing what Mills and Whittle describe as the AI pricing problem: “the prices generative AI companies must charge are higher than the prices consumers are likely willing to pay, given the value consumers receive from these products”. I’m increasingly convinced that inference is effectively being offered at a loss in ways that need to be at the centre of how we see LLMs within organisations: our access to AI is being subsidised and that subsidy is likely to end soon. Mills and Whittle break down the cost structure of labs providing access to their models:
- The cost of developing the models: the cost of building the infrastructure to train the models, the compute used in training and the labour costs of the developers
- The costs of making the models available: the development and labour costs of making the models available as software, building the infrastructure required for inference and the inference costs of how users utilise the models
When I’ve tried to raise this problem (in my own possibly less authoritative sociological register) I’ve inevitably met the belief that ‘technology will make it cheaper’. The most frequent example is DeepSeek but my understanding is that the aggressive use of synthetic data from frontier models was a huge part of reducing their training costs. They effectively skipped one of the costliest bits of the process by relying on other firms who had already done it. While there are undoubtedly technological developments which can reduce costs of training and inference, the parallel imperative towards pushing the frontier means bigger and more expensive models over time, at least for the large AI labs. That is the closest thing they have to a ‘moat’. So while technology will plausible reduce fixed costs in some dimensions, there are countervailing trends pushing up fixed costs in other dimensions. In other words it just seems obviously implausible to me that we see a significant reduction across the entire cost structure. It will remain extremely expensive to build, train and operate these models. Even if cost-per-token falls the labs will still have to claw back huge capital investments through inference pricing.
These aren’t reflected in huge subsidies at the moment to institutional users across public sector organisations, including the American government:
The GSA (2025b) has struck agreements with OpenAI and Anthropic to access their technologies for only $1 per agency, while Google will provide its Gemini for Government product at a cost of only $0.47 per agency, with xAI’s technologies costing only $0.42 per agency (GSA, 2025c). Microsoft is providing its Copilot product for free (GSA, 2025d).
If they are correct that profitable firms would have to charge more than consumers are willing to pay, that is a huge problem for the labs. It also means we’re likely to see a period of intense volatility when all manner of explicit and opaque strategies are used in order to experiment with different ways of fiddling with the overall cost structure. The nearest term one is going to be a shift from *pricing by seat *to pricing by inference at least once current contracts come to an end. But there will be other modes as well because AI labs are currently selling an extremely expensive service to organisations at a significant loss. What does this mean for universities who have subscribed to enterprise AI? I can see three potential pathways here:
- They exit from the space entirely leaving LLM-access a matter of staff and student individual preference. The information governance problem remains and the potential to work on culture and integrity is lost, but a huge cost is removed.
- They pivot towards adapting open-weights models for sector-specific purposes. This would likely need to be consortium based due to the costs involved in getting it right, but it could be the best of both words in a sense.
- They develop internal processes to distinguish between better or worse uses of LLMs which would ultimately entail a form of internal inference rationing. I suspect this would be just guidance initially but if normative prompting isn’t sufficient to reduce costs then at some point someone is going to start mapping inference onto cost centres within the university. An awful lot could flow from that, none of it good.
The easiest way out of this dilemma would be if staff and students simply don’t engage with the platform in the first place. There’s enough evidence of polarisation around enterprise AI and reluctance to trust in house provision of AI that enterprise platforms might just not take off in the first place. But if they do then I struggle to see any options other than the other three but this is a first speculative attempt to map out the issues here.
I’m thinking of organising a workshop about this. If you’re interested please get in touch!