cd /news/large-language-models/how-much-should-an-llm-decide-buildi… · home › topics › large-language-models › article
[ARTICLE · art-174073] src=dev.to ↗ pub= topic=large-language-models verified=true sentiment=↑ positive

How much should an LLM decide? Building an avatar that reviews your training week with FastAPI, Gemini and Jev, a judge model

A developer built loadline, a FastAPI web app that imports Apple Watch workouts and has a 3D avatar deliver a weekly training review, with Gemini writing the avatar's lines and a judge model called Jev checking them before they are spoken. The design splits work so that code, not the LLM, makes every numeric judgment: fatigue is derived from fitness and fatigue exponential moving averages over 42 and 7 days, and weekly load is compared against the user's own nine-week average, with Gemini receiving those results and only writing the words. The developer said the split was needed because a wrong comparison could lead to wrong advice, and because the same records should always produce the same judgment so it can be tested.

recommendation

Compute fatigue and load comparisons in plain functions and pass the results to the LLM as fixed judgments it must not contradict, so the same records always yield the same review.

more in playbook: Designing agent systems
by read8 min views2 publishedNov 22, 2026

loadline is a web app that reviews your training week with an avatar. It imports the workouts you record on an Apple Watch. Once a week, a 3D avatar speaks a short review of the week: how tired you are, and how much you trained compared with your usual weeks. It then shows two or three options for the next week, and one of them is always to rest or train easy.

You can try the app without signing up. Click "Try as a guest" on the sign-in screen, and the app opens a dashboard with demo data and a weekly review.

The backend is FastAPI. Gemini writes the avatar's lines, and Jev, a judge model that only returns probabilities, checks them before the avatar speaks. This post is about the question I spent the most time on while designing it: how much should the LLM decide?

The simplest design sends a week of records to Gemini and asks it for a review. Gemini then has to compare numbers on its own, such as this week's training load against the average of the past nine weeks, or fitness against fatigue. An LLM can get a comparison like that wrong, and the review still sounds confident.

In this app, a wrong comparison leads to wrong advice. If the model reads a hard week as an easy one, the avatar may tell a tired user to train more.

I also wanted two properties that an LLM does not give on its own. The same records should always produce the same judgment, so that I can test it. And the weekly review should always be created, even when Gemini is busy or Jev does not respond.

So I split the work. Code makes every judgment that involves numbers, Gemini writes the words, and code decides whether the avatar can use those words.

A weekly review goes through four steps.

Only steps 2 and 3 use AI.

Code makes two judgments from the records. Both are plain functions in domain/, so the same records always give the same result, and unit tests cover them.

The first judgment asks whether the user is tired. The app computes daily training load from heart rate, and then fitness and fatigue as exponential moving averages of that load over 42 days and 7 days. Form is fitness minus fatigue. domain/situation.py divides form by fitness and maps the ratio to fatigued, fresh or steady. Because it uses a ratio, the same thresholds work regardless of how high the user's fitness is. Weeks with too few records, weeks without training, and weeks right after a break get their own values (not_enough_data, no_training, returning) before the ratio is checked.

The second judgment compares the week's load with the user's own average over the past nine weeks (domain/effort.py). Below 0.8 is light, 1.5 or above is heavy, and anything between is usual.

Code also decides whether Gemini may offer more training as an option. After a heavy week, the answer is no. Gemini receives this answer together with the two judgments and numbers that code has already computed, such as the number of workouts and the change from the previous week. Gemini never has to compare numbers itself.

Gemini decides what the avatar says and how it says it. The request has three parts (domain/review_prompt.py, adapters/gemini.py).

system_instruction in the Gemini API): rules such as never contradicting the judgments, encouragement first, no medical advice, always one option to rest or train easy, and a line that takes about 15 seconds to speak The system prompt states the split directly: "situation was decided by our own code from the numbers. Trust it and never contradict it."

Reading the numbers aloud adds little, because the dashboard already shows them. So the system prompt lists six well-known training principles and asks Gemini to pick the one that fits the judgments and use it as the reason for its advice. One of them is "Weeks of steady training add up to more than one big week." The prompt also limits numbers: the first sentence of the line has none, and the whole line mentions at most one. VO2max goes to Gemini only as a trend (up, flat or down), not as a value.

Here is the line Gemini wrote for the demo user's latest week. The code judged the week as steady with training usual, and the user's notes mention a race:

You kept your routine going smoothly after the race, and that consistency is wonderful. Weeks of steady training add up to more than a single big effort, and your body is responding nicely to the regular work.

Fifteen seconds is too short to explain the reasoning, so Gemini also writes an explanation of 80 to 120 words. The app shows it only when the user opens "Read more", and the avatar does not read it aloud.

Jev (by TypeSafe AI) is a model for judgments only and does not write text. You send it a text and some yes-or-no questions, and it returns the probability of "yes" for each question. The code decides what to do with those probabilities.

After Gemini writes the review, code joins the line with the labels and descriptions of the options and sends the text to Jev with three questions (domain/line_check.py):

The third question counts only in weeks where code has already decided not to offer more training. The system prompt gives Gemini the same rule, but code checks it again instead of relying on the prompt.

If any of these probabilities is 0.35 or higher, the app does not use Gemini's line and switches to the fallback lines. The threshold is below 0.5 so that the app also drops lines that Jev is unsure about.

Jev also reads the user's notes before the request to Gemini and answers whether they mention pain or sickness (domain/note_check.py). If the probability is 0.35 or higher, code sets a health flag, and options to raise volume or intensity are no longer candidates.

The same call that checks the line also asks Jev which expression fits it: happy, relaxed, sad or surprised. Jev returns its choice with a confidence. If the confidence is below 0.6, code uses the calm expression, relaxed (domain/expression.py).

Gemini can be busy, Jev can time out, and Gemini can return a line that breaks the length rules. The app handles all of these cases in the same way.

The code that calls external services lives in adapters/. When a call fails for any reason, the adapter logs the reason and returns None. The Gemini adapter, for example, tries the models in order from the newest generation to older ones. It moves to the next model on a 429, a 5xx or a network error, and it retries the same model once if the JSON fails the checks on length and the number of options. It returns None after a timeout, an authentication error, or when no model gives a valid answer.

The service in services/reviews.py only sees None. It does not know why the call failed, and it does not need to. For any None from Gemini or Jev, and for any line that Jev flags, it uses the fallback lines in domain/fallback_review.py. A line that Jev could not check counts as a line that failed the check. The fallback lines are prepared in English and Japanese and chosen to match the two judgments, so even a fallback review fits the week.

As a result, a weekly review is always created. If the voice cannot be generated either, the app saves the review without audio and shows only the captions.

The split between code and AI also decides the folder structure of the backend (api/app/).

Folder What it does What it does not do
routers/ Receives requests, checks who the user is, and calls services/ Calculations or judgments
services/ Loads data from the DB, calls domain/ andadapters/ in order, and saves the results Detailed calculations or judgments
domain/ Calculations and judgments, such as the two judgments and whether a line can be used Access to the DB or external services
adapters/ Calls Gemini, Jev, tts/ , Cloud Text-to-Speech and Cloud Storage Judgments

services/ does not import Gemini or Jev. It defines Protocols such as ReviewWriter and LineChecker and depends only on them. The only place that connects these Protocols to the real adapters is routers/deps.py, through FastAPI's Depends:

@lru_cache
def get_review_writer() -> ReviewWriter | None:
    return gemini_writer(get_settings())

ReviewWriterDep = Annotated[ReviewWriter | None, Depends(get_review_writer)]

The return type allows None. When the Gemini project is not configured, get_review_writer returns None, and the service uses the fallback lines, in the same way as when a call fails.

In tests, app.dependency_overrides replaces these providers. The API tests set them to None, so every review in those tests uses the fallback lines. The service tests pass fakes that return fixed answers, so they can check what happens when Jev flags a line without calling Jev. When I added Japanese voices, the change was one more adapter for Google's Text-to-Speech behind the same Speaker Protocol.

This split is not proven yet. I plan to keep using loadline with my own training records and check whether the weekly reviews match how my body actually feels. The thresholds for fatigue and the system prompt will change based on what I find.

Two smaller changes are also on my list. The avatar appears full-body and small on the dashboard, so its expression is hard to see, and I want to show it closer. Jev already returns a confidence for the expression, so the avatar could also show a stronger or softer expression depending on that confidence.

If you have built an app where an LLM gives advice, I would like to hear how you decided what to leave to the model. The demo and the source code are linked at the top of this post.

── more in #large-language-models 4 stories · sorted by recency
── more on @loadline 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/how-much-should-an-l…] indexed:0 read:8min 2026-11-22 · —