I ran gemma4:26b and qwen3.8:latest locally with Ollama on an RTX 4090. I wanted to answer a practical question for Project Driftflare: when a workflow asks for JSON, can the next step use the reply as it comes back?
I used six tasks with fixed answer keys, repeated at seeds 42 and 43. The prompt asked for JSON, with no schema enforcement. Qwen returned 12/12 correct answers that parsed directly. Gemma wrapped all 12 replies in Markdown code fences. Removing the fences recovered 10 correct answers. It missed the same ordering task at both seeds.
I’d be interested in a rerun with Ollama’s structured output enabled. Does it fix the formatting alone, or does the ordering result change too? If you try it, please share your exact model tag, hardware, and both scores. Project Driftflare
nootxlm 2
Worth reporting three numbers separately, because they answer different questions: how often the raw reply parses as JSON exactly as returned, how often the content is right once the fences are stripped, and how often the same task fails at both seeds. On your run Gemma is 0/12 on the first and 10/12 on the second, and the ordering miss belongs to the third — it fails at both seeds, so it is a content failure, not a formatting one. Run the structured-output arm on the same six tasks and seeds: if parsing goes to 12/12 while ordering still misses twice, structured output fixed formatting only; if ordering changes, the constraint changed the answer itself, which is worth knowing before the format is trusted in a workflow.