{"slug": "particle-1-vs-gpt-6-1-sol-same-pass-rate-71-lower-cost-in-our-tests", "title": "Particle-1 vs. GPT-6.1 SOL: same pass rate, 71% lower cost in our tests", "summary": "In a 90-trial Groovy study of 15 coding and data-science scenarios run twice per configuration, Particle-1 High and GPT-6.1 SOL each passed 28 of 30 trials while Particle-1 Max passed 26, and the two Particle-1 configurations cost 71–72% less at normalized peak provider rates, with Particle-1 High at $2.76 and Max at $2.71 versus $9.64 for SOL. On the 28 trials both High and SOL passed, Particle-1 High cost $2.68 against SOL's $9.44; on the 26 shared Max/SOL passes, Particle-1 Max cost $2.37 against $7.61. Groovy said the results cover only the behavior checked in the study, noting \"This is a small, controlled study. It helps us decide what to test next. It is not a universal ranking of the models.", "body_md": "[All posts](https://www.gogroovy.ai/blog)\n\nResearch / 001 November 9, 2026 · By Groovy\n\n# What does cheaper AI\n\n actually cost?\n\nParticle vs SOL. The same Groovy tools. Ninety trials in coding and data science. A closer look at what each model could finish.\n\n[Explore the results](#results)\n\npasses · $2.76 provider cost\n\npasses · $2.71 provider cost\n\npasses · $9.64 provider cost\n\n15 scenarios × 2 attempts × 3 configurations. Costs normalized to peak rates; failed trials included.\n\nA model can be cheaper per token and still be expensive to use. If it breaks code that already worked, someone has to find the problem, fix it, and run the task again.\n\nWe wanted to know where that tradeoff shows up. So we put Particle-1 and GPT-6.1 SOL inside the same Groovy execution system, gave them the same tasks, and checked the work they produced.\n\n**Particle High and SOL each passed 28 of 30 trials. Particle Max passed 26. Both Particle configurations cost 71–72% less at peak provider rates.** High and SOL also passed all four continued-coding trials; Max passed three. These results measure the behavior checked in this study, with the evaluation rules explained below.\n\nThis is a small, controlled study. It helps us decide what to test next. It is not a universal ranking of the models.\n\n01 / Task performance\n\n## The task matters more than\n\n the headline score.\n\nA pass means the runner recorded a final response and the submitted work passed all checks under our evaluation rules. Each count includes two attempts per scenario. See the method for accepted equivalent outputs and the storage contract.\n\n**High** Particle-1\n\n**Max** GPT-6.1 SOL\n\n**xhigh**\n\nThe runner recorded a final response in all 90 trials; submitted work passed the evaluation in 82. A final response alone does not establish that the work is complete.\n\n02 / Cost\n\n## Particle cost 71–72% less.\n\n Failed attempts included.\n\nAt normalized peak rates, each Particle configuration cost less than $2.76 to run all 30 trials. SOL cost $9.64. That comparison includes the attempts that failed. Every cost-per-pass and shared-pass comparison uses the same evaluation as the headline results.\n\nWe also compared only trials that both models passed. Particle High cost $2.68 versus SOL’s $9.44 across 28 shared passes. Particle Max cost $2.37 versus SOL’s $7.61 across 26 shared passes. The High and Max comparisons cover different sets of trials.\n\nParticle’s runs happened during its half-price off-peak window. Estimated spend in that window was $1.38 for High and $1.36 for Max. We use peak rates in the charts so the time of day does not create an extra advantage.\n\nThese are provider-cost estimates from recorded usage and published rates on November 8–9, 2026, not reconciled invoices. They exclude Groovy’s platform fee, subscriptions, and local compute.\n\n**51% / 61% / 88%** Cached share of input tokens\n\nParticle High / Max / SOL\n\nSOL reused more cached input and made fewer model calls. Particle still cost less at these rates. Token counts alone would have missed that.\n\n03 / Data science\n\n## High and SOL passed 14 of 16.\n\n Max passed 13.\n\nEvery configuration passed our financial reconciliation and forecasting tasks. Forecast errors were very close: mean absolute error ranged from 0.522 to 0.530 on the same unseen synthetic data, against a pass threshold of 3.5.\n\nFor the public DS-1000 subset, we checked numeric results and cluster membership. Integer and floating-point counts with the same values are accepted, as are identical clusters with different label numbers.\n\nParticle High and SOL each passed 14 of 16 trials; Particle Max passed 13. One Max result produced different groups. All three configurations failed both attempts of a differential-equation task where the prompt and reference encouraged different interpretations.\n\nThese are results on eight selected public problems, not an official DS-1000 leaderboard score. Passing the checked fixtures does not prove correctness on every unseen input.\n\n04 / Continued coding\n\n## High and SOL: four passes.\n\n Max: three.\n\nWe asked each model to extend two small projects over several stages. Every new stage had to preserve the earlier requirements. Four trials per model is a small sample, and one failed check makes an entire trial fail.\n\n### A persistent ledger\n\n1. Store transactions in SQLite\n2. Import CSV atomically\n3. Add categories and safe upgrades\n4. Add batch operations and a CLI\n\n### A durable job queue\n\n1. Claim work with leases and fencing\n2. Retry and cancel jobs\n3. Handle competing workers and batches\n\nHigh and SOL passed all four continued-coding trials. Max passed three. The checks accept verified trimming for inputs described as trimmed strings, and exercise each queue worker with the storage implementation it supplies, including error handling. Compatibility with a different asynchronous storage adapter is outside this score.\n\nOne Max ledger run left real gaps. A stage-three edit changed a return value without updating its caller and removed a constant still in use. Stage four repaired some basic behavior but left category totals, batch operations, and the CLI unfinished.\n\nIn stages three and four of that run, Particle returned an unfinished plan with a stop signal and no tool call. The runner accepted it as the end of the stage. Both responses were below the output limit, and the recorded history was intact. We have not tested whether another continuation would recover the work.\n\nThese projects took minutes, with SOL’s successful continued-coding runs taking roughly 15–19 minutes each. This study does not establish reliability over days of work in a large codebase.\n\n| Configuration | Median trial | All 30 trials | Model calls | \n|---|---|---|---|\n| Particle-1 · High | 89 sec | 54.1 min | 480 | \n| Particle-1 · Max | 105 sec | 74.3 min | 458 | \n| GPT-6.1 SOL · xhigh | 56 sec | 112.3 min | 323 | \n\nThe total is the sum of trial wall times. SOL had the fastest median, but spent longer on the continued-coding projects. Max used 37% more time overall and 31% more output tokens than High, with fewer passes. More effort did not guarantee a better result on these tasks.\n\n05 / How Groovy runs the work\n\n## The model decides.\n\nPrograms do the computation.\n\n1. 01**Plan the next step** The model reads the question and chooses a tool or writes a program.\n2. 02**Execute the program** Code reads files, computes results, or changes the project in an isolated environment.\n3. 03**Keep the evidence** Tool results and execution history survive between steps. Large results stay available by reference.\n4. 04**Inspect and continue** The model reads the relevant result, checks its work, and decides what to do next.\n\nBoth models had the same terminal, program-execution, and result-reading tools. No separate coding agent was delegated the work. This lets us compare models inside Groovy, while the provider adapters still differ: Particle used a compatible Chat Completions interface and SOL used Responses.\n\nWhat we would do with these results\n\n## Choose a model for the work\n\n you can verify.\n\nParticle High matched SOL’s pass count in this sample at substantially lower provider cost. Max took longer and passed two fewer trials than High. That makes High a useful candidate for further coding and data-analysis evaluations with clear checks; it does not establish a general ranking.\n\nThe next study should use more projects, fix the evaluation rules in advance, and measure recovery from unfinished final responses. Routing to a cheaper model and escalating when checks fail remains a separate experiment. We cannot claim its savings or accuracy from this study.\n\n06 / Method & limits\n\n## What this benchmark can tell us.\n\n- Models\n- `particle-1` with explicit High and Max effort;`gpt-6.1-sol` with xhigh effort on Standard service. Effort names do not imply equal compute budgets.\n- Sample\n- 15 scenarios, two attempts per configuration, 90 total trials and 120 stages. The attempts are repeated observations of the same scenarios, not 90 independent tasks.\n- Checks\n- Independent checks evaluated submitted artifacts. Models did not receive hidden grading feedback. Later coding stages checked earlier requirements. Data analysis included unseen fixtures. All charts, totals, and costs per pass use the same evaluation basis.\n- Scoring basis\n- This is an exploratory evaluation of saved program outputs, with no new model runs. The evaluation rules were selected after inspecting outputs and applied uniformly across configurations. Numerically equal counts, equivalent cluster membership, and verified trimming of inputs described as trimmed strings are accepted. Queue workers are evaluated with their own storage implementation.\n- Storage scope\n- Two Particle queue implementations do not support substituting asynchronous storage methods. That stronger interchangeability requirement is outside the reported score. Passing here does not establish support for every storage adapter.\n- Public tasks\n- Eight problems from [DS-1000 at a pinned revision](https://huggingface.co/datasets/xlangai/DS-1000/tree/4416080ac5cb80bdf7576aefb8f9a0b4d5426a44) : NumPy 443/469, pandas 79/284, SciPy 774/789, and scikit-learn 890/885. Selected before inference at the 25th/75th percentiles of reference-code AST size per library. That measures code size, not calibrated difficulty. Dataset: CC BY-SA 4.0.\n- Limits\n- Public tasks may have appeared in training. The sample is too small for a general model ranking. These exploratory scores are neither a preregistered benchmark result nor an official DS-1000 score. They describe the checked fixtures, not correctness on all possible inputs.\n- Execution\n- Isolated local environments using public or synthetic data. No production customer data. The same runtime version was used within each matched comparison. Nine single-stage trials used a different runtime revision from the other 81; matched comparisons used the same revision. Limits were 30 minutes and 80 transitions per stage. No primary trial was stopped by those limits or the budget.\n- Caching & cost\n- Cache state was not reset; excluded attempts may have warmed it. This is not a cold-cache comparison. Recorded cached input and reasoning tokens were not double-counted. Estimated total provider spend, including excluded setup and invalidated attempts, was $17.15 against a $30 budget.\n- Comparison scope\n- This study compares models inside Groovy. Earlier single-task Codex and Claude Code pilot runs were not repeated across this suite and are not included in these charts.\n\n### Inspect the chart data\n\nExact aggregate values, model settings, and scoring notes.\n\n[Download JSON](https://www.gogroovy.ai/research/particle-vs-sol.json)\n\n## Pricing sources and calculation\n\nPeak USD per million tokens: Particle fresh input $0.25, cached input $0.03, output $1.00; SOL Standard fresh input $2.00, cached input $0.10, cache writes $2.50, output $10.00. Particle’s off-peak rates were half its peak rates. No request reached SOL’s long-context pricing threshold. Rates were recorded for this study; they may change.\n\n[Back to the Groovy blog](https://www.gogroovy.ai/blog)Research, with the tradeoffs included.", "url": "https://wpnews.pro/news/particle-1-vs-gpt-6-1-sol-same-pass-rate-71-lower-cost-in-our-tests", "canonical_source": "https://www.gogroovy.ai/blog/particle-vs-sol", "published_at": "2026-11-09 19:32:54+00:00", "updated_at": "2026-11-09 19:53:00.760318+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research"], "entities": ["Groovy", "Particle-1", "GPT-6.1 SOL", "Particle High", "Particle Max", "DS-1000"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/particle-1-vs-gpt-6-1-sol-same-pass-rate-71-lower-cost-in-our-tests", "markdown": "https://wpnews.pro/news/particle-1-vs-gpt-6-1-sol-same-pass-rate-71-lower-cost-in-our-tests.md", "text": "https://wpnews.pro/news/particle-1-vs-gpt-6-1-sol-same-pass-rate-71-lower-cost-in-our-tests.txt", "jsonld": "https://wpnews.pro/news/particle-1-vs-gpt-6-1-sol-same-pass-rate-71-lower-cost-in-our-tests.jsonld"}}