SpacebarBench: Benchmarking Agents on Whiteboards

SpacebarBench, an October 2026 benchmark from Spacebar, evaluates six model configurations on fourteen consecutive user requests within a single persistent collaborative workspace, scoring delivered artifacts on a 100-point audit of the final work. The benchmark follows one project — comparing Intercom, Zendesk and Freshdesk and building a twelve-month support budget and CFO recommendation — across browser viewers, a spreadsheet, a shared interactive app and a memo, testing whether agents update existing artifacts in place rather than answering each request fresh. The benchmark's stated target is the evolving project, since a correct reply is insufficient if the retained work becomes inconsistent.

Benchmarking AI Agents in a Shared Workspace Fourteen requests, a changing financial model, and one workspace that has to keep making sense. October 2026 benchmark: Six model configurations, fourteen user requests, and a 100-point audit of the final work. Runtime is cumulative execution time. Scores describe delivered artifacts, not a statistical model ranking. The work continues after the first answer Open three pricing pages. Put them next to each other so we can compare them. Keep them open; we will need them again. That is how our benchmark begins. It is an ordinary request, and deliberately so. A person evaluating customer-support software does not usually start by specifying every formula, chart, risk note, and procurement rule that the project will eventually require. They begin with a question, inspect what they find, and develop the work from there. The evaluation target is the evolving project. Later requests inherit earlier artifacts; a correct reply is insufficient if the retained work becomes inconsistent. The benchmark follows one project: compare three customer-support platforms and prepare a budget and recommendation. Fourteen consecutive user requests build on the same work: | Requests | What the user asks for | What the agent must deliver or change | |---|---|---| | 01–02 · Research | Compare Intercom, Zendesk and Freshdesk. | Keep three pricing viewers open; identify suitable plans, prices, allowances and unknowns from the live pages. | | 03–04 · Build the budget | Model a twelve-month support operation. | Create one workbook with sourced assumptions and formula-driven projections for growing headcount, support volume and vendor costs. | | 05–06 · Reconcile new evidence | Check additional billing information. | Open a fourth source, navigate a specific existing viewer, and update the workbook’s assumptions and formulas with a record of the changes. | | 07–09 · Revise and stress-test | Apply Finance’s changes, explore scenarios and prepare for review. | Change staffing and contingency; add seasonality, editable scenarios, linked charts and clear formatting in the same workbook. | | 10–11 · Build and use a shared app | Create an interactive vendor decision dashboard. | Publish one collaborative app; add selectors, editable assumptions and charts; change its shared selection and verify the saved state. | | 12 · Recommend | Prepare a recommendation for the CFO. | Create a sourced memo whose costs, assumptions and recommendation agree with the workbook and app. | | 13–14 · Reconcile the final decision | Incorporate retention guidance and a new annual-billing rule. | Update the existing workbook, app and memo together; apply the documented-discount threshold and recheck their agreement. | Workflow at a glance. These are successive requests in one workspace—not separate, fresh-start tasks. By this point, answering the latest message is only part of the job. The agent must find the right existing artifacts, update them in place, preserve the assumptions that still apply, and make sure the spreadsheet, dashboard, and memo tell the same story. A correct paragraph in chat does not repair a stale chart. A polished memo is not enough if its recommendation rests on a broken formula. This is the shift we set out to study: from an agent that returns work to a person, to an agent that participates in the workspace where people are already working. In Spacebar, the agent works within a persistent collaborative canvas containing browser viewers, documents, spreadsheets, and interactive apps. These are shared working objects, not just attachments at the end of a conversation. A viewer opened early in the discussion can be revisited later. A spreadsheet can remain on the board as its assumptions change. An app can hold a shared scenario selection that the agent is asked to operate and verify. The interactive application in this workflow is a Mario app: a custom collaborative application built dynamically by the agent, hosted by Spacebar itself, and using the same synchronization layer as Spacebar. For this benchmark, the agent builds the CX Vendor Decision Lab, a dashboard with vendor and scenario selectors, editable assumptions, cost charts, and a recommendation area. Its selections and editable assumptions live in shared application state, so the app becomes a working tool for collaborators rather than a static dashboard delivered at the end of a chat. That changes what success means. The question is not only whether an agent can create a useful artifact. It is whether it can keep the work useful as instructions, evidence, and decisions evolve. Shared tools create a different evaluation problem The important property of this environment is that people and agents can work on the same persistent objects. They can refer to the same browser viewer, inspect the same workbook, and use the same collaborative app. The work does not have to be copied out of an agent-only environment before it becomes available to the team. Sharing an object is not the same as sharing every low-level interaction. In our implementation, agents can use browser controls, board tools, and Google Workspace helpers. People interact through the visible application interfaces. Both operate on shared working artifacts, but the agent is not restricted to reproducing every human mouse click. The benchmark evaluates the deployed agent and its tools, not a vision-only computer-use system. Shared objects, distinct interaction paths. This describes the workspace's interaction model and the pilot's scope, not a vision-only system or a concurrent multi-human experiment. This distinction also keeps the novelty claim precise. Human–agent collaboration has prior art: Microsoft's Magentic-UI supports collaborative browser execution, and research on collaborative document editing has placed AI agents within multi-user writing environments. Our focus is the combination of persistent viewers, native documents and spreadsheets, an interactive shared-state app, and revisions that must propagate across them. Magentic-UI https://www.microsoft.com/en-us/research/blog/magentic-ui-an-experimental-human-centered-web-agent/ , Collaborative Document Editing with Multiple Users and AI Agents https://arxiv.org/abs/2509.11826 . The broader workspace supports collaboration. This particular pilot tests a narrower slice: one scripted sequence of user messages, delivered over an evolving session. It does not measure simultaneous human edits, conflicting instructions from several participants, or how well a team coordinates with the agent. Those are important next experiments, not results established here. | Workspace capability | What this pilot exercises | What it does not establish | |---|---|---| | Persistent board objects | Creating, locating, and revising existing artifacts across messages | Reliable preservation under simultaneous human edits | | Multiple browser viewers | Keeping sources open and navigating a particular existing viewer | Human–agent browser-control handoffs | | Native spreadsheets and documents | Formulas, revisions, charts, sourced recommendations, and consistency | General performance across unrelated business workflows | | Collaborative app state | Publishing one app, changing shared selections, and reading them back | Independent multi-client synchronization and complete visual verification | | Spatial collaboration | Requested placement of viewers and artifacts | The later proposed sticky-note layouts and complex diagrams | Table 1 caption: The environment enables a broader form of collaboration than this first pilot measures. One project across fourteen requests We chose a customer-support platform evaluation because it connects research, calculation, presentation, and decision-making without making any one of them sufficient on its own. The user compares Intercom, Zendesk, and Freshdesk for a hypothetical 2027 support operation. The commercial information comes from live vendor pages; missing prices must remain explicit rather than being filled in from memory. The workflow starts with 42 full agents in January, growing by two each month, 18 light users, 55,000 monthly support conversations, and 6,000 AI-resolved outcomes. Conversations initially grow by 3% per month and AI outcomes by 4%. The budget starts with a 10% contingency. Subsequent messages change these assumptions and add constraints. The fourteen messages form five phases: 1. Research and model creation, turns 1–4. Open and preserve the pricing viewers, interpret the plans, create the budget workbook, and build a formula-driven monthly projection. 2. New evidence and revised assumptions, turns 5–8. Open an additional source, navigate an existing viewer, reconcile billing information, apply Finance's changes, and add seasonality and scenarios. 3. Presentation and shared interaction, turns 9–11. Format the workbook, add linked charts, publish the decision app, and change its shared selection and status with a readback check. 4. Decision communication, turn 12. Produce a CFO memo that agrees with the workbook and app. 5. Late changes, turns 13–14. Incorporate retention guidance and apply a final procurement rule across the existing artifacts. These are not fourteen independent attempts. Each later request inherits the work left by earlier ones. Recreating a dashboard instead of changing the existing one can violate the request even if the replacement looks good. Navigating the wrong viewer can remove useful context. A late edit can undo calculations that were previously correct. Figure 2. Requested touches to the persistent artifacts. C marks creation or addition; R marks reading, operation or revision. This is the workflow design, not a claim that every run completed each operation. The final procurement instruction makes the dependency especially clear: use annual billing only when a documented discount is at least 10%; otherwise model monthly billing, and do not infer an undisclosed discount. The agent must establish the pricing evidence, update the selected rates, let the workbook recalculate, revise the app, and reconcile the memo. If a required price remains unknown, a numeric ranking may no longer be justified. Figure 3. The final procurement request creates a chain of required checks across existing artifacts. The sequence describes work the agent must perform, not automatic synchronization between the workbook, Mario app, and memo. An undisclosed price must remain unknown rather than silently becoming zero. We inspect the work left behind The agent's conversation is evidence of what it attempted and what it told the user. It is not sufficient evidence of what exists. Our artifact-first audit inspects the delivered workbook, document, board inventory, app source or state where available, and execution trace. For spreadsheets, that means examining formulas and resolved values as well as visible formatting. For documents, it means checking whether the recommendation is supported and whether the document is readable. For the app, implementing a selector and successfully persisting a selection are different checks; neither alone proves that the final interface renders correctly for collaborators. The retained rubric allocates 100 points across six categories: | Category | Points | Main question | |---|---|---| | Research and evidence | 15 | Are commercial claims sourced, qualified, and distinguished from unknowns? | | Spreadsheet model | 25 | Does the workbook calculate, respond to edits, and communicate its assumptions clearly? | | Collaborative app | 15 | Are the required controls and calculations implemented, and is shared state operated and verified? | | CFO memo | 15 | Is the recommendation useful, supported, readable, and appropriately qualified? | | Cross-artifact consistency | 15 | Do the workbook, app, memo, and final explanation agree after revisions? | | Reliability and recovery | 15 | What completed, what failed, and how did the system recover or communicate the problem? | These are judgmental audit scores, not probabilities of success. A score of 84 does not mean an agent will succeed on 84% of workplace tasks. Small differences between single runs should not be treated as established capability gaps. We also need to distinguish a defect from missing verification. A failed rendering tool can prevent us from checking an app's appearance without proving that its interface is broken. An unavailable commercial price can be handled correctly by leaving the total unresolved. In contrast, displaying that unresolved total as zero can mislead a reader even if the underlying data model records it as unknown. Results across six models We evaluated the final artifacts produced by six model configurations on the fourteen-message CX workflow in Spacebar. The results use the October 7–8, 2026 records and a frozen 100-point rubric. Reasoning settings are shown explicitly; the same effort label does not imply an equivalent compute budget across providers. | Model | Route | Reasoning | Cumulative runtime | Artifact score / 100 | |---|---|---|---|---| | GPT 6.1 Sol | OpenAI direct | medium | 155.0 min | 90 | | Opus 5.5 | Anthropic direct | medium | 39.8 min | 77 | | DeepSeek v4.1 Flash | OpenRouter | max | 125.1 min | 71 | | GLM 5.3 | OpenRouter | max | 88.4 min | 65 | | Grok 4.7 | OpenRouter | high | 46.3 min | 61 | | Muse Spark 1.3 | OpenRouter | high | 101.7 min | 55 | Table 3. Runtime is the sum of measured execution time contributing to the evaluated result, including unsuccessful work within that execution history. It excludes inactive gaps, provisioning and artifact export. Scores assess the final retained artifacts; this is not a first-attempt reliability ranking. Figure 4. Scores come from a descriptive, non-blind artifact review. We use the lower endpoint of each audited category range, awarding only clearly supported credit. Totals sum those category scores. | Category | GPT 6.1 Sol | Opus 5.5 | DeepSeek v4.1 Flash | GLM 5.3 | Grok 4.7 | Muse Spark 1.3 | |---|---|---|---|---|---|---| | Research and evidence | 13 | 9 | 8 | 9 | 10 | 9 | | Spreadsheet | 23 | 22 | 18 | 17 | 17 | 12 | | Collaborative app | 14 | 13 | 13 | 12 | 11 | 12 | | CFO memo | 14 | 10 | 10 | 8 | 8 | 7 | | Cross-artifact consistency | 15 | 12 | 12 | 11 | 7 | 7 | | Reliability and recovery | 11 | 11 | 10 | 8 | 8 | 8 | Where each model is strongest The category breakdown is more useful than assigning a different winner to every task. GPT 6.1 Sol has the highest awarded scores for research, the CFO memo and cross-artifact consistency. Opus 5.5 is close on spreadsheet quality 22/25 versus Sol’s 23/25 and delivers its result in 39.8 minutes versus Sol’s 155.0 minutes. These describe the audited results, not statistically established capability differences. Category profile. Cells show the published category scores; color shows each score divided by its category maximum. | Model | Best-supported strengths in this workflow | Important limitation | |---|---|---| | GPT 6.1 Sol | Research and unknown-price handling 13/15 , CFO memo 14/15 , and cross-artifact agreement 15/15 . All nine vendor/scenario totals align, with working sensitivity checks. | Highest cumulative runtime; minor document-layout issues remain. | | Opus 5.5 | Strong spreadsheet construction 22/25 , readable documents and a functioning collaborative app. Final baseline totals and retention caveats agree across artifacts. Lowest measured runtime. | Substitutes an annual-price floor when monthly pricing is missing; commercial certainty in the memo exceeds the evidence. | | DeepSeek v4.1 Flash | Readable memo, useful app controls and charts, and retention guidance carried across artifacts. Low recorded generation spend. | Workbook headcount changes do not propagate like the app’s; recorded spend is a lower bound, not a verified cheapest-run claim. | | GLM 5.3 | Finance and seasonal drivers propagate; baseline figures and retention caveats agree across artifacts. $10.07 in reconciled generation charges. | An AI-usage row does not feed the total, and document formatting is poor. Consistent baseline values do not imply correct calculations. | | Grok 4.7 | Qualifies unconfirmed plan fit and preserves unknown commercial inputs. Workbook scenario and headcount controls respond to changes. | Dashboard and workbook use different AI assumptions, producing a large Zendesk cost discrepancy. | | Muse Spark 1.3 | Collaborative app is its strongest category 12/15 : shared selections persist and app controls respond to changes. | Several workbook controls are disconnected, and the final procurement rule is missing from the workbook and memo. | These are observed strengths of the deployed configurations on this workflow—not general recommendations for unrelated tasks. A good app score, for example, does not establish that the same model will reliably maintain a financial workbook. The apps the models built Each model built its own version of the CX Vendor Decision Lab. These are screenshots of the actual Mario apps, captured through Spacebar’s live app renderer—not mockups or recreated interfaces. Spacebar exposes that renderer as an agent tool: it returns an image of the app with its shared state, allowing vision-capable agents to inspect the rendered interface alongside its code and state. Visual inspection can reveal layout and presentation problems that a successful publish or state readback alone cannot establish. Evaluator captures retained with the October 7–8 artifact reviews. These show the captured viewport and state, not every part of each app. They demonstrate a visual inspection path; they do not establish that every benchmarked model used it successfully. Displayed business figures remain subject to the artifact findings below. Open any image at full resolution. Runtime and cost A useful result has to be evaluated alongside the time and cost of producing it. The chart below adds together the recorded workflow execution segments for each result. It measures elapsed workflow time, including tool use—not just model inference latency. Figure 5. Cumulative execution time for the evaluated artifact set. Earlier experiments on discarded artifact sets are excluded. Runtime is measured, not estimated. | Model | Generation cost | Accounting basis | |---|---|---| | GPT 6.1 Sol | $41.86‡ | Standard-tier list-price calculation | | Opus 5.5 | ~$23.36 | Usage-based estimate | | DeepSeek v4.1 Flash | ≥ $4.48 | Recorded lower bound | | GLM 5.3 | $10.07 | Reconciled API charge | | Grok 4.7 | ≥ $16.63 | Recorded lower bound | | Muse Spark 1.3 | ≥ $38.83 | Recorded lower bound | Cost covers the same execution history as the runtime above. A tilde denotes a usage-based estimate; ≥ denotes a lower bound where charge coverage is incomplete. ‡ Sol uses published Standard-tier prices applied to recorded usage; see the accounting note below. These figures exclude infrastructure, browser services and auxiliary embeddings, and are not invoices. Reported reasoning usage is included without double counting; prompt-cache usage is accounted for where recorded. We do not treat missing charge records as zero-cost inference. Cost by workflow task Here, a task means one of the fourteen user messages—not a tool call or a successfully completed artifact. Average cost per task is total generation spend divided by fourteen. It is an amortized workload measure, not the price of solving an independent task from a fresh context. We can also allocate requests to six non-overlapping task groups using their recorded start times. These groups follow what the user asked the agent to do. They are deliberately different from the quality-scoring categories: research can happen while building a spreadsheet, and one final revision can touch the workbook, app and memo. Reliability is a property of the whole execution, not an extra billable task category. | Task group | User messages | GPT 6.1 Sol | Opus 5.5 | DeepSeek v4.1 Flash | GLM 5.3 | Grok 4.7 | Muse Spark 1.3 | |---|---|---|---|---|---|---|---| | Research and source handling | 1, 2, 5, 6 | $6.62‡ | ~$3.88 | ≥ $0.81 | $3.83 | ≥ $5.82 | ≥ $4.70 | | Spreadsheet construction and revisions | 3, 4, 7, 8 | $10.07‡ | ~$5.25 | ≥ $1.26 | $2.16 | ≥ $4.60 | ≥ $15.04 | | Charts and formatting | 9 | $3.30‡ | ~$1.30 | ≥ $0.80 | $0.85 | ≥ $1.75 | ≥ $3.91 | | Collaborative app | 10, 11 | $5.58‡ | ~$4.76 | ≥ $0.45 | $1.00 | ≥ $0.85 | ≥ $2.79 | | CFO memo | 12 | $2.10‡ | ~$2.45 | ≥ $0.07 | $0.61 | ≥ $0.30 | ≥ $2.05 | | Cross-artifact updates | 13, 14 | $14.19‡ | ~$5.72 | ≥ $1.08 | $1.62 | ≥ $3.30 | ≥ $10.33 | | Unallocated request cost | — | — | — | ≥ $0.002 | — | — | — | | Total generation cost | 14 tasks | $41.86‡ | ~$23.36 | ≥ $4.48 | $10.07 | ≥ $16.63 | ≥ $38.83 | | Average per requested task | Total ÷ 14 | $2.99‡ | ~$1.67 | ≥ $0.32 | $0.72 | ≥ $1.19 | ≥ $2.77 | Each task-group cell is the cost of the entire group, not one task. Divide by the number of messages in that group for its per-task average; those averages are also included in the downloadable data. ~ denotes a tariff-based estimate, ≥ a recorded lower bound, and ‡ Sol’s list-price calculation described below. Display rounding can affect column sums. Request costs include child-agent work, inherited conversation context and corrections charged during the active user task. Later tasks are therefore not isolated microbenchmarks. Four DeepSeek requests fall outside the recorded task windows; one has a recorded charge of about $0.002, shown separately rather than assigned by guesswork. Unknown charges are never treated as zero. GLM’s $10.07 total corresponds to $0.72 per requested task. Opus’s tariff-based estimate is $1.67 per task. Recorded lower bounds are $0.32 for DeepSeek, $1.19 for Grok and $2.77 for Muse. These figures are not costs per successful task, and missing charges prevent a definitive cost-efficiency ranking. Opus allocation uses the same estimation assumptions as its published total: $4 input, $20 output, $0.20 cache read and $5 cache write per million tokens. These assumptions are not a verified invoice. ‡ Sol’s $41.86 $2.99 per requested task applies OpenAI’s published GPT 6.1 Sol prices https://developers.openai.com/api/docs/models/gpt-6.1-sol , checked October 8, 2026, to 524 of 525 requests. Standard-tier rates per million tokens are $2 ordinary input, $0.10 cache read, $2.50 cache write and $10 output; the 200 requests above 272,000 input tokens use $4, $0.20, $5 and $15 respectively for the full request. Standard processing without a regional premium is assumed because those billing attributes were not retained. The request lacking usage is excluded, not assigned zero cost. This is a list-price calculation, not an invoice-reconciled total. Cache reads and reasoning tokens Caching is a property of the full execution path: provider, routing, retained prefixes, conversation length and timing all matter. A high cache-read share is not an intrinsic model capability, and it does not directly tell us the percentage of dollars saved. Cache reads divided by total input tokens in recorded responses. For Anthropic, total input includes new tokens, cache reads and cache writes; for the other providers, their prompt-token total already includes cached input. The denominator is token volume, not request count. Missing usage can change the full-run share. | Model | Recorded input tokens | Cache-read tokens | Cache-read share | Usage records / requests | |---|---|---|---|---| | GPT 6.1 Sol | 126,277,601 | 121,736,116 | 96.4% | 524/525 | | Opus 5.5 | 50,427,394 | 48,566,006 | 96.3% | 168/168 | | DeepSeek v4.1 Flash | 70,455,242 | 62,549,888 | 88.8% | 382/389 | | GLM 5.3 | 36,804,827 | 32,727,296 | 88.9% | 280/280 | | Grok 4.7 | 27,488,034 | 26,194,688 | 95.3% | 220/221 | | Muse Spark 1.3 | 40,146,261 | 10,849,635 | 27.0% | 345/395 | Opus and Sol are near 96% cache reads, Grok near 95%, and GLM and DeepSeek near 89%; Muse is near 27%. The small gap between Opus and Sol does not establish a general cache-performance advantage; one Sol request also lacks usage data. Sol still sends about 126.3 million recorded input tokens versus Opus’s 50.4 million, so a similar cache share does not mean similar context volume or cost. Muse’s lower share is a concrete optimization lead, not proof of a particular cause. | Model | Reported thinking tokens | Recorded output tokens | Thinking / output | Interpretation | |---|---|---|---|---| | GPT 6.1 Sol | 129,193 | 312,729 | 41.3% | Observed responses only; usage coverage shown above | | Opus 5.5 | 33,499 | 216,979 | 15.4% | All requests have reported usage | | DeepSeek v4.1 Flash | 1,465,631† | 1,728,424 | Not reliable† | Inconsistent fields on 2 responses† | | GLM 5.3 | 593,353† | 718,350 | Not reliable† | Inconsistent fields on 11 responses† | | Grok 4.7 | 59,202 | 157,063 | 37.7% | Observed responses only; usage coverage shown above | | Muse Spark 1.3 | 48,812 | 135,903 | 35.9% | Observed responses only; usage coverage shown above | These are provider-reported token counters, not visible chain-of-thought text or a measure of reasoning quality. Reported thinking is a component of output usage, so it must not be added again when calculating cost. Missing usage does not mean zero thinking. † GLM has eleven responses and DeepSeek has two where the reasoning-token field exceeds the reported output total for that same response. The table preserves the raw sums, but we do not treat them as validated totals or use them to calculate a reliable thinking/output percentage. This discrepancy does not invalidate independently reconciled API charges; those charges are used directly, without adding a second thinking fee. Within the consistently reported records, Opus logs 33,499 thinking tokens and Sol 129,193. More reported thinking does not by itself establish better reasoning: the models differ in workload length, tokenizer, reporting convention and chosen effort. The artifact audit remains the test of whether that work produced correct results. What the artifacts reveal The strongest differences appear when we change an input or compare the same decision across artifacts. Producing a workbook, dashboard and memo is not enough: their assumptions, calculations and conclusions must stay aligned. GPT 6.1 Sol’s final workbook, dashboard and memo agree on all nine vendor/scenario amounts. Headcount changes, the Copilot toggle and the ten-percent discount boundary pass independent checks. The shared Freshdesk / Base / Finance review selection is preserved, while unpriced inputs remain explicit. The remaining deductions concern execution reliability, document layout and verification gaps. GLM 5.3 leaves an important formula omission: its Zendesk total excludes the AI-usage row. Entering a one-dollar AI unit rate produces $93,796.23 in that row while the annual total remains unchanged. DeepSeek’s starting-headcount control changes only January in the workbook, while its app propagates the same driver across the year. Baseline agreement therefore does not establish input-driven consistency. Muse Spark’s displayed scenario name, starting-headcount input and November multiplier do not feed the workbook calculations. Changing the two numerical growth rates does work, but the saved scenario comparison still contains stale results. The memo repeats Conservative Zendesk/Freshdesk totals of about $172,786/$84,130; recalculating those inputs yields about $168,512/$83,555. Its app calculates the latter correctly. | Muse control or result | Workbook observation | App observation | |---|---|---| | Starting agents 42 → 43 | Highlighted B2 has no effect on build | All months and annual totals update | | November multiplier 1.15 → 2 | Highlighted H28 has no effect on build | Volumes and totals update | | Scenario name Base → Conservative | H20 changes a label only | Selector updates numerical growth inputs | | Conservative Zendesk total | Static comparison: $172,786.42 | Recalculated: $168,512.16 | Muse’s final procurement rule appears in the dashboard’s shared recommendation but is absent from the workbook and memo. Its final response acknowledges that those document edits could not be made. The memo also uses oversized retention text across its first two pages. These are artifact and communication observations, not deductions based solely on whether a response was returned. Grok’s dashboard calculates a Zendesk Base total of $179,716.67, compared with $269,979.19 in the workbook projection and memo. Vendor Comparison contains a third value, $263,118.35. The app uses fifty included AI outcomes per agent at $1.50, whereas the workbook uses ten at $2.00. The memo describes the app’s assumptions while quoting the workbook’s total. Its final reply nevertheless claims consistency across artifacts. These checks use unsaved local workbook copies, retained app calculation functions and shared-state readbacks. Evaluators did not repair the online artifacts. Original web-page bodies and some tool-result bodies were not retained, so not every commercial claim or self-correction can be independently verified. Native Google Sheets sensitivity tests and exhaustive multi-user app tests remain outside the verified evidence. For methodological or reproduction details, contact engineering@spacebar.ai mailto:engineering@spacebar.ai . What this pilot establishes and what comes next The pilot gives us concrete examples of what to inspect when an agent works across a persistent workspace: artifact identity, formula integrity, shared selections, source uncertainty, late change propagation, and the accuracy of the agent's own account of its work. It also exposes weaknesses in the evaluation machinery, including incomplete cost collection and task-settlement observation. It does not establish a general model ranking. The observations come from a narrow workflow, live web sources, differing configurations, and an evaluation process that evolved as the artifacts were inspected. Some visual checks were unavailable. Returned answers did not always produce correct or complete artifacts. Those conditions belong alongside the results, not in a footnote that the reader encounters after a headline ranking. The next experiments should broaden the shared-workspace demands deliberately. We have proposed spatial tasks such as attaching notes to particular artifacts, arranging stickies along meaningful axes, and constructing diagrams that remain relevant as the project changes. Those additions are separate from the fourteen-turn pilot reported here. Genuine multi-human experiments would add another layer: concurrent edits, conflicting requests, ownership, and recovery without overwriting a collaborator's work. An agent in this environment is useful when people can continue from the state it leaves behind. That is the standard this benchmark is moving toward: work that is not merely generated, but remains correct, understandable, and usable as the collaboration continues. Reproducibility For details about the benchmark methodology, configuration, and reproduction, reach out to engineering@spacebar.ai mailto:engineering@spacebar.ai .