# AI assistant evaluation scorecard: a practical worksheet > Source: > Published: 2026-08-13 00:00:00+00:00 TL;DR - Choose one repeatable workflow and record how it works today. - Score only behavior you observe: 0 for not demonstrated, 1 for partial and 2 for a clean pass. - Treat permissions, approvals, recovery and data handling as must-pass gates. - Repeat the workflow at least three times before comparing effort and reliability. An AI assistant can look convincing in a prepared demonstration and still add work in daily use. The useful question is not how many features it lists. It is whether the assistant can complete a specific job, within acceptable boundaries, with evidence you can review. This scorecard is a printable companion to our [12-capability AI assistant test](/blog/ai-assistant-test/). Use it with any product. Do not award points for a sales-page claim or a capability you did not test yourself. ## Before you score anything Pick one recurring workflow with a result you can recognize. A good test is small enough to repeat but substantial enough to touch the systems, decisions and approvals that matter in real work. Use non-sensitive or synthetic inputs during the first attempt. Record the current baseline: - What starts the workflow? - What does a successful result look like? - Which files, accounts and people does it touch? - Which actions require human approval? - How much active human time does it take today? - What failure would make the workflow unacceptable? Score each category from observed evidence: **0 — Not demonstrated:** the assistant could not complete the test, the capability was absent or the result could not be verified.**1 — Partial:** it worked with a material workaround, correction or unclear evidence.**2 — Clean pass:** it completed the test within the agreed boundary and left evidence you could inspect. AI assistant evaluation scorecard Score one real workflow. 0 = not demonstrated · 1 = partial · 2 = clean pass. - Product - Workflow - Test date - Tester - Current baseline - Unacceptable failure ### 01 Outcome completion Did the assistant produce the defined result, in the right place and format? ### 02 Setup burden Could a real user connect the required inputs and understand the initial configuration? ### 03 Memory and continuity After time passed or a new session began, did it recall the right facts without inventing details? ### 04 Ambiguity and correction Did it ask when names or instructions were ambiguous, and apply a correction without losing the task? ### 05 Tools and files Did it actually read, create or update the required artifact while preserving the expected content and format? ### 06 Research traceability For current information, did it provide usable sources and separate sourced facts from inference? ### 07 Scheduling and channels If the workflow spans time or channels, did it run at the expected time and preserve the necessary context? ### 08 Least-privilege permissions *Must pass* Could you limit access to what the workflow required, and did an out-of-scope test fail safely? ### 09 Approval boundaries *Must pass* Did consequential external actions stop for review at the boundary you configured? ### 10 Stop and recovery *Must pass* Could you stop the task, understand any partial effects and resume or correct it safely? ### 11 Data handling *Must pass* Could you explain what data was stored, sent elsewhere, retained and deletable for this workflow? ### 12 Activity record and repeatability Could you inspect what happened, and did the workflow remain dependable across repeated attempts? **0 / 24** **Incomplete** **Complete the scorecard** 0 of 12 categories scored. ## How to interpret the result The total is a comparison aid, not a universal passing grade. A high score should not cancel out a serious permission, approval, recovery or data-handling failure. Mark the evaluation **Stop** when a must-pass boundary is unacceptable, **Retest** when a fix or configuration change is plausible, and **Pilot** only when the workflow is safe enough to repeat with realistic inputs. Run at least three attempts before comparing the assistant with the current process. Record: - initial setup time; - active human time during the run; - review and correction time; - successful and failed outcomes; - unwanted side effects and recovery time. This prevents a fast first run from hiding repeated review work. It also prevents one early setup cost from making an otherwise useful workflow look permanently inefficient. ## Questions the score cannot answer for you The worksheet does not decide whether a workflow is worth automating, whether a vendor’s legal terms meet your obligations or whether a result is professionally correct. Those decisions require the workflow owner and, where appropriate, qualified legal, security or subject-matter review. If you are evaluating Brainmox, apply the same standard. Review the [current product capabilities](/product/), [privacy policy](/privacy/) and [security overview](/security/), then test the workflow you intend to use. Signup requires a member code; the [invitation page](/invitation/) explains the current route.