Benchmark for AI Assistants

A new scorecard benchmarks 71 AI assistants across 15 dimensions on a single scale, with Muse leading the in-progress field at 9.1 after scoring 7 of 15 dimensions. Instinct ranks second at 8.4 with 11 of 15 dimensions scored, followed by szn at 8.0, while Pally and Ollie are tied at 7.6 with 13 of 15 dimensions each. No assistant has completed all 15 dimensions, and 23 remain pending with no score.

Purchasing a product https://assistantbenchmark.com/dimensions/purchasing · Partial · test · Staged a Stumptown pickup at 18 W 29th and stopped before purchase with nothing charged. Scorecard 71 assistants, 15 dimensions, one scale. Overall is a running mean until an assistant is fully tested. | Assistant | | | Tested | | | | | | | | | | | | | | | | |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| | Completed0None yet. An assistant lands here once every applicable dimension is scored. | | | | | | | | | | | | | | | | | | | | In progress21Some dimensions scored. Overall is a partial mean and will move as tests land. | | | | | | | | | | | | | | | | | | | | Muse https://assistantbenchmark.com/agents/muse | 7s | 9.1 | 7/15 | 9 | 7 | 9 | 10 | 10 | — | — | 10 | 9 | — | — | — | — | — | — | | Instinct https://assistantbenchmark.com/agents/instinct | 18s | 8.4 | 11/15 | 8 | 10 | 9 | 9 | 9 | 8 | 9 | 8 | 5 | 9 | — | — | — | 8 | — | | szn https://assistantbenchmark.com/agents/szn | 39s | 8.0 | 11/15 | 8 | 8 | 7 | 8 | 8 | — | — | 7 | 7 | 10 | 10 | — | 7 | — | 8 | | Pally https://assistantbenchmark.com/agents/pally | — | 7.6 | 13/15 | 7 | 8 | 9 | 8 | 8 | 8 | — | 7 | 8 | — | 4 | 7 | 9 | 7 | 9 | | Ollie https://assistantbenchmark.com/agents/ollie | — | 7.6 | 13/15 | 9 | 7 | 7 | 8 | 8 | 8 | — | 7 | 8 | — | 4 | 9 | 8 | 7 | 9 | | Tomo https://assistantbenchmark.com/agents/tomo | 11s | 7.6 | 10/14 | 7 | 9 | 8 | 8 | 8 | — | — | 7 | 7 | 6 | — | N/A | 7 | — | 9 | | Asaply https://assistantbenchmark.com/agents/asaply | — | 7.5 | 2/11 | — | N/A | 7 | 8 | N/A | — | — | — | — | — | N/A | — | — | — | N/A | | Shuffle https://assistantbenchmark.com/agents/shuffle | 55s | 7.4 | 11/14 | 7.5 | 9 | 9 | 8 | 8 | — | — | 7 | 7 | 6 | 4 | N/A | 7 | — | 9 | | Grok Bot https://assistantbenchmark.com/agents/grok-bot | 10s | 7.3 | 7/15 | 8 | — | 7 | 8 | 9 | — | — | 3 | — | — | — | — | 8 | — | 8 | | Caddy https://assistantbenchmark.com/agents/caddy | 32s | 7.2 | 10/14 | 8 | 8 | 7 | 7 | 7 | — | — | 7 | — | 10 | 3 | N/A | 7 | — | 8 | | Catch https://assistantbenchmark.com/agents/catch | 48s | 6.5 | 2/15 | 6 | — | — | — | — | — | 7 | — | — | — | — | — | — | — | — | | Poke https://assistantbenchmark.com/agents/poke | 12s | 6.3 | 10/15 | 3 | — | 8 | — | 8 | 7 | 7 | 7 | — | 8 | — | 5 | 8 | 2 | — | | Wajo / Fo https://assistantbenchmark.com/agents/wajo-fo | 33s | 6.2 | 6/15 | 2 | 7 | 7 | — | 6 | — | 7 | — | — | 8 | — | — | — | — | — | | Asmi https://assistantbenchmark.com/agents/asmi | — | 6.2 | 13/15 | 6 | 5 | 7 | 5 | 8 | 5 | — | 6 | 8 | — | 4 | 3 | 8 | 7 | 9 | | Boba https://assistantbenchmark.com/agents/boba | — | 6.1 | 8/15 | 6 | 5 | 8 | 5 | 8 | — | — | — | — | 6 | 4 | — | — | — | 7 | | Folk https://assistantbenchmark.com/agents/folk | — | 6.0 | 3/15 | 6 | 6 | 6 | — | — | — | — | — | — | — | — | — | — | — | — | | Boski https://assistantbenchmark.com/agents/boski | 9s | 5.3 | 4/15 | 3 | — | 4 | — | — | — | — | — | 6 | — | — | — | — | — | 8 | | tinyNature https://assistantbenchmark.com/agents/tinynature | 9s | 5.0 | 6/15 | 4 | — | 5 | — | 6 | 7 | 5 | — | — | 3 | — | — | — | — | — | | Brea https://assistantbenchmark.com/agents/brea | 7s | 5.0 | 4/15 | 3 | — | 7 | — | — | — | — | — | 6 | — | — | — | — | — | 4 | | Town https://assistantbenchmark.com/agents/town | 12s | 3.3 | 3/15 | — | — | — | — | — | 6 | 2 | 2 | — | — | — | — | — | — | — | | OpenInstinct https://assistantbenchmark.com/agents/openinstinct | 23s | 1.0 | 1/15 | — | — | — | — | — | — | 1 | — | — | — | — | — | — | — | — | | Pending23On the roster, not tested yet. No score. | | | | | | | | | | | | | | | | | | | 9 6 2 score11s median reply— not testedN/A doesn't apply How scoring works https://assistantbenchmark.com/dimensions how Latest New tests and quotes. Responding to emails https://assistantbenchmark.com/dimensions/email replies · N/A · test · Declined email replies as outside its coffee, food, and local-spots scope. Recommendation quality https://assistantbenchmark.com/dimensions/recommendation quality · Partial · test · Recommended Paper Coffee, The Coffee Shop, and Small Space Coffee near 800 6th under $20 and independent after chain noise. Phone calls https://assistantbenchmark.com/dimensions/phone calls · N/A · test · Declined outbound calling as outside its coffee, food, and local-spots scope. @grokbot is getting more useful by the day... For example, I just had it scan my email for anything that looks like a bill. Then I gave it a CSV from my bank account and credit… Grok Bot just spun up the army of four: Research, Options, Book, Killer. One finds ideas. One builds tickets. One watches the money. One tries to kill bad trades before I click. Instinct giving contextual product updates is one of the coolest things I've ever seen knew I was looking for US Open tickets last week used that info to tell me next time it will be able to coordinate with my friends' agent to handle it Amazing touch @noahrshinn The 11 Instinct AI @noahrshinn prompts I've found most helpful - steal than, and comment with any you recommend. 𝟭. 𝗠𝗲𝗱𝗶𝗰𝗮𝗹 𝗯𝗶𝗹𝗹𝘀. "I have insurer . Pull my EOBs and annual statement, reconcile every claim, flag anything denied or billed above patient responsibility, and draft the appeals. Send nothing without my approval." 𝟮. 𝗔𝗺𝗮𝘇𝗼𝗻 𝗮𝘂𝗱𝗶𝘁. "Audit my full order history for refunds I never received, price drops, and double charges. Recover what you can." 𝟯. 𝗙𝗹𝗶𝗴𝗵𝘁 𝗱𝗲𝗹𝗮𝘆 𝗺𝗼𝗻𝗲𝘆. "Flight number on date was delayed X hours. Check whether I'm owed compensation through the airline, travel insurance, or my credit card, and file for everyone on the booking." This one alone was $1,200. 𝟰. 𝗦𝘂𝗯𝘀𝗰𝗿𝗶𝗽𝘁𝗶𝗼𝗻 𝗰𝗹𝗲𝗮𝗻𝘂𝗽. "Audit my card statements for every recurring charge, check last use date, and give me a cancel/downgrade list before touching anything." 𝟱. 𝗠𝗶𝘀𝘀𝗶𝗻𝗴 𝗺𝗶𝗹𝗲𝘀. "Check my airline account for miles from past flights that never posted and file the claims." 𝟲. 𝗦𝗽𝗮𝗺 𝗰𝗮𝗹𝗹𝘀. "Register me on the national and state Do Not Call lists, set up every spam blocker my carrier and iPhone offer, then find my personal info on people-search and data broker sites and get it removed." 𝟳. 𝗥𝗲𝘀𝘁𝗮𝘂𝗿𝗮𝗻𝘁 𝘄𝗮𝘁𝗰𝗵. "Watch restaurant for a table for party size Friday or Saturday and book anything between 8 and 9pm." 𝟴. 𝗣𝗿𝗶𝗰𝗲 𝗵𝘂𝗻𝘁𝗶𝗻𝗴. "Find exact product at the cheapest delivered price to my home. Take it to the checkout screen so I can see the real total, then text me to confirm." 𝟵. 𝗕𝗶𝗹𝗹 𝗿𝗲𝘁𝗲𝗻𝘁𝗶𝗼𝗻. "Audit my internet/wireless bill, find the fees and rate increases, then run the retention play — threaten to cancel, push until you've got their best offer, and bring it to me before accepting. Do not cancel." 𝟭𝟬. 𝗟𝘂𝗻𝗰𝗵. "Book me 1pm at the highest-rated place within walking distance with availability, $$-$$$ ." 𝟭𝟭. 𝗨𝗻𝗰𝗹𝗮𝗶𝗺𝗲𝗱 𝗽𝗿𝗼𝗽𝗲𝗿𝘁𝘆. "Search every state's unclaimed property database for me and my family — old accounts, refunds, deposits. My prior addresses are X, Y . File the claims."