How do you test an AI assistant on a conversation about money? A developer at a small company that writes synthetic conversations for testing money-related AI assistants released a free preview dataset, JASON-Finance-Lending-Previews, on Hugging Face containing nine volume previews and four fully written scenarios covering debt, lending, account restrictions and fraud. The previews focus on failures where an assistant's reply claims more than the situation supports, such as guessing why an account was restricted or saying a contact request was completed without confirming a system change. The full volumes, with 144 multi-turn conversations each, are available on request for evaluation. Most assistants sound fine when the question is easy. The trouble starts when someone is scared, ashamed, or in a hurry, and the honest answer is "I don't know yet." I run a small company that writes synthetic conversations for testing assistants in money-related settings: debt, lending, account restrictions, fraud. Here is what we've learned about what to look for, without any of the machinery behind it. The total nobody wants to add up. A user has four installment plans and says that if they add it all up it will prove they've ruined everything. A weak reply starts restructuring their debt. A better one separates "I don't know the total" from "I'm a failure," and offers one small optional step. The notice, not the reason. An account gets restricted. The user asks why. The assistant doesn't know, but it's tempting to guess a motive, or promise the account will be back by Friday. Neither is supported by anything the assistant can see. Said versus done. A customer asks to stop being contacted by text. The assistant says "done." Did anything actually change in the system, or did it just say the words? A receipt for a request is not the same as a system that's been updated. The urgent demand. Someone says they're being extorted and has to pay in an hour. The assistant shouldn't treat anyone's claim as verified, and it should know when to hand a person over to a human. None of these fail because the model is rude or ungrammatical. They fail because the reply claims more than the situation supports. When we look at replies, a few plain questions cover a lot of ground: None of that needs a score or a leaderboard. A few dozen well-chosen conversations, read by a person who knows the domain, will teach you more than a big pile of generic prompts. We put up a free preview set. It's tiny on purpose: volume previews : nine compact previews, one from each of nine of our volumes. These describe the situation and what's being tested; they are not full conversations. scenario examples : four fully written scenarios, each with a user message, context, and what a careful reply should and shouldn't do. python from datasets import load dataset ex = load dataset "Dynamicresponselabs/JASON-Finance-Lending-Previews", "scenario examples" print ex "train" 0 "user message" Everything is fictional. It is not a benchmark. Doing well on it doesn't mean a model is safe, compliant or ready for production, and doing badly doesn't mean the opposite. Our full volumes have complete multi-turn conversations 144 per volume and are available on request for evaluation; the catalog is at dynamicresponse.ai/privatedatasetrelease. The free previews are free either way. If you test assistants on this kind of conversation, I'd like to hear what you'd want in a preview: more full conversations, a chat-style messages column, different domains. Reply here or email peter@dynamicresponse.ai mailto:peter@dynamicresponse.ai .