{"slug": "financial-text-classifiers-fine-tune-don-t-prompt", "title": "Financial text classifiers: fine-tune, don't prompt.", "summary": "A LoRA fine-tune of a 0.6B-parameter base model scored 82.1 F1 on Financial PhraseBank sentiment and 91.2 on 77-way Banking77 intent, versus 29.9 and 11.6 for the same model prompted zero-shot, according to a Numerata benchmark across seven public classification tasks published October 8, 2026. The LoRA fine-tune beat both TF-IDF and full fine-tuning on six of seven tasks and answered in about 30 ms, and reached 95.5 macro-F1 on TREC-QC with 480 labeled examples against 5,452 for full fine-tuning, a 91% reduction. Numerata's recipe freezes pretrained weights, trains a low-rank update, and merges it back into the base weights so the deployed model is a single ordinary checkpoint.", "body_md": "Financial text classifiers: fine-tune, don't prompt.\n\nOCTOBER 8, 2026 · NUMERATA TEAM\n\nA financial text classifier assigns each piece of financial text,\na news sentence, a client chat message, a compliance-flagged email,\nto one label from a fixed set. The fastest way to build one today is\nto ask a general-purpose LLM for the label. The most accurate way, by\na wide margin, is to fine-tune a small model on a few hundred labeled\nexamples. In Numerata's benchmark across seven public classification\ntasks, a prompted 0.6B-parameter base model scored 29.9\nF1 on Financial PhraseBank sentiment and 11.6\non 77-way Banking77 intent. A LoRA fine-tune of the identical model\nscored 82.1 and 91.2, beat both\nTF-IDF and full fine-tuning on six of seven tasks, and answered in\nabout 30 ms.\n\nThis post walks through the four standard ways to build a text\nclassifier, the full benchmark results, why each approach wins or\nloses where it does, and a reproducible recipe for building your own.\n\n+52.2 PTS VS. PROMPTING\n\nFinancial PhraseBank sentiment, same 0.6B model, same hardware: 29.9 F1 prompted, 82.1 after LoRA fine-tuning.\n\n6 OF 7 TASKS WON\n\nAgainst TF-IDF and full fine-tuning. The seventh, PhraseBank, is within 0.8 points of full tuning.\n\n91% FEWER LABELS\n\nTo reach 95.5 macro-F1 on TREC-QC: 480 labeled examples, against 5,452 for full fine-tuning.\n\nWhat is a financial text classifier?\n\nClassification is the workhorse of financial NLP. It's less\nglamorous than research agents or document Q&A, but it's where\nmost of the volume is: every headline that needs a sentiment score,\nevery client message that needs routing, every outbound\ncommunication that needs a surveillance flag. The output is one label\nfrom a closed set, so the task is easy to specify, easy to evaluate,\nand very easy to run millions of times a day. That last property is\nwhy model size and latency matter more here than almost anywhere\nelse.\n\nThe seven benchmark tasks below are public datasets, but each one\nis a direct stand-in for a production workload on a trading desk or\nin a bank's operations stack.\n\nBenchmark\n\nClasses\n\nWhat it tests\n\nFinancial workload it stands in for\n\nFinancial PhraseBank\n\n3\n\nSentiment of sentences from financial news\n\nNews and filings sentiment signals, earnings-call tone\n\nBanking77\n\n77\n\nFine-grained intent of retail banking queries\n\nClient service routing, chatbot triage, ticket tagging\n\nQuery routing for research assistants and internal search\n\nFour ways to build a text classifier\n\nEvery approach in the benchmark turns text into a label. They\ndiffer in where the knowledge of your label set lives: in a prompt,\nin word counts, or in the model's weights.\n\nPrompted base model (zero-shot). Describe the\nlabels in a prompt and ask the model to answer with one. No\ntraining at all. The label set lives entirely in the context window,\nso the model has to read it on every call and then generate a\nlabel as text, which your code has to parse.\n\nTF-IDF + linear classifier. The classic\nbaseline. Represent each document as weighted word and n-gram\ncounts, then fit logistic regression or a linear SVM. Trains in\nseconds on a CPU. It has no notion of word order or meaning beyond\nthe n-grams it sees, which is a strength on keyword-driven tasks\nand a weakness on phrasing-driven ones.\n\nFull fine-tuning. Update every weight in the\npretrained model on your labeled data. Maximum flexibility, but\nwith a few hundred examples and hundreds of millions of free\nparameters, it overfits easily and can overwrite useful pretrained\nknowledge.\n\nLoRA fine-tuning, merged. Freeze the\npretrained weights and train a small low-rank update alongside\nthem. After training, fold the update back into the base weights so\nthe deployed model is a single ordinary checkpoint. This is the\napproach labeled \"Numerata\" in the results.\n\nBenchmark setup\n\nAll four approaches were run against the same seven tasks. The\nthree model-based approaches share one 0.6B-parameter base model, so\nevery difference in the results comes from how the model was adapted,\nnot from model size. Scores are F1 on each task's held-out test set,\nhigher is better (macro-F1 for TREC-QC). Latency is P50 at batch size\none. Every number was produced on a single NVIDIA GH200 inside a\nself-hosted environment, with no data or model leaving it.\n\nResults: accuracy across seven tasks\n\nTask\n\nClasses\n\nPrompted base\n\nTF-IDF\n\nFull tune\n\nNumerata (LoRA)\n\nSMS Spam\n\n2\n\n0.0\n\n90.2\n\n94.6\n\n97.1\n\nAG News\n\n4\n\n67.0\n\n88.4\n\n91.6\n\n92.0\n\nFinancial PhraseBank\n\n3\n\n29.9\n\n66.5\n\n82.9\n\n82.1\n\nBanking77\n\n77\n\n11.6\n\n86.6\n\n85.4\n\n91.2\n\nSpamAssassin\n\n2\n\n3.9\n\n94.0\n\n87.1\n\n96.9\n\nEmotion\n\n6\n\n29.5\n\n81.4\n\n86.9\n\n89.1\n\nTREC-QC\n\n6\n\n7.2\n\n76.0\n\n96.7\n\n96.9\n\nMean of 7\n\n21.3\n\n83.3\n\n89.3\n\n92.2\n\nThree patterns stand out. Prompting is not a weaker version of\nthe other approaches, it's a different regime entirely, averaging\n21.3 F1 against 83 to 92 for everything else. TF-IDF is a far\nstronger baseline than its reputation suggests, and beats full\nfine-tuning on two tasks. And the LoRA fine-tune is the only\napproach that is never meaningfully behind on any task.\n\nWhy prompting a small model fails at classification\n\nThe prompted scores are not a rounding error. On Financial\nPhraseBank, a three-way task, 29.9 F1 is roughly what you would get\nby guessing. On SMS Spam the prompted model scores 0.0, and on\nSpamAssassin 3.9: it essentially never catches the spam it was\nasked to find. Three mechanisms explain most of this.\n\nThe label set has to fit in working memory.\nFor Banking77, the prompt has to enumerate 77 intents, many of them\nnear-synonyms (\"card not working\" vs. \"card swallowed\" vs. \"declined\ncard payment\"). A 0.6B model reading that list once per request has\nno stable internal representation of the boundaries between them.\nIt scored 11.6. TF-IDF, which learned those boundaries from\nexamples, scored 86.6.\n\nBase-model priors don't match your taxonomy.\nFinancial sentiment is not everyday sentiment. \"Operating profit\nfell less than expected\" is positive news. \"The company will\nmaintain its dividend\" is usually neutral. A model that has never\nbeen shown your labeling convention defaults to the everyday\nreading.\n\nGenerated labels have to be parsed. A prompted\nmodel returns free text. Any response that isn't exactly a valid\nlabel, a hedge, an explanation, a label that isn't in the set,\ncounts as a miss. Small models are especially prone to this, and on\na binary task where the model drifts toward one answer, F1 on the\npositive class collapses toward zero.\n\nLarger hosted models prompt considerably better than a 0.6B model\ndoes. But they cost far more per call, they're slower, and for most\nfinancial text they mean sending client communications or\nnon-public information to a third party. Fine-tuning gets a small\nmodel to the accuracy you need, inside your own environment.\n\nWhen TF-IDF is good enough, and when it isn't\n\nTF-IDF scored 94.0 on SpamAssassin, beating full fine-tuning by\n6.9 points, and 86.6 on Banking77, beating it by 1.2. Both are\ntasks where specific words carry most of the signal: spam has a\nvocabulary, and banking intents are usually named by their nouns\n(\"top-up\", \"PIN\", \"exchange rate\").\n\nIt's weakest where meaning depends on how words combine. On\nFinancial PhraseBank it scored 66.5, more than 15 points behind both\nfine-tuned models, because \"profit rose\" and \"loss rose\" share most of\ntheir n-grams and mean opposite things. TREC-QC is similar: question\ntype depends on structure (\"how many\" vs. \"how did\"), and TF-IDF\ntrailed by about 21 points.\n\nThe practical rule: always train TF-IDF first. It\ntakes minutes, it tells you how much a model has to beat to earn its\nserving cost, and on a keyword-driven task it may be all you\nneed.\n\nWhy LoRA beat full fine-tuning\n\nLoRA (low-rank adaptation) freezes each pretrained weight matrix\nW and learns an update ΔW = BA, where\nB and A are thin matrices of rank\nr, typically 8 to 64. For a 1,024 × 1,024\nprojection at rank 16, that's about 33,000 trainable parameters\ninstead of about a million. Across a whole model, the trainable\nparameter count usually drops to around 1% of the total or less.\n\nThat constraint is the point. With a few hundred or a few\nthousand labeled examples, full fine-tuning has far more freedom than\nthe data can pin down, so it tends to memorize the training set and\ndrift away from the general language knowledge that made the\npretrained model useful. The low-rank update acts as a strong\nregularizer: the model can only move in a small number of directions,\nso it keeps its pretrained representations and learns the decision\nboundary on top of them.\n\nThe benchmark shows exactly where that matters. The gap between\nLoRA and full tuning is widest on SpamAssassin (+9.8) and Banking77\n(+5.8), where training data is limited relative to the variety in\nthe inputs or the size of the label space. It's narrowest on TREC-QC\n(+0.2) and AG News (+0.4), where full tuning had enough data to\nconverge. On Financial PhraseBank full tuning edged ahead by 0.8, a\ntie for practical purposes.\n\nData efficiency: how many labels do you need?\n\nFor most financial teams, labeled data is the real bottleneck.\nLabels have to come from analysts or compliance staff whose time is\nexpensive, and sometimes the text itself can't be sent to an outside\nlabeling vendor. So the useful question is not \"which approach is\nbest with unlimited data\" but \"which approach is best with the 200\nexamples we can realistically label this month\".\n\nTo answer it, the benchmark trained each approach on TREC-QC at\nfive dataset sizes, from 60 labeled examples up to the full 5,452.\n\nAt 120 labeled examples, the LoRA fine-tune\nreached 83.6 macro-F1. Full fine-tuning on the same\n120 examples scored about 48, and TF-IDF about 42: leads of 35.5 and\n41.4 points. The LoRA run crossed 95.5 at 480\nexamples. Full fine-tuning needed the entire 5,452-example\ntraining set to reach a comparable score (96.7). The LoRA run got\nthere with 91% fewer labels; put the other way, full\nfine-tuning needed more than 11 times the labeling for the same\nresult.\n\nThat's the difference between a labeling project and a labeling\nafternoon. 120 examples is something a single analyst can produce in\na few hours, and a dataset small enough to review line by line before\nit ever touches a model, which matters when compliance has to sign\noff on training data.\n\nLatency: why the fine-tuned model is faster too\n\nYou might expect the more accurate model to be the slower one.\nHere it's the opposite.\n\nThe merged fine-tune runs at about 30 ms P50 at\nbatch size one. Prompting the same base model takes 63 to 174\nms depending on the task, 2 to 6 times slower. Two things\naccount for it:\n\nPrompt length. A prompted classifier has to\nprocess the instructions and the full label list on every request\nbefore it reads the input. The more labels, the longer the prefill,\nwhich is why the prompted range is so wide across tasks. A\nfine-tuned classifier processes only the input text.\n\nOutput length. A prompted model generates a\nlabel as one or more tokens, often with preamble. A fine-tuned\nclassifier produces a single prediction, one forward pass.\n\nMerging matters for the same reason. Because the LoRA update is\nfolded into the base weights before deployment, the served model has\nexactly the architecture and cost of the base model. There's no\nadapter lookup or extra matrix multiply per layer. At 30 ms a\nclassifier can sit inline in a request path, scoring every headline\nas it arrives or every message before it's routed, instead of running\nas an overnight batch.\n\nHow to build a financial text classifier\n\nThe recipe below follows the benchmark's lessons. The code is a\nminimal, standard Hugging Face transformers +\npeft setup, one reasonable way to do it rather than the\nexact benchmark harness.\n\nPin down the taxonomy. Write a one-line\ndefinition and two examples for every label, including the\nambiguous cases. Most classifier failures in production are\ntaxonomy disagreements, not model errors.\n\nLabel a few hundred examples, stratified.\nStart with 100 to 500, making sure rare classes are represented.\nHold out a separate test set of at least a few hundred that nobody\ntrains on or tunes against.\n\nTrain a TF-IDF baseline. It sets the bar the\nmodel has to clear.\n\nLoRA fine-tune a small model with a\nclassification head.\n\nEvaluate with macro-F1 and a confusion\nmatrix, not accuracy alone.\n\nMerge, serve, and monitor. Track the predicted\nlabel distribution over time. A shift is usually the first sign of\ninput drift.\n\nPYTHON · LORA CLASSIFIER\n\n``` python\nfrom transformers import AutoTokenizer, AutoModelForSequenceClassification\nfrom peft import LoraConfig, TaskType, get_peft_model\n\nBASE = \"path/to/small-base-model\"            # any ~0.5-1B open-weight model\nLABELS = [\"negative\", \"neutral\", \"positive\"]\n\ntok = AutoTokenizer.from_pretrained(BASE)\nif tok.pad_token is None:\n    tok.pad_token = tok.eos_token\n\nmodel = AutoModelForSequenceClassification.from_pretrained(\n    BASE,\n    num_labels=len(LABELS),\n    id2label=dict(enumerate(LABELS)),\n    label2id={l: i for i, l in enumerate(LABELS)},\n)\nmodel.config.pad_token_id = tok.pad_token_id\n\n# SEQ_CLS keeps the new classification head fully trainable;\n# only the low-rank updates are trained inside the transformer.\nlora = LoraConfig(\n    task_type=TaskType.SEQ_CLS,\n    r=16,\n    lora_alpha=32,\n    lora_dropout=0.05,\n    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\"],\n)\nmodel = get_peft_model(model, lora)\nmodel.print_trainable_parameters()\n\n# ... train with transformers.Trainer on your labeled split ...\n\n# Fold BA into W: one ordinary checkpoint, no adapter at serve time.\nmerged = model.merge_and_unload()\nmerged.save_pretrained(\"fin-sentiment-merged\")\ntok.save_pretrained(\"fin-sentiment-merged\")\n```\n\nPYTHON · EVALUATION\n\n``` python\nfrom sklearn.metrics import classification_report, confusion_matrix, f1_score\n\n# y_true, y_pred: label ids on the held-out test set.\n# For a prompted baseline, map unparseable outputs to a \"wrong\" id\n# rather than dropping them, or the comparison is flattering.\nprint(\"macro-F1:\", f1_score(y_true, y_pred, average=\"macro\"))\nprint(classification_report(y_true, y_pred, target_names=LABELS, digits=3))\nprint(confusion_matrix(y_true, y_pred))\n```\n\nA note on metrics. Financial label sets are almost always\nimbalanced: most messages aren't spam, most news sentences are\nneutral. A model that always predicts the majority class can post a\nhigh accuracy while never catching the class you built it for.\nMacro-F1 averages each class's F1 with equal weight, so that model\nscores poorly, which is what you want your metric to tell you.\n\nWhat this benchmark does and doesn't tell you\n\nIt's a controlled comparison: one base model, one GPU, seven\npublic datasets, four approaches. That makes it good evidence for the\nrelative ranking of approaches on short-text classification at small\nmodel scale. It doesn't tell you how a 7B or 70B prompted model would\ndo (better than 0.6B, at many times the serving cost), and your own\ndata will be messier than a curated public benchmark. Your taxonomy\nwill have more edge cases, and your labels more disagreement. The\nright move is to rerun the same comparison on a few hundred of your\nown examples. The setup above is small enough to do it in a day.\n\nFAQ\n\nWhat is a financial text classifier?\n\nA financial text classifier is a model that assigns each piece of financial text, such as a news sentence, a client message, or an analyst note, to one label from a fixed set: positive, neutral, or negative sentiment; one of dozens of banking intents; spam or not spam. Common uses include sentiment scoring on news and filings, routing client requests, and flagging suspicious communications for surveillance review.\n\nCan I just prompt an LLM to classify financial text?\n\nNot reliably with a small model. In Numerata's benchmark, a prompted 0.6B base model scored 29.9 F1 on three-way Financial PhraseBank sentiment, roughly chance, and 11.6 on 77-way Banking77 intent. LoRA fine-tuning the identical model raised those to 82.1 and 91.2. Larger hosted models prompt better, but cost more per call and send your text outside your environment.\n\nHow many labeled examples do I need to train a text classifier?\n\nFewer than most teams expect when fine-tuning a pretrained model with LoRA. On TREC-QC, 120 labeled examples got the LoRA fine-tune to 83.6 macro-F1, 35.5 points ahead of full fine-tuning on the same data. It reached 95.5 at 480 examples; full fine-tuning needed the full 5,452-example training set to get there, so the LoRA run used 91% fewer labels.\n\nIs LoRA fine-tuning as accurate as full fine-tuning for classification?\n\nIn this benchmark it was more accurate on six of seven tasks and within 0.8 points on the seventh. Averaged across all seven tasks, the LoRA fine-tune scored 92.2 F1 against 89.3 for full fine-tuning. The gap was largest where training data is small relative to the label space, such as Banking77, where LoRA scored 91.2 against 85.4.\n\nIs TF-IDF still a useful baseline for financial text classification?\n\nYes. TF-IDF with a linear classifier scored 94.0 on SpamAssassin, beating full fine-tuning, and 86.6 on Banking77, beating full fine-tuning there too. It is weakest where meaning depends on phrasing rather than vocabulary, such as financial sentiment, where it scored 66.5. Train it first: it takes minutes and tells you how much a model has to beat.\n\nHow fast is a fine-tuned small-model classifier?\n\nThe merged LoRA classifier in this benchmark ran at about 30 ms P50 latency at batch size one on a single GH200, compared with 63 to 174 ms for prompting the same base model. Merging the adapter into the base weights means there is no adapter overhead at serving time.\n\nWhy use macro-F1 instead of accuracy for financial classifiers?\n\nFinancial label distributions are usually imbalanced: most messages are not spam, most news sentences are neutral. Accuracy rewards a model for predicting the majority class. Macro-F1 averages the F1 score of every class with equal weight, so a model that never predicts the rare class, often the one you care about, scores poorly even when its accuracy looks high.\n\nWhere this fits at Numerata\n\nThe benchmark was run on Numerata's own stack: the LoRA training\non P95, the training layer, and the merged\nclassifiers served on NinetyFive, the inference\nruntime, all on one GH200 inside the environment. Your labeled data,\nyour training runs, and the finished model stay on infrastructure you\ncontrol: private cloud, on-prem, or fully air-gapped.", "url": "https://wpnews.pro/news/financial-text-classifiers-fine-tune-don-t-prompt", "canonical_source": "https://numerata.com/blog/post-financial-text-classifiers", "published_at": "2026-10-08 12:00:00+00:00", "updated_at": "2026-10-20 19:45:32.306047+00:00", "lang": "en", "topics": ["natural-language-processing", "machine-learning", "ai-research", "ai-tools"], "entities": ["Numerata", "Financial PhraseBank", "Banking77", "TREC-QC", "LoRA"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/financial-text-classifiers-fine-tune-don-t-prompt", "markdown": "https://wpnews.pro/news/financial-text-classifiers-fine-tune-don-t-prompt.md", "text": "https://wpnews.pro/news/financial-text-classifiers-fine-tune-don-t-prompt.txt", "jsonld": "https://wpnews.pro/news/financial-text-classifiers-fine-tune-don-t-prompt.jsonld"}}