Self-hosting AstaBrief-8B for cited reports: check every bracket before anyone reads it A developer self-hosted Ai2's AstaBrief-8B, an Apache 2.0 cited-report model built on Qwen3-8B, on a single 24 GB RTX A5000 and measured its inference behavior and citation accuracy. Across 11 generated reports containing 380 references, 363 (95.5%) matched a supplied excerpt, but eight reports cited 1-3 labels the prompt never contained, and the developer recommends a set-difference check against sent labels before human review. Batching four reports cut wall time from 209 s sequential to 85 s, while a concurrency misconfiguration and a silent first-token timeout bug caused a report to vanish without error. You have search over your own documents. Next comes a report with a reference after every claim, and the Allen Institute for AI Ai2 released AstaBrief-8B for that writing step. The weights the trained model file you download are Apache 2.0, and the base is the Qwen team's Qwen3-8B. We ran it on one NVIDIA RTX A5000, a 24 GB card, and counted every reference it wrote. Serving it is easy. One check in the code around it is not optional. Ai2 trained the model on one fixed instruction file. The model card warns that swapping it may give "degraded or inconsistent behavior". So your code fills two blanks, the question and the excerpts, and keeps even the line "Note that the current year is 2025". No report stated the wrong year; whether it ranks 2025 papers as the latest work, we left untested. Each excerpt carries a label: an ID, the first author, the year and a citation count how many other papers cite it . The model copies it into square brackets after each claim, and should write " LLM Memory " when it relies on its own recollection. Check the licence first. Apache 2.0 covers the weights only; the instruction file is CC-BY-NC-4.0, non-commercial. A paid product must clear that licence or write its own prompt in the same format, with the risk the card describes. Our SDK example ships no copy: it fetches the file from Ai2 at run time, under Ai2's terms. vLLM, the open-source program that loads a model onto the GPU and answers requests, treats AstaBrief as an ordinary Qwen3 model, with no custom code. Text is counted in tokens, about three quarters of a word each. The weights fill 16.4 GB of the 24 GB. We capped each request at 16,384 tokens question, excerpts and report , the length of Ai2's final training round. The leftover memory holds two requests at that cap side by side. Our real requests were nearer 7,000 tokens, so about four fit. Alone on the card, one report took 41.5 to 64.7 s, at roughly 39 tokens a second. Sending four is more telling. Sent together, four reports took 85 s, against 209 s one after another. The first try missed that, through a bug of ours. Concurrency is how many requests a model may work on at once, and pinning a model to one GPU machine through our SDK set it to one. The four waited in line: 198 s, and one report vanished. One settings call raises the limit today. The vanished report revealed a second, unfixed bug. If the first token the opening piece of the answer takes over 3 minutes, the request closes empty with no error. The card still finishes a report nobody receives. The AstaBrief-8B test notes on inferencekey.com https://inferencekey.com/news/astabrief-8b-cited-research-reports cover both, plus the workaround. On your stack, test the concurrency limit at each layer, first-token time under load, and whether a timeout fails loudly. The 11 reports held 380 references, and 363 95.5% match an excerpt we sent. Good, and also the trap: the other 17 look just like the genuine ones. Eight of the 11 reports cited 1 to 3 labels the prompt never contained. Part of them were our own excerpts with the label garbled. We sent W4411306417 | Thapliyal et al. | 2025 | Citations: 45 and got back 4411306415 | Thapliyal et al. | 2025 | Citations: 111 . The rest were recalled papers, dressed in the same format and missing the tag " LLM Memory " showed up once across 11 reports . Plain numeric IDs cured the missing W, not the wrong labels. You know every label you sent, so the check is a set difference: python import re sent = set labels in prompt exactly as you formatted them cited = set re.findall r"\ ^\ \ + \ ", report unknown = cited - sent flag before a human reads it Two cleanups belong there too. Each report opens with an empty