Show HN: Grade text from the CLI with custom rulesets and Jev Developer lukstei released slop-grader, a CLI tool that grades text files against custom rulesets using the Jev System One model from typesafe.ai. The tool batches lines into groups of 255 and evaluates each rule across a batch in a single call, so a 300-line document with 5 rules runs in 10 parallel requests, avoiding the thousands of text-generation requests a per-line generative LLM check would require. Rules can be written in Markdown or JSON, and the tool targets use cases including AI writing habits, technical documentation, code review diffs, CSV audits, contracts, support transcripts, and incident postmortems. Rule-based slop grader for text files, powered by Jev https://typesafe.ai . slop-grader runs as a two-step loop: grade text with the CLI, then paste the output to your AI agent to plan the improvements. Run slop-grader on a document like examples/slop.md https://github.com/lukstei/slop-grader/blob/main/examples/slop.md : npx @lukstei/slop-grader@latest -r no-ai-slop examples/slop.md Pass the output to your AI agent: - The agent distinguishes real violations from false positives and generates concrete replacements Example with Gemini 3.8 Flash https://github.com/lukstei/slop-grader/blob/main/examples/plan.md : Line 1 — banned word - Original: 🚀 The Ultimate Paradigm Shift in Modern Data Architecture - Fix: Modern Data Architecture - Reason: Removes the banned phrase "paradigm shift" and decorative emoji. Line 7 — binary contrast + faux insight + colon reveal + fake profound kicker - Original: What most people get wrong about databases is simple: it's not about speed, it's about trust. - Fix: Database design balances speed and trust. - Reason: Removes rhetorical framing and fake insight. - After your review the plan is applied to produce an improved document https://github.com/lukstei/slop-grader/blob/main/examples/slop-improved.md . How does evaluation work? Evaluation separates line-level checks spotting specific patterns or phrases from document-level checks evaluating tone or overall structure . Documents have hundreds of lines, but the number of rules is fixed. Sending one API request per line would mean hundreds of calls. Instead, slop-grader groups lines into batches of 255 and evaluates each rule across the batch in a single call. A 300-line document with 5 rules runs in 10 parallel requests. Document rules run in a single request across the entire text. How is this different from using an LLM to check text? Standard generative LLMs evaluate an entire document in a single prompt against a list of rules. On longer texts, they skip lines, miss rules, and report wrong line numbers. Running a separate check for every line and rule with a generative LLM is impractical. A 300-line draft tested against 10 rules would require 3,000 text-generation requests, which is slow and expensive. slop-grader uses a System One model Jev https://typesafe.ai . System One models answer discrete semantic questions with typed probabilities without generating text. Because these judgments return numbers instead of prose tokens, slop-grader can test every line against every rule separately and in parallel. Can I use custom rules for my use case? Yes. You can write custom rules in Markdown -r ./my-rules.md or JSON. Rules ask plain-text questions about a single line or the whole document, evaluated against criteria you define. See Custom rulesets custom-rulesets in the Rulesets rulesets section for syntax details and validation instructions. Which use cases exist for this tool? slop-grader works on any plain text, structured file, or code diff: - Catch AI writing habits, filler adverbs, and rhetorical formulas in drafts and essays. - Check technical documentation for missing code examples, disorganized steps, or unexplained jargon. - Scan code review diffs for swallowed errors, hardcoded credentials, and unjustified type assertions. - Audit CSV spreadsheets and transaction logs for values exceeding approval thresholds. - Review contracts and legal agreements for uncapped indemnification obligations. - Check customer support transcripts for unreleased feature commitments or unauthorized discounts. - Evaluate incident postmortems to confirm they address systemic defenses instead of individual human error. See the Rulesets rulesets section for pre-built rules and example templates. - Node.js 18+ - An API key for your chosen provider: - TYPESAFE API KEY — typesafe.ai https://typesafe.ai uses jev provider - OPENROUTER API KEY — openrouter.ai https://openrouter.ai uses openrouter provider export TYPESAFE API KEY=... npx @lukstei/slop-grader@latest -r no-ai-slop -r article-scores my-draft.md Pass built-in rulesets by name -r article-scores : | Ruleset | What it checks | |---|---| | article-scores https://github.com/lukstei/slop-grader/blob/main/rules/article-scores.md | Document-level scores: engagement, narrative arc, closing strength | | tech-docs https://github.com/lukstei/slop-grader/blob/main/rules/tech-docs.md | Technical documentation patterns: structure, task orientation, completeness, code examples, minimizing complexity | | grammar-english https://github.com/lukstei/slop-grader/blob/main/rules/grammar-english.md | English grammar: spelling and confused words, agreement, verb tenses, prepositions, pronouns, sentence structure, comparatives | | grammar-german https://github.com/lukstei/slop-grader/blob/main/rules/grammar-german.md | German grammar: spelling and confused words, agreement and inflection, word order, punctuation and typography | | no-ai-slop https://github.com/lukstei/slop-grader/blob/main/rules/no-ai-slop.md | Banned words, empty adverbs, puffery, colon reveals, bold lead-in lists, weasel attribution, dramatic fragmentation | Define custom rules in Markdown -r ./my-rules.md , organized under Line Rules and Document Rules sections. You can be creative and ask any plain-text question about a single line or the whole document. Rules work on any text, git diffs, server logs, legal contracts, or structured text like CSV files: Line Rules empty adverb Does the line use an adverb that adds nothing to the meaning? Criteria - true : The adverb could be deleted without changing the sentence. - false : The adverb carries real emphasis or spoken rhythm. Document Rules narrative arc Rate the narrative arc of the document. Criteria - No clear arc — sections feel disconnected - Loosely organized — a theme but no build - Clear progression — each section sets up the next - Tight arc — the ending pays off the opening Structured text CSV transaction audit Line Rules suspicious refund Does this CSV transaction row show a refund exceeding $500 without a manager approval ID in column 6? Criteria - true : The row records a refund over $500 and column 6 lacks an approval ID. - false : The amount is $500 or less, column 6 contains an approval ID, or the row is not a refund. Incident postmortems systemic analysis Document Rules root cause depth Evaluate whether this postmortem addresses systemic engineering safeguards instead of individual human error. Criteria - Blames operator error without addressing missing guardrails - Identifies immediate triggers but ignores underlying architecture - Identifies failure modes and plans concrete monitoring or test coverage - Proposes systemic automated defenses, blast-radius containment, and architectural fixes Legal agreements contract risks Line Rules uncapped indemnity Does this clause expose the company to uncapped indemnification for third-party claims? Criteria - true : The clause creates an indemnification obligation without liability caps. - false : The obligation falls under the standard aggregate liability limit. Customer conversations support transcripts Line Rules unauthorized promise Does this agent response promise an unreleased feature date or custom contract concession? Criteria - true : Agent commits to an unannounced date or non-standard term. - false : Agent refers customer to public docs or defers to account managers. Code review unjustified type assertions Line Rules unjustified type cast Does this line use a type assertion as , non-null assertion , or loose cast to silence a compiler error without proper narrowing or input validation? Criteria - true : Casts away type safety without an upstream type guard, schema validation, or explanatory comment. - false : Type is narrowed safely, or the assertion bridges an external API boundary with runtime checks. Security audit hardcoded credentials and secrets Line Rules hardcoded secret Does this line contain a hardcoded API key, bearer token, private key, or password rather than referencing an environment variable or secret manager? Criteria - true : Line contains a literal credential, private token, or hardcoded secret string. - false : Line references an environment variable, config placeholder, mock test fixture, or public key. Code quality silent error swallowing Line Rules swallowed error Does this catch block or fallback expression silence an unexpected error without diagnostic logging or recovery? Criteria - true : Catches an exception and returns null or an empty default without logging context. - false : Logs the error with context, rethrows, or implements a documented recovery strategy. Git workflow commit message intent Document Rules commit intent Does this commit message or PR description explain the motivation and problem context rather than merely describing code changes? Criteria - Mechanical change list only with no rationale - Mentions the fix with minimal explanation of the problem - Explains the failure trigger, bug condition, and rationale clearly - Details root cause, design tradeoffs considered, and verification evidence See docs/SYNTAX.md https://github.com/lukstei/slop-grader/blob/main/docs/SYNTAX.md for the complete Markdown rule syntax specification and validation reference. Validate ruleset syntax offline without an API key: npx @lukstei/slop-grader@latest --check -r ./my-rules.md Use the create-slop-grader-rules https://github.com/lukstei/slop-grader/blob/main/skills/create-slop-grader-rules/SKILL.md skill to create and validate custom rulesets with an AI assistant. Custom JSON rulesets -r ./my-rules.json are also supported. npx @lukstei/slop-grader@latest -c|--check -r