Releasing a new sentiment classification model.

DEJAN released a sentiment classification model on Hugging Face that sorts text chunks into seven classes from very positive to very negative, scoring 84% of labels exactly right and 95% within one point. The model, small enough to run on a laptop, is available through a public demo, inside DEJAN App under Pages → Sentiment, and via a Hugging Face quick-start script, and was trained on 2,782 pieces generated across 26 formats with 107 pieces each. DEJAN said it is releasing the model "as a gift to the SEO community," targeting reviews, brand mentions, content audits, competitor pages and SERP research.

Releasing a new sentiment classification model. Our new sentiment model is free on Hugging Face. It scores every heading, list item and sentence from very positive to very negative, gets 84% of labels exactly right and 95% within one point, and is built into DEJAN App and a public demo. TL;DR. We are releasing our latest model as a gift to the SEO community. The model classifies text chunks into seven sentiment classes and is small enough to run on a laptop. You can download it from Hugging Face https://huggingface.co/dejanai/sentiment-classifier , try it in our public demo https://dejan.ai/tools/sentiment/ , or use it inside DEJAN App https://app.dejan.ai/property/43?tab=pages&sub=sentiment&run=5 . Three ways to use it - Public demo: dejan.ai/tools/sentiment/ https://dejan.ai/tools/sentiment/ . Paste a page URL or any text. Three free runs, then sign in with Google. Any run can be shared with a link, like this short example https://dejan.ai/tools/sentiment/r/RUSdLE1Q8Du9GQYR . - DEJAN App: Pages → Sentiment in every project, and in the Tools menu. Runs are saved per project, each run has its own address, and owners and editors can share a run publicly. Here is a run in our demo project, on our own Google Store alt text post https://app.dejan.ai/property/43?tab=pages&sub=sentiment&run=5 . - Your own pipeline: the model is on Hugging Face https://huggingface.co/dejanai/sentiment-classifier with a quick-start script that splits a page into chunks and scores each one. Pages → Sentiment in DEJAN App, on the demo project. Green chunks are positive, red chunks are negative, grey chunks are neutral. What a run shows - Overall sentiment: the average score of all chunks, from +3 very positive to −3 very negative , weighted by each chunk's word count. A two-word line such as "Sign in" counts for little, and a long paragraph counts for a lot. - Page statistics: words, characters, chunks, words per chunk, reading time, and a Flesch reading ease score worked out from full sentences only. - Heatmap: the page text with each chunk coloured by its label. - Table: every chunk with its label and score, sortable and exportable as CSV. - Breakdown: how many chunks and words fall in each of the seven labels. - Keywords: the top words and two- and three-word phrases, with common words left out. What you can use it for - Reviews: score your own and competitors' reviews sentence by sentence, and find which product features people praise or complain about. - Brand mentions: measure how your brand is described in articles, forums, social posts and AI answers. - Content audits: find negative or off-tone sentences on landing pages, product pages and service pages. - Competitor pages: compare the tone of your comparison and product pages with the pages that outrank them. - SERP research: see whether the top-ranking content for a query is enthusiastic, balanced or critical. The model reads one chunk at a time, up to about 90 words. A whole page has to be split first, which the demo, DEJAN App and the Hugging Face script all do for you. How we made the training data We wanted chunks labelled on seven points across the formats SEOs work with, so we wrote our own training data. Every piece went through three separate steps: 1. Writer: Google's latest primary model wrote one piece in a given format about a seeded topic. It never saw the labels. 2. Segmenter: a Python script split the piece into chunks while keeping its structure: headings, bullet points, table rows, code lines, bylines and labels, stage directions, and paragraphs split into sentences. 3. Labeller: a second, separate call to the same model labelled every numbered chunk on the seven-point scale, reading each chunk in the context of the whole piece. The writer cycled through 26 formats, 107 pieces each, for 2,782 pieces in total: blog posts, articles, product reviews, testimonials, tweet threads, YouTube and podcast transcripts, meeting notes, technical papers, white papers, essays, web pages, product and service pages, product and service comparisons, news reports, opinion columns, press releases, email newsletters, forum threads, customer support chats, case studies, how-to guides, travel diaries and FAQ pages. Topics were seeded from five groups: products 639 pieces , services 579 , brands 569 , invented people 491 and everyday subjects such as parenting or personal finance 478 . This is the prompt the writer received for the first 546 pieces: Write one {fmt} {length} about {topic}; choose the specific subject yourself. Use markdown and whatever structure suits a real {fmt} headings, bullet points, speaker names, timestamps, and so on . The tone should range from very negative to very positive, as far as a {fmt} naturally allows, moving back and forth the way a real writer's would, with some purely factual passages. Do not move steadily from one end to the other. Write naturally, without exaggerated wording. Output only the {fmt}. Those 546 mixed-tone pieces produced 10,615 neutral chunks but only 960 somewhat positive and 1,080 very negative ones. To balance the labels, we wrote 2,236 more pieces, each aimed at one label, with a hint on how a single sentence carries that tone. For "somewhat positive" the hint was "mild, qualified approval within the same sentence like 'pretty decent', 'fairly good', 'not bad at all', 'a modest improvement' ". After each batch the counts were redone, and the next batch went to the labels furthest behind. The labeller still labelled every chunk on its own merits, so a piece aimed at "negative" could contain positive and neutral chunks. Labelled chunks per label, before and after balancing. Neutral was never targeted, so it stays the largest label. The writer also had habits that would have taught the classifier the wrong thing. It named too many people Marcus and Vance, and used "brilliant" and "maddening" for strong opinions. Each piece was given four random names from a list of 120 first and 120 last names, and a short list of alternative strong words. Stored pieces were cleaned the same way: about 80% of the overused words and of the phrases that leaked from the writing hints were swapped for equivalents of the same strength, with the labels kept. The 83,142 labelled chunks by type. One in five chunks is a heading, label, bullet point or other piece of page structure. Training The model is Google's Gemma 3 270M https://huggingface.co/google/gemma-3-270m with a seven-way classification head. We split the data by piece, 80% for training, 10% for validation and 10% for testing, so no piece has chunks on both sides of a split. That gave 66,085 training chunks, 8,308 validation chunks and 8,749 test chunks. - The loss was weighted by inverse label frequency, so the larger neutral label did not dominate. - The input embedding, 168M of the 268M weights, was frozen so the run fits on an 8 GB GPU. Everything else trained. - Batches of 32 chunks, up to 128 tokens each, at a peak learning rate of 2e-5. - Training stopped after five validation checks in a row without a lower validation loss. That happened at step 2,600, after 7.5 minutes. The checkpoint kept is the one from step 2,100. Validation accuracy during training. Most of the gain comes in the first 200 steps. Results on the test set | Measure | Score | |---|---| | Exact label | 84.0% | | Exact label or one point off | 95.2% | | Macro F1 average over the seven labels | 83.3 | Most mistakes are one step away from the correct label. The weakest label is "somewhat negative" F1 70 : 19% of those chunks were labelled "negative". The strongest are neutral F1 90 , very negative 89 , very positive 88 and somewhat positive 88 . Test set confusion matrix. Each row shows where the chunks with that correct label ended up. F1 score per label on the test set. Scope - English only. May perform in other languages with lower accuracy. - Each chunk is scored on its own, so a sentence such as "And yet it works" can be misread without the sentence before it. - The training and test text was written and labelled by AI. The model has not yet been tested on real-world review datasets. License The model is free to use under the DEJAN link attribution license https://dejan.ai/link-attribution/ . It is built on Google's Gemma, so use is also subject to the Gemma Terms of Use https://ai.google.dev/gemma/terms .