Named Entity Recognition: fine-tune or create new model?

Another option now: a model that reads the entity types from its input, so adding PRICE and PRODUCT means describing them instead of relabelling and retraining. I built one, ReDefiNER (GitHub - Jungroo-AI/redefiner: A 0.8B model that extracts the entities you define in a plain-English rulebook. It runs on a CPU and is free under Apache 2.0. · GitHub, Apache 2.0, 0.8B, runs on a CPU):

from redefiner import Tagger

tagger = Tagger.from_pretrained("jungroo-ai/redefiner-0.8b")

rulebook = """[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] PRODUCT
[DESCRIPTION] A product line or model name.

[TYPE] PRICE
[DESCRIPTION] A price or a price limit.
PR1 A price limit keeps its limit word in the span: "under $50", "below 30 dollars", "less than $200".
"""

tagger.extract("your text here", rulebook)

The README shows that one added line changing “$120” to “under $120” with nothing retrained. You can keep PERSON, ORG and the rest by listing them too. If you have labelled data, use it as a test set: on public benchmarks it scores a few points below large API models, so check it on your own text.