My fine-tuned model scored 100%... The benchmark was lying
A developer fine-tuned Mistral 7B on a laptop to detect personal data in log lines and support messages, achieving a perfect 100% score on an initial test set. However, after rebuilding the test set from real public data…