Classifying files based solely on presence of IDs, numbers and other regex patterns, from experience, leads to much false positives and negatives. A technical manual document with a 9 digit number could trigger a SSN pattern check and get wrongly classified as having personal data /Confidential.
Personal data on a HR document would be treated differently from personal data on a customer list.
Would identifying the document category (tech manual, customer list, employee record) be a good additional layer to classifier logic? I'm trying to build a non-AI, light model and embedding based classifier that can semantically understand documents, and like opinions on how useful such a thing would be.
Comments URL: [https://news.ycombinator.com/item?id=49280386](https://news.ycombinator.com/item?id=49280386)
Points: 1
source & further reading
news.ycombinator.com — original article
Ask HN: Is AI code verification becoming your main bottleneck?
Show HN: A better, free client for iMessage power-users
Show HN: Reallyfrom.me – vouch that your message is from you