Originally published on tamiz.pro.
Modern large language models (LLMs) produce text indistinguishable from human writing in many cases. As these systems proliferate, detecting synthetic content has become critical for content moderation, academic integrity, and information security. While deep learning dominates current detection research, classical machine learning methods remain valuable for their interpretability, low computational cost, and effectiveness in constrained environments.
Classical machine learning approaches offer three key advantages:
These properties make classical approaches ideal for edge deployments or as complementary systems to deep learning detectors.
Successful classical detection relies on extracting discriminative linguistic features. Key categories include:
from collections import Counter
def extract_ngrams(text, n=3):
tokens = text.lower().split()
return [' '.join(tokens[i:i+n]) for i in range(len(tokens)-n+1)]
human_ngrams = extract_ngrams("The quick brown fox jumps over the lazy dog")
ai_ngrams = extract_ngrams("The rapid brown fox leaps above the dormant canine")
LLMs often generate semantically coherent but statistically anomalous n-gram patterns compared to natural writing.
| Feature | Human Text | AI Text |
|---|---|---|
| Sentence Complexity | Higher variance | More uniform |
| Passive Voice Rate | 15-20% | 5-8% |
| Discourse Markers | 3-5 per 100 words | 0.5-1 per 100 words |
These metrics can be calculated using libraries like syntok
or nltk
.
LLM outputs frequently display:
These results approach but don't yet surpass deep learning models (93-97% F1), but maintain advantages in edge cases.
LLMs evolve rapidly while classical models require retraining. Solution: Use adaptive training pipelines that ingest new human/LLM samples weekly.
Manual feature creation is labor-intensive. Consider automated feature selection:
from sklearn.feature_selection import SelectKBest
selector = SelectKBest(score_func=chi2, k=500) # Select top 500 features
X_selected = selector.fit_transform(X, y)
Use SHAP values to visualize feature importance:
explainer = shap.LinearExplainer(model)
shap_values = explainer.shap_values(X_test)
shap.summary_plot(shap_values, X_test)
This helps validate that models are learning meaningful patterns (e.g., detecting unnatural conjunction usage).
While deep learning will dominate cutting-edge detection research, classical methods provide essential capabilities in deployment scenarios with limited compute resources. The best detection systems of the future will likely integrate both paradigms, using classical models for real-time filtering and deep learning for final classification.