Wire three Hugging Face classifiers into one local pipeline that flags synthetic media before it reaches users.
What you'll build #
A local Python pipeline that routes incoming files to three open-source classifiers — one each for images, audio, and text — and emits a JSON verdict (synthetic
or authentic
) plus a non-zero exit code you can use to gate uploads before they reach your users.
Prerequisites #
- Python 3.10+ (verified with 3.12;
Transformersv5 requires ≥3.10)
transformers
5.15.0,PyTorch2.13.0 (Transformers v5 needs torch ≥2.5), PillowFFmpegon your PATH — the audio pipeline shells out to it to decode files- ~2 GB free disk: the three model checkpoints total roughly 1.2 GB on first download
- No GPU required. Everything here runs on CPU; a GPU just makes it faster.
- macOS or Linux assumed for shell commands; everything works on Windows with the usual venv path changes
1. Set up the environment #
mkdir synthetic-gate && cd synthetic-gate
python3 -m venv .venv && source .venv/bin/activate
pip install "transformers==5.15.0" torch pillow
Install FFmpeg if you don't have it:
brew install ffmpeg
sudo apt-get install -y ffmpeg
2. Meet the three classifiers #
You want one battle-tested checkpoint per modality, all loadable through the same pipeline()
API so the routing code stays trivial:
Images:— a Swin transformer fine-tuned on Wikimedia-vs-SDXL image pairs, ~98% validation accuracy. Labels:Organika/sdxl-detector
artificial
/human
. Heads up: it's licensed CC-BY-NC-3.0, so it's fine for evaluation and internal research but not commercial deployment — for that, swap in the Apache-2.0(EfficientNet-B4, loads viaDafilab/ai-image-detector
timm
instead ofpipeline
).Audio:— wav2vec2-base fine-tuned for spoofed-voice detection, 99.7% accuracy on its eval set, Apache-2.0. Labels:MelodyMachine/Deepfake-audio-detection-V2
fake
/real
. Expects 16 kHz input; the pipeline resamples for you when FFmpeg decodes the file.Text:— a RoBERTa-base classifier from Fakespot (Mozilla) for English text, Apache-2.0. Labels:fakespot-ai/roberta-base-ai-text-detection-v1
Human
/AI
. RoBERTa caps at 512 tokens, so longer inputs must be truncated or chunked.
One thing to internalize before shipping any of this: detector scores are risk signals, not ground truth. All three models were trained against specific generator families, and accuracy drops on generators released after training, on heavily compressed re-uploads, and on out-of-domain content. Route high scores to a human review queue; don't auto-ban on them.
3. Write the routing pipeline #
Save this as detect.py
:
import json
import sys
from pathlib import Path
from transformers import pipeline
MODELS = {
"image": ("image-classification", "Organika/sdxl-detector"),
"audio": ("audio-classification", "MelodyMachine/Deepfake-audio-detection-V2"),
"text": ("text-classification", "fakespot-ai/roberta-base-ai-text-detection-v1"),
}
EXTENSIONS = {
".jpg": "image", ".jpeg": "image", ".png": "image", ".webp": "image",
".wav": "audio", ".mp3": "audio", ".flac": "audio", ".m4a": "audio",
".txt": "text", ".md": "text",
}
SYNTHETIC = {"artificial", "fake", "AI"}
_pipes = {}
def get_pipe(modality):
if modality not in _pipes:
task, model = MODELS[modality]
_pipes[modality] = pipeline(task, model=model)
return _pipes[modality]
def detect(path: Path) -> dict:
modality = EXTENSIONS.get(path.suffix.lower())
if modality is None:
raise SystemExit(f"unsupported extension: {path.suffix}")
pipe = get_pipe(modality)
if modality == "text":
scores = pipe(path.read_text(encoding="utf-8"), truncation=True, top_k=None)
else:
scores = pipe(str(path))
best = max(scores, key=lambda s: s["score"])
return {
"file": path.name,
"modality": modality,
"verdict": "synthetic" if best["label"] in SYNTHETIC else "authentic",
"label": best["label"],
"score": round(best["score"], 4),
"model": MODELS[modality][1],
}
def main():
results = [detect(Path(p)) for p in sys.argv[1:]]
print(json.dumps(results, indent=2))
if any(r["verdict"] == "synthetic" and r["score"] >= 0.90 for r in results):
sys.exit(1)
if __name__ == "__main__":
main()
The pipelines are cached in _pipes
so each model loads once per process — model load dominates runtime, so in production you'd keep this warm behind a small HTTP service rather than spawning a process per file. The 0.90 gate threshold is a starting point; tune it against your own false-positive tolerance.
4. Feed it real inputs #
Collect one known-real and one known-synthetic sample per modality: a photo off your phone plus an image from any diffusion model; a voice memo plus a clip from a TTS service; a paragraph you wrote plus one straight out of a chatbot pasted into llm.txt
.
python detect.py phone-photo.jpg diffusion-render.png voicememo.wav llm.txt
First run downloads all three checkpoints from the Hugging Face Hub (~1.2 GB) into ~/.cache/huggingface
; later runs load from cache and classify each file in a few hundred milliseconds to a few seconds on CPU.
Verify it works #
You should see one JSON object per file, with the synthetic samples flagged:
[
{
"file": "phone-photo.jpg",
"modality": "image",
"verdict": "authentic",
"label": "human",
"score": 0.9971,
"model": "Organika/sdxl-detector"
},
{
"file": "diffusion-render.png",
"modality": "image",
"verdict": "synthetic",
"label": "artificial",
"score": 0.9998,
"model": "Organika/sdxl-detector"
},
{
"file": "voicememo.wav",
"modality": "audio",
"verdict": "authentic",
"label": "real",
"score": 0.9842,
"model": "MelodyMachine/Deepfake-audio-detection-V2"
},
{
"file": "llm.txt",
"modality": "text",
"verdict": "synthetic",
"label": "AI",
"score": 0.9564,
"model": "fakespot-ai/roberta-base-ai-text-detection-v1"
}
]
Your exact scores will differ. Then confirm the gate fired:
echo $? # → 1, because at least one file scored ≥0.90 synthetic
Run it again with only the real samples and echo $?
should print 0
.
Troubleshooting #
** ValueError: ffmpeg was not found but is required to load audio files from filename** — the audio pipeline decodes files by invoking FFmpeg. Install it (
brew install ffmpeg
/ apt-get install -y ffmpeg
) and make sure it's on the PATH of the process running the script, which is easy to miss inside slim Docker images and systemd units.** Token indices sequence length is longer than the specified maximum sequence length for this model (743 > 512) followed by IndexError: index out of range in self** — you dropped
truncation=True
from the text call. Restore it; and remember truncation means only the first 512 tokens are scored, so chunk long documents and score each chunk if you care about the tail.** OSError: We couldn't connect to 'https://huggingface.co' to load this file** — first run needs Hub access. If your servers are egress-restricted, pre-download on a machine that isn't (
hf download Organika/sdxl-detector
, once per model — the hf
CLI ships with huggingface_hub
), copy ~/.cache/huggingface
across, and set HF_HUB_OFFLINE=1
in production.Everything scores authentic, even known fakes — check the input actually reached the model intact. Screenshots of AI images, re-encoded thumbnails, and voice notes transcoded by a chat app strip the artifacts detectors key on. Test with the original file, and treat detector evasion via re-encoding as a known limitation, not a bug in your wiring.
Next steps #
Single-model verdicts are the weakest form of this pipeline. To harden it: ensemble two or more detectors per modality and require agreement before flagging; add provenance checks with C2PA Content Credentials so signed-authentic media can skip ML scoring entirely; and benchmark on public datasets (CIFAKE and GenImage for images, ASVspoof for audio) before trusting any threshold. When you outgrow the CLI, wrap get_pipe
/detect
in a FastAPI service with the pipelines loaded at startup, and log every score — your own traffic distribution is the fine-tuning dataset that will eventually beat any off-the-shelf checkpoint.
Sources & further reading #
Organika/sdxl-detector model card— huggingface.co - MelodyMachine/Deepfake-audio-detection-V2 model card— huggingface.co - fakespot-ai/roberta-base-ai-text-detection-v1 model card— huggingface.co - Transformers Pipelines documentation— huggingface.co - Hugging Face hf CLI guide— huggingface.co - transformers on PyPI— pypi.org
Emeka Okafor· Security Editor
Emeka has spent over a decade tracking threat actors, vulnerability disclosures, and the evolving landscape of application security, bringing a sharp continent-spanning perspective to his reporting. He's known for translating dense CVE advisories into clear, actionable context that developers and security teams alike actually read.
Discussion 0 #
No comments yet
Be the first to weigh in.