# Detect AI-Generated Images, Audio, and Text with Open-Source Models > Source: > Published: 2026-08-12 11:40:46+00:00 # Detect AI-Generated Images, Audio, and Text with Open-Source Models Wire three Hugging Face classifiers into one local pipeline that flags synthetic media before it reaches users. [Emeka Okafor](https://sourcefeed.dev/u/emeka_okafor) ## What you'll build A local Python pipeline that routes incoming files to three open-source classifiers — one each for images, audio, and text — and emits a JSON verdict (`synthetic` or `authentic` ) plus a non-zero exit code you can use to gate uploads before they reach your users. ## Prerequisites - Python 3.10+ (verified with 3.12; [Transformers](https://huggingface.co/docs/transformers)v5 requires ≥3.10) `transformers` 5.15.0,[PyTorch](https://pytorch.org)2.13.0 (Transformers v5 needs torch ≥2.5), Pillow[FFmpeg](https://ffmpeg.org)on your PATH — the audio pipeline shells out to it to decode files- ~2 GB free disk: the three model checkpoints total roughly 1.2 GB on first download - No GPU required. Everything here runs on CPU; a GPU just makes it faster. - macOS or Linux assumed for shell commands; everything works on Windows with the usual venv path changes ## 1. Set up the environment ``` mkdir synthetic-gate && cd synthetic-gate python3 -m venv .venv && source .venv/bin/activate pip install "transformers==5.15.0" torch pillow ``` Install FFmpeg if you don't have it: ``` # macOS brew install ffmpeg # Debian/Ubuntu sudo apt-get install -y ffmpeg ``` ## 2. Meet the three classifiers You want one battle-tested checkpoint per modality, all loadable through the same `pipeline()` API so the routing code stays trivial: **Images:**— a Swin transformer fine-tuned on Wikimedia-vs-SDXL image pairs, ~98% validation accuracy. Labels:`Organika/sdxl-detector` `artificial` /`human` . Heads up: it's licensed CC-BY-NC-3.0, so it's fine for evaluation and internal research but not commercial deployment — for that, swap in the Apache-2.0(EfficientNet-B4, loads via`Dafilab/ai-image-detector` `timm` instead of`pipeline` ).**Audio:**— wav2vec2-base fine-tuned for spoofed-voice detection, 99.7% accuracy on its eval set, Apache-2.0. Labels:`MelodyMachine/Deepfake-audio-detection-V2` `fake` /`real` . Expects 16 kHz input; the pipeline resamples for you when FFmpeg decodes the file.**Text:**— a RoBERTa-base classifier from Fakespot (Mozilla) for English text, Apache-2.0. Labels:`fakespot-ai/roberta-base-ai-text-detection-v1` `Human` /`AI` . RoBERTa caps at 512 tokens, so longer inputs must be truncated or chunked. One thing to internalize before shipping any of this: detector scores are risk signals, not ground truth. All three models were trained against specific generator families, and accuracy drops on generators released after training, on heavily compressed re-uploads, and on out-of-domain content. Route high scores to a human review queue; don't auto-ban on them. ## 3. Write the routing pipeline Save this as `detect.py` : ``` python import json import sys from pathlib import Path from transformers import pipeline MODELS = { "image": ("image-classification", "Organika/sdxl-detector"), "audio": ("audio-classification", "MelodyMachine/Deepfake-audio-detection-V2"), "text": ("text-classification", "fakespot-ai/roberta-base-ai-text-detection-v1"), } EXTENSIONS = { ".jpg": "image", ".jpeg": "image", ".png": "image", ".webp": "image", ".wav": "audio", ".mp3": "audio", ".flac": "audio", ".m4a": "audio", ".txt": "text", ".md": "text", } # Each model names its labels differently; anything in this set means "machine-made". SYNTHETIC = {"artificial", "fake", "AI"} _pipes = {} def get_pipe(modality): if modality not in _pipes: task, model = MODELS[modality] _pipes[modality] = pipeline(task, model=model) return _pipes[modality] def detect(path: Path) -> dict: modality = EXTENSIONS.get(path.suffix.lower()) if modality is None: raise SystemExit(f"unsupported extension: {path.suffix}") pipe = get_pipe(modality) if modality == "text": # truncation is forwarded to the tokenizer; RoBERTa can't take >512 tokens scores = pipe(path.read_text(encoding="utf-8"), truncation=True, top_k=None) else: scores = pipe(str(path)) best = max(scores, key=lambda s: s["score"]) return { "file": path.name, "modality": modality, "verdict": "synthetic" if best["label"] in SYNTHETIC else "authentic", "label": best["label"], "score": round(best["score"], 4), "model": MODELS[modality][1], } def main(): results = [detect(Path(p)) for p in sys.argv[1:]] print(json.dumps(results, indent=2)) # exit 1 if anything is confidently synthetic, so CI or an upload hook can gate on it if any(r["verdict"] == "synthetic" and r["score"] >= 0.90 for r in results): sys.exit(1) if __name__ == "__main__": main() ``` The pipelines are cached in `_pipes` so each model loads once per process — model load dominates runtime, so in production you'd keep this warm behind a small HTTP service rather than spawning a process per file. The 0.90 gate threshold is a starting point; tune it against your own false-positive tolerance. ## 4. Feed it real inputs Collect one known-real and one known-synthetic sample per modality: a photo off your phone plus an image from any diffusion model; a voice memo plus a clip from a TTS service; a paragraph you wrote plus one straight out of a chatbot pasted into `llm.txt` . ``` python detect.py phone-photo.jpg diffusion-render.png voicememo.wav llm.txt ``` First run downloads all three checkpoints from the Hugging Face Hub (~1.2 GB) into `~/.cache/huggingface` ; later runs load from cache and classify each file in a few hundred milliseconds to a few seconds on CPU. ## Verify it works You should see one JSON object per file, with the synthetic samples flagged: ``` [ { "file": "phone-photo.jpg", "modality": "image", "verdict": "authentic", "label": "human", "score": 0.9971, "model": "Organika/sdxl-detector" }, { "file": "diffusion-render.png", "modality": "image", "verdict": "synthetic", "label": "artificial", "score": 0.9998, "model": "Organika/sdxl-detector" }, { "file": "voicememo.wav", "modality": "audio", "verdict": "authentic", "label": "real", "score": 0.9842, "model": "MelodyMachine/Deepfake-audio-detection-V2" }, { "file": "llm.txt", "modality": "text", "verdict": "synthetic", "label": "AI", "score": 0.9564, "model": "fakespot-ai/roberta-base-ai-text-detection-v1" } ] ``` Your exact scores will differ. Then confirm the gate fired: ``` echo $? # → 1, because at least one file scored ≥0.90 synthetic ``` Run it again with only the real samples and `echo $?` should print `0` . ## Troubleshooting ** ValueError: ffmpeg was not found but is required to load audio files from filename** — the audio pipeline decodes files by invoking FFmpeg. Install it ( `brew install ffmpeg` / `apt-get install -y ffmpeg` ) and make sure it's on the PATH of the process running the script, which is easy to miss inside slim Docker images and systemd units.** Token indices sequence length is longer than the specified maximum sequence length for this model (743 > 512) followed by IndexError: index out of range in self** — you dropped `truncation=True` from the text call. Restore it; and remember truncation means only the first 512 tokens are scored, so chunk long documents and score each chunk if you care about the tail.** OSError: We couldn't connect to 'https://huggingface.co' to load this file** — first run needs Hub access. If your servers are egress-restricted, pre-download on a machine that isn't ( `hf download Organika/sdxl-detector` , once per model — the `hf` CLI ships with `huggingface_hub` ), copy `~/.cache/huggingface` across, and set `HF_HUB_OFFLINE=1` in production.**Everything scores authentic, even known fakes** — check the input actually reached the model intact. Screenshots of AI images, re-encoded thumbnails, and voice notes transcoded by a chat app strip the artifacts detectors key on. Test with the original file, and treat detector evasion via re-encoding as a known limitation, not a bug in your wiring. ## Next steps Single-model verdicts are the weakest form of this pipeline. To harden it: ensemble two or more detectors per modality and require agreement before flagging; add provenance checks with [C2PA Content Credentials](https://c2pa.org) so signed-authentic media can skip ML scoring entirely; and benchmark on public datasets (CIFAKE and GenImage for images, ASVspoof for audio) before trusting any threshold. When you outgrow the CLI, wrap `get_pipe` /`detect` in a FastAPI service with the pipelines loaded at startup, and log every score — your own traffic distribution is the fine-tuning dataset that will eventually beat any off-the-shelf checkpoint. ## Sources & further reading - [Organika/sdxl-detector model card](https://huggingface.co/Organika/sdxl-detector)— huggingface.co - [MelodyMachine/Deepfake-audio-detection-V2 model card](https://huggingface.co/MelodyMachine/Deepfake-audio-detection-V2)— huggingface.co - [fakespot-ai/roberta-base-ai-text-detection-v1 model card](https://huggingface.co/fakespot-ai/roberta-base-ai-text-detection-v1)— huggingface.co - [Transformers Pipelines documentation](https://huggingface.co/docs/transformers/main_classes/pipelines)— huggingface.co - [Hugging Face hf CLI guide](https://huggingface.co/docs/huggingface_hub/guides/cli)— huggingface.co - [transformers on PyPI](https://pypi.org/project/transformers/)— pypi.org [Emeka Okafor](https://sourcefeed.dev/u/emeka_okafor)· Security Editor Emeka has spent over a decade tracking threat actors, vulnerability disclosures, and the evolving landscape of application security, bringing a sharp continent-spanning perspective to his reporting. He's known for translating dense CVE advisories into clear, actionable context that developers and security teams alike actually read. ## Discussion 0 No comments yet Be the first to weigh in.