A 280-line Python script that sends a folder of student submissions to the Proofademic AI Detector API, handles rate limits and async jobs, and produces a class summary PDF with sentence-level highlights.
Image generated by AI
Checking one essay for AI writing is easy. Checking 30 of them the night before grades are due is where it gets painful: copy, paste, wait, screenshot, repeat. This tutorial automates the whole thing.
You'll build a Python script that takes a folder of .txt and .docx submissions, scans every one with the Proofademic AI Detector API, and writes two files: a CSV you can drop into a spreadsheet and a PDF report with a ranked class summary plus one page per student, where every sentence is highlighted as AI-generated or original.
I ran it on a sample class of eight submissions. It caught all five ChatGPT essays, cleared both human essays at under 2% AI, and found a ChatGPT closing paragraph hidden inside an otherwise human essay that the overall score alone would have missed.
class_submissions/ reports/
├── 01_ava_brooks.txt ├── class_report.pdf (summary + one page per student)
├── 02_daniel_kim.txt ──▶ ├── results.csv (one row per student)
├── 04_marcus_reed.docx └── raw/ (cached API responses)
└── ...
The script has four parts:
| Part | What it does |
|---|---|
Reads .txt ,.md and.docx files and turns07_priya_shah.txt into "Priya Shah" |
|
| API client | Calls the Proofademic AI Detector with retries for 429 and5xx responses |
| Batch scanner | Submits async jobs, keeps them under your plan's pending limit, polls with backoff, caches results |
| Reporter | Writes a CSV and a PDF with a color-coded summary and sentence highlights |
Get an API key. Sign in to the Proofademic Developer Portal, open API Keys, and click Create API Key. Give it a name, switch on AI Detector under Model Access, and pick an expiration. The full key (it starts with prfd_) is shown only once, so copy it straight into an environment variable.
The Create API Key form: give the key a name, turn on AI Detector and pick an expiration.
New accounts get a free trial with 2,000 credits for 5 days, no card required. The AI Detector costs 1 credit per word, so the trial covers about 2,000 words of student writing. That's enough to follow this tutorial with a small class. Paid plans start at 300,000 credits a month.
Install the dependencies:
pip install requests reportlab python-docx
export PROOFADEMIC_API_KEY="prfd_your_key_here"
A few API details shape the design of the script, all from the Proofademic API docs:
| Detail | Value | Why it matters |
|---|---|---|
| Endpoint | POST /api/detector/ |
One call per submission |
| Auth | X-API-Key header |
Keep the key in an env var, never in code |
| Sync rate limit | 10 requests per minute | Too slow for a full class |
| Async mode | "poll": true , thenGET /api/jobs/{task_id}/ |
Not subject to the per-minute limit |
| Pending async jobs | 2 on Trial, 5 on 300K, up to 50 on 25M | The batch loop must respect this |
| Minimum length | 5 words | Shorter texts return 400 |
| Result expiry | 1 hour | Save results as soon as they arrive |
Name each file with a number and the student's name, like 01_ava_brooks.txt or 04_marcus_reed.docx. The reads plain text and Word files and derives a readable name from the filename.
def read_submission(path: Path) -> str:
if path.suffix.lower() == ".docx":
import docx # python-docx
return "\n\n".join(p.text for p in docx.Document(path).paragraphs if p.text.strip())
return path.read_text(encoding="utf-8", errors="ignore")
def student_name(path: Path) -> str:
parts = [p for p in path.stem.split("_") if not p.isdigit()]
return " ".join(p.capitalize() for p in parts) or path.stem
def load_class(folder: Path) -> list[dict]:
subs = []
for path in sorted(folder.iterdir()):
if path.suffix.lower() not in {".txt", ".md", ".docx"}:
continue
text = read_submission(path).strip()
subs.append({"file": path.name, "student": student_name(path),
"text": text, "words": len(text.split())})
return subs
Paragraphs from Word files are joined with blank lines. That matters later, because the PDF uses those breaks to rebuild each student's paragraphs.
The client is a thin wrapper around requests.Session. It sends the key on every request and retries 429 Too Many Requests and 5xx errors, using the Retry-After header when the API sends one.
BASE_URL = os.environ.get("PROOFADEMIC_BASE_URL", "https://developer-portal.proofademic.ai/api")
class Proofademic:
def __init__(self, api_key: str):
self.s = requests.Session()
self.s.headers.update({"X-API-Key": api_key, "Content-Type": "application/json"})
def _request(self, method: str, path: str, **kw) -> requests.Response:
for attempt in range(6):
r = self.s.request(method, f"{BASE_URL}{path}", timeout=60, **kw)
if r.status_code == 429 or r.status_code >= 500:
wait = int(r.headers.get("Retry-After", 2 ** attempt))
print(f" {r.status_code} from API, retrying in {wait}s")
time.sleep(wait)
continue
return r
r.raise_for_status()
return r
def submit(self, text: str) -> dict:
r = self._request("POST", "/detector/", json={"content": text, "poll": True})
if r.status_code not in (200, 202):
raise RuntimeError(f"{r.status_code}: {r.text}")
return r.json()
def job(self, task_id: str) -> dict:
r = self._request("GET", f"/jobs/{task_id}/")
if r.status_code != 200:
raise RuntimeError(f"{r.status_code}: {r.text}")
return r.json()
Errors like 400, 401 and 403 aren't retried on purpose. They mean something you need to fix: a text under 5 words, a bad key, a key without the AI Detector scope, or not enough credits.
Here's what a finished job looks like. The items array is the useful part for teachers, because it scores every sentence:
{
"status": "success",
"result": "ai",
"ai_score": 0.9436,
"items": [
{
"text": "It is not just about getting good grades or passing exams.",
"prediction": "ai-generated",
"ai_score": 0.9998
}
],
"word_count": 143,
"credits_remaining": 1857
}
The synchronous endpoint allows 10 requests a minute. That's fine for one essay, but it would take three minutes just to send a class of 30, and any mistake costs you a rate-limit wait.
Async mode avoids that. You submit with "poll": true, get a task_id back immediately with HTTP 202, then poll /api/jobs/{task_id}/ until the job is done. The only limit is how many jobs can be pending at once, which depends on your plan.
def scan_all(api: Proofademic, subs: list[dict], cache_dir: Path, max_pending: int) -> None:
cache_dir.mkdir(parents=True, exist_ok=True)
queue, pending = [], {}
for sub in subs:
cached = cache_dir / f"{sub['file']}.json"
if sub["words"] < MIN_WORDS:
sub["error"] = f"too short ({sub['words']} words)"
elif cached.exists():
sub["result"] = json.loads(cached.read_text())
else:
queue.append(sub)
while queue or pending:
while queue and len(pending) < max_pending:
sub = queue.pop(0)
job = api.submit(sub["text"])
if job["status"] == "success": # sync fallback
sub["result"] = job
(cache_dir / f"{sub['file']}.json").write_text(json.dumps(job))
else:
pending[job["task_id"]] = {"sub": sub, "wait": 2, "next": time.time() + 2}
print(f" submitted {sub['file']} ({sub['words']} words), "
f"credits left: {job.get('credits_remaining')}")
time.sleep(1)
for task_id, p in list(pending.items()):
if time.time() < p["next"]:
continue
job = api.job(task_id)
if job["status"] == "processing":
p["wait"] = min(p["wait"] * 2, 30) # 2s, 4s, 8s ... capped at 30s
p["next"] = time.time() + p["wait"]
continue
sub = p["sub"]
if job["status"] == "success":
sub["result"] = job
(cache_dir / f"{sub['file']}.json").write_text(json.dumps(job))
print(f" done {sub['file']}: {job['ai_score']:.0%} AI")
else:
sub["error"] = job.get("error", "failed")
print(f" failed {sub['file']}: {sub['error']}")
del pending[task_id]
Three details in this loop save you money and headaches:
reports/raw/. If the script crashes halfway or you rerun it to tweak the PDF, finished submissions aren't scanned again, so you aren't charged twice.max_pending defaults to 2, the Trial limit. On a paid plan, raise it with --max-pending 5 (or higher) and the class finishes faster.
The score bands come straight from the Proofademic docs: under 0.3 is human, 0.3 to 0.5 likely human, 0.5 to 0.7 likely AI, and 0.7 and up AI-generated.
def band(score: float) -> tuple[str, str]:
if score < 0.3:
return "Human", "#2e7d32"
if score < 0.5:
return "Likely human", "#7cb342"
if score < 0.7:
return "Likely AI", "#f9a825"
return "AI-generated", "#c62828"
def sentence_color(item: dict) -> str:
return "#ffcdd2" if item["prediction"] == "ai-generated" else "#dcedc8"
The PDF uses ReportLab's platypus layer. The first page is a summary table ranked by AI score. Then each student gets a page with their text, rebuilt into the original paragraphs, where each sentence gets a red or green background:
for item in r.get("items") or []:
start = s["text"].find(item["text"], pos)
if start > pos and "\n" in s["text"][pos:start] and spans:
paragraphs.append(spans) # a line break means a new paragraph
spans = []
pos = max(pos, start + len(item["text"]))
spans.append(f"<span backColor='{sentence_color(item)}'>{escape(item['text'])}</span>")
One rule in the summary table turned out to matter more than I expected. If a submission's overall verdict is human but some sentences were still flagged, the "AI sentences" cell turns red:
if ai and r["ai_score"] < 0.5:
style.append(("TEXTCOLOR", (4, i), (4, i), colors.HexColor("#c62828")))
style.append(("FONTNAME", (4, i), (4, i), "Helvetica-Bold"))
You'll see why in the results.
Check how many credits a run will use before spending any:
$ python scan_class.py class_submissions --dry-run
8 submissions, 1196 words, about 1196 credits
Then run the real scan:
python scan_class.py class_submissions \
--title "Period 3: What does education mean to you?" \
--max-pending 2
The API's word count can run slightly above a plain split() count, so leave a little room. My eight submissions were estimated at 1,196 words and used 1,208 credits.
To test the script, I built a sample class of eight submissions to the prompt "What does education mean to you?":
.docx file to test the Word .
Every ChatGPT essay was flagged, both human essays passed, and the mixed essay was caught at the sentence level. Lower AI scores are better for human writing.
| Student | Who actually wrote it | AI score | AI sentences | Verdict |
|---|---|---|---|---|
| Ava Brooks | ChatGPT | 94.4% | 7 of 8 | AI-generated |
| Lena Ortiz | ChatGPT | 75.1% | 6 of 9 | AI-generated |
| Owen Patel | ChatGPT | 73.8% | 5 of 8 | AI-generated |
| Marcus Reed | ChatGPT (.docx) | 73.4% | 6 of 10 | AI-generated |
| Sam Carter | ChatGPT | 62.8% | 5 of 10 | Likely AI |
| Priya Shah | Human + ChatGPT ending | 24.6% | 2 of 12 | Human |
| Noor Haddad | Human | 1.7% | 0 of 4 | Human |
| Daniel Kim | Human | 0.4% | 0 of 8 | Human |
Here's the first page of the PDF the script produced:
Page 1 of class_report.pdf: the ranked class summary. Priya's sentence count is red because her "human" essay still has flagged sentences.
Every ChatGPT essay was caught. All five scored above 50%, with 5 to 7 sentences flagged in each. Even Sam Carter's essay, the most personal-sounding of the five, came back as likely AI.
Both human essays were cleared. Douglass scored 0.4% and Washington 1.7%, with zero flagged sentences between them.
The mixed essay is the reason sentence-level results matter. Priya Shah's overall score was 24.6%, which on its own reads as human. But the report shows that the two flagged sentences are both in the ChatGPT closing paragraph, while every sentence of the Keller passage is green:
Priya Shah's page: the Keller passage is all green, and the two flagged sentences sit in the ChatGPT closing paragraph.
If you only looked at a single score per student, you'd have missed it. That's why the summary turns the sentence count red whenever a "human" submission still has flagged sentences.
Each AI-flagged essay gets a page like this one, so you can see exactly which lines triggered the score:
Ava Brooks's page: 7 of 8 sentences flagged as AI-generated.
A detector score is evidence, not a verdict. Before acting on any result:
The PDF says this at the top of every report, so whoever reads it later gets the same reminder.
Also keep student data in mind. Store the API key in an environment variable, keep the reports/ folder somewhere access-controlled, and delete the raw/ cache when the grading period ends.
class_submissions folder.callback_url and a webhook_secret, and let Proofademic POST results to your server as each job finishes.results.csv and compare scores across assignments to spot sudden changes.pypdf and attach it to feedback.
With about 280 lines of Python, you can turn "paste 30 essays into a detector one at a time" into a single command that produces a ranked class summary and a sentence-level report for every student.
The design choices that matter most are the ones the API pushes you toward: use async mode with a pending-job limit instead of hammering the sync endpoint, cache every result so reruns are free, and put sentence-level results in front of the reader, not just one score. In my test, that last choice is what caught a ChatGPT paragraph hidden in an otherwise human essay.
Grab a Proofademic API key, point the script at a folder, and run --dry-run first to see what it will cost.
"""
scan_class.py: batch-scan a folder of student submissions with the
Proofademic AI Detector API and export a PDF report plus a CSV.
Usage:
export PROOFADEMIC_API_KEY="prfd_..."
python scan_class.py class_submissions --title "Period 3: What does education mean to you?"
Requirements:
pip install requests reportlab python-docx
"""
import argparse
import csv
import json
import os
import sys
import time
from datetime import date
from pathlib import Path
from xml.sax.saxutils import escape
import requests
from reportlab.lib import colors
from reportlab.lib.pagesizes import A4
from reportlab.lib.styles import ParagraphStyle, getSampleStyleSheet
from reportlab.lib.units import mm
from reportlab.platypus import (PageBreak, Paragraph, SimpleDocTemplate,
Spacer, Table, TableStyle)
BASE_URL = os.environ.get("PROOFADEMIC_BASE_URL", "https://developer-portal.proofademic.ai/api")
MIN_WORDS = 5 # the API rejects shorter texts with a 400
def read_submission(path: Path) -> str:
if path.suffix.lower() == ".docx":
import docx # python-docx
return "\n\n".join(p.text for p in docx.Document(path).paragraphs if p.text.strip())
return path.read_text(encoding="utf-8", errors="ignore")
def student_name(path: Path) -> str:
parts = [p for p in path.stem.split("_") if not p.isdigit()]
return " ".join(p.capitalize() for p in parts) or path.stem
def load_class(folder: Path) -> list[dict]:
subs = []
for path in sorted(folder.iterdir()):
if path.suffix.lower() not in {".txt", ".md", ".docx"}:
continue
text = read_submission(path).strip()
subs.append({"file": path.name, "student": student_name(path),
"text": text, "words": len(text.split())})
return subs
class Proofademic:
def __init__(self, api_key: str):
self.s = requests.Session()
self.s.headers.update({"X-API-Key": api_key, "Content-Type": "application/json"})
def _request(self, method: str, path: str, **kw) -> requests.Response:
for attempt in range(6):
r = self.s.request(method, f"{BASE_URL}{path}", timeout=60, **kw)
if r.status_code == 429 or r.status_code >= 500:
wait = int(r.headers.get("Retry-After", 2 ** attempt))
print(f" {r.status_code} from API, retrying in {wait}s")
time.sleep(wait)
continue
return r
r.raise_for_status()
return r
def submit(self, text: str) -> dict:
r = self._request("POST", "/detector/", json={"content": text, "poll": True})
if r.status_code not in (200, 202):
raise RuntimeError(f"{r.status_code}: {r.text}")
return r.json()
def job(self, task_id: str) -> dict:
r = self._request("GET", f"/jobs/{task_id}/")
if r.status_code != 200:
raise RuntimeError(f"{r.status_code}: {r.text}")
return r.json()
def scan_all(api: Proofademic, subs: list[dict], cache_dir: Path, max_pending: int) -> None:
cache_dir.mkdir(parents=True, exist_ok=True)
queue, pending = [], {}
for sub in subs:
cached = cache_dir / f"{sub['file']}.json"
if sub["words"] < MIN_WORDS:
sub["error"] = f"too short ({sub['words']} words)"
elif cached.exists():
sub["result"] = json.loads(cached.read_text())
else:
queue.append(sub)
while queue or pending:
while queue and len(pending) < max_pending:
sub = queue.pop(0)
job = api.submit(sub["text"])
if job["status"] == "success": # sync fallback
sub["result"] = job
(cache_dir / f"{sub['file']}.json").write_text(json.dumps(job))
else:
pending[job["task_id"]] = {"sub": sub, "wait": 2, "next": time.time() + 2}
print(f" submitted {sub['file']} ({sub['words']} words), "
f"credits left: {job.get('credits_remaining')}")
time.sleep(1)
for task_id, p in list(pending.items()):
if time.time() < p["next"]:
continue
job = api.job(task_id)
if job["status"] == "processing":
p["wait"] = min(p["wait"] * 2, 30) # 2s, 4s, 8s ... capped at 30s
p["next"] = time.time() + p["wait"]
continue
sub = p["sub"]
if job["status"] == "success":
sub["result"] = job
(cache_dir / f"{sub['file']}.json").write_text(json.dumps(job))
print(f" done {sub['file']}: {job['ai_score']:.0%} AI")
else:
sub["error"] = job.get("error", "failed")
print(f" failed {sub['file']}: {sub['error']}")
del pending[task_id]
def band(score: float) -> tuple[str, str]:
if score < 0.3:
return "Human", "#2e7d32"
if score < 0.5:
return "Likely human", "#7cb342"
if score < 0.7:
return "Likely AI", "#f9a825"
return "AI-generated", "#c62828"
def sentence_color(item: dict) -> str:
return "#ffcdd2" if item["prediction"] == "ai-generated" else "#dcedc8"
def write_csv(subs: list[dict], path: Path) -> None:
with path.open("w", newline="") as f:
w = csv.writer(f)
w.writerow(["student", "file", "words", "ai_score", "verdict", "ai_sentences", "total_sentences", "error"])
for s in subs:
r = s.get("result")
if r:
items = r.get("items") or []
ai = sum(1 for i in items if i["prediction"] == "ai-generated")
w.writerow([s["student"], s["file"], s["words"], round(r["ai_score"], 4),
band(r["ai_score"])[0], ai, len(items), ""])
else:
w.writerow([s["student"], s["file"], s["words"], "", "", "", "", s.get("error", "")])
def write_pdf(subs: list[dict], path: Path, title: str) -> None:
styles = getSampleStyleSheet()
small = ParagraphStyle("small", parent=styles["Normal"], fontSize=8.5, textColor=colors.HexColor("#555555"))
body = ParagraphStyle("body", parent=styles["Normal"], fontSize=10, leading=15)
story = [Paragraph(escape(title), styles["Title"]),
Paragraph(f"AI writing scan, {date.today():%B %d, %Y}. Scores from the Proofademic AI Detector API. "
"Treat a high score as a reason to talk with the student, not as proof on its own.", small),
Spacer(1, 6 * mm)]
rows = [["Student", "Words", "AI score", "Verdict", "AI sentences"]]
style = [("BACKGROUND", (0, 0), (-1, 0), colors.HexColor("#263238")),
("TEXTCOLOR", (0, 0), (-1, 0), colors.white),
("FONTNAME", (0, 0), (-1, 0), "Helvetica-Bold"),
("GRID", (0, 0), (-1, -1), 0.4, colors.HexColor("#cfd8dc")),
("ROWBACKGROUNDS", (0, 1), (-1, -1), [colors.white, colors.HexColor("#f5f7f8")]),
("ALIGN", (1, 1), (-1, -1), "CENTER")]
ranked = sorted(subs, key=lambda s: -(s.get("result") or {}).get("ai_score", -1))
for i, s in enumerate(ranked, start=1):
r = s.get("result")
if not r:
rows.append([s["student"], s["words"], "n/a", s.get("error", "not scanned"), ""])
continue
items = r.get("items") or []
label, color = band(r["ai_score"])
ai = sum(1 for x in items if x["prediction"] == "ai-generated")
rows.append([s["student"], s["words"], f"{r['ai_score']:.0%}", label, f"{ai} of {len(items)}"])
style.append(("TEXTCOLOR", (3, i), (3, i), colors.HexColor(color)))
style.append(("FONTNAME", (3, i), (3, i), "Helvetica-Bold"))
if ai and r["ai_score"] < 0.5:
style.append(("TEXTCOLOR", (4, i), (4, i), colors.HexColor("#c62828")))
style.append(("FONTNAME", (4, i), (4, i), "Helvetica-Bold"))
table = Table(rows, colWidths=[55 * mm, 18 * mm, 24 * mm, 38 * mm, 28 * mm])
table.setStyle(TableStyle(style))
story += [table, Spacer(1, 4 * mm),
Paragraph("Bands: under 30% Human, 30 to 50% Likely human, 50 to 70% Likely AI, "
"70% and up AI-generated. A red sentence count on a human verdict means part "
"of the text was flagged, so check that student's page.", small)]
for s in ranked:
r = s.get("result")
if not r:
continue
label, color = band(r["ai_score"])
story += [PageBreak(),
Paragraph(escape(s["student"]), styles["Heading1"]),
Paragraph(f"File: {escape(s['file'])} | {s['words']} words | "
f"AI score: <b>{r['ai_score']:.0%}</b> | "
f"<font color='{color}'><b>{label}</b></font>", body),
Spacer(1, 3 * mm),
Paragraph("Highlighted sentences: red = flagged as AI-generated, green = original.", small),
Spacer(1, 3 * mm)]
paragraphs, spans, pos = [], [], 0
for item in r.get("items") or []:
start = s["text"].find(item["text"], pos)
if start > pos and "\n" in s["text"][pos:start] and spans:
paragraphs.append(spans)
spans = []
pos = max(pos, start + len(item["text"]))
spans.append(f"<span backColor='{sentence_color(item)}'>{escape(item['text'])}</span>")
paragraphs.append(spans)
for spans in paragraphs:
story += [Paragraph(" ".join(spans), body), Spacer(1, 3 * mm)]
SimpleDocTemplate(str(path), pagesize=A4, title=title,
leftMargin=18 * mm, rightMargin=18 * mm,
topMargin=16 * mm, bottomMargin=16 * mm).build(story)
def main() -> None:
ap = argparse.ArgumentParser(description="Batch-scan a class for AI writing with Proofademic.")
ap.add_argument("folder", type=Path, help="folder of .txt, .md or .docx submissions")
ap.add_argument("--out", type=Path, default=Path("reports"))
ap.add_argument("--title", default="Class AI Writing Report")
ap.add_argument("--max-pending", type=int, default=2, help="async job limit for your plan (Trial = 2)")
ap.add_argument("--dry-run", action="store_true", help="count words and credits without calling the API")
args = ap.parse_args()
subs = load_class(args.folder)
total = sum(s["words"] for s in subs)
print(f"{len(subs)} submissions, {total} words, about {total} credits")
if args.dry_run:
return
key = os.environ.get("PROOFADEMIC_API_KEY")
if not key:
sys.exit("Set PROOFADEMIC_API_KEY first.")
args.out.mkdir(parents=True, exist_ok=True)
scan_all(Proofademic(key), subs, args.out / "raw", args.max_pending)
write_csv(subs, args.out / "results.csv")
write_pdf(subs, args.out / "class_report.pdf", args.title)
print(f"Saved {args.out / 'class_report.pdf'} and {args.out / 'results.csv'}")
if __name__ == "__main__":
main()
How much does it cost to scan a class?
The AI Detector costs 1 credit per word. A class of 30 essays at 500 words each is about 15,000 credits. The free trial includes 2,000 credits, and the smallest paid plan includes 300,000 credits a month for $49.
Why use async mode instead of the regular endpoint?
The synchronous endpoint is limited to 10 requests per minute. Async requests aren't subject to that limit. They're only limited by your credits and by how many jobs can be pending at once (2 on the trial, 5 on the 300K plan, up to 50 on the largest plan).
Can it read Word documents and Google Docs?
It reads .txt, .md and .docx files. Download Google Docs as .docx or plain text first, or extend read_submission() to handle other formats.
What happens if the script crashes halfway through?
Every finished result is saved to reports/raw/. Run the script again and it skips anything already scanned, so you only pay for the submissions that didn't finish.
What's the shortest text it can scan?
The API requires at least 5 words. The script skips shorter submissions and marks them in the report instead of sending them.
Should I use the AI score to grade students?
No. Use it to decide which submissions to look at more closely, then read the flagged sentences and talk to the student. The report is a starting point for that conversation.