If you follow AI safety, you know the uncomfortable truth: modern LLM guardrails are failing not because of complex zero-day exploits, but because of sloppy heuristics, brittle filters, and naive prompt-matching.
We built AURA to be pragmatic: deterministic, testable, fully auditable, and completely immune to the hallucination of its own telemetry.
Today, we’re unpacking AURA v0.1.0 — explaining our architectural decisions, showing real code excerpts, diving into non-linear risk scoring math, and solving the silent problem of lost repository analytics on GitHub.
Data flow in AURA is intentionally simple, pipeline-driven, and fully reproducible:
public_cases/*.json) → Cleaned, validated, and normalized via scripts/normalize-percases.ts scripts/extract-triggers.ts) → Contextual regex and sliding-window token analysis. config/signal-mapping.json and saved to config/trigger-weights.json. scripts/recalc_confidence.ts) → Non-linear math transform + cross-check audit logic via scripts/policy/crossCheckAdapter.ts. analytics/traffic-history.json via GitHub Actions.
We compute a transparent raw evidence sum (confidence_raw) and convert it into a bounded score via a diminishing‑returns exponential:
confidence = 1 − e^(−α × confidence_raw)
This keeps scores in [0...1] and avoids noisy amplification from many weak cues. Implementation (excerpt from scripts/recalc_confidence.ts):
// computedRaw is the honest sum of trigger weights + cross-check contributions
const alpha = (typeof cfg.normAlpha === 'number') ? cfg.normAlpha : 1.0;
const normalized = 1 - Math.exp(-alpha * computedRaw);
let newVal = Math.round(normalized * 100) / 100;
if (newVal < minFloor) newVal = minFloor;
e.confidence_raw = Math.round(computedRaw * 100) / 100;
e.confidence = newVal;
confidence_raw = 3.0 yields ≈ 0.59, requiring compounding signals for a hard policy block.
Keyword lists are a recipe for false positives. Matching a naive string like "generate 500" catches harmless test cases alongside malicious payloads.
To balance precision and recall, AURA v0.1.0 introduces ordered-within-window token matching.
Excerpt from scripts/extract-triggers.ts showing how AURA tolerates small syntactic noise without matching scattered words across an entire prompt:
function containsOrderedWithinWindow(haystack: string[], needle: string[], window = 5): boolean {
if (needle.length === 0) return false;
if (needle.length === 1) return haystack.indexOf(needle[0]) !== -1;
for (let i = 0; i < haystack.length; i++) {
if (haystack[i] !== needle[0]) continue;
let idx = i + 1;
let matched = 1;
for (let k = 1; k < needle.length && idx < Math.min(haystack.length, i + window + 1); idx++) {
if (haystack[idx] === needle[k]) { matched++; k++; }
}
if (matched === needle.length) return true;
}
return false;
}
In config/trigger-extraction.json, bulk asset creation explicitly requires a deception target:
{
"trigger": "non-consensual pattern generation",
"pattern": "\\bgenerate\\s+\\d+\\s+(?:deceptive\\s+assets|phishing\\s+emails|fake\\s+documents|fake\\s+profiles|fake\\s+accounts|malicious\\s+payloads|spam\\s+emails|synthetic\\s+attacks)\\b",
"description": "Bulk-generation demand with deception-specific targets."
}
For generic terms like "audit", we mandate multi-token co-occurrence:
{
"trigger": "unauthorized audit camouflage",
"cues": ["audit", "unauthorized", "independent", "bypass", "without permission"],
"description": "Require co-occurrence of 'audit' with authorization‑bypass phrasing."
}
I don't know about you, but I got tired of seeing my project's growth through the lens of GitHub’s default 14-day window. GitHub silently wipes daily clone and view metrics after two weeks, leaving open-source maintainers completely blind to long-term adoption trends unless they buy third-party analytics dashboards.
To solve this, AURA v0.1.0 includes an automated, self-healing snapshot pipeline in .github/workflows/traffic-history.yml.
- name: Auto-merge PR and delete branch
uses: actions/github-script@v6
with:
github-token: ${{ secrets.TRAFFIC_TOKEN }}
script: |
const head = `auto/traffic-report-${process.env.GITHUB_RUN_ID}`;
const { data: prs } = await github.rest.pulls.list({ owner: context.repo.owner, repo: context.repo.repo, head: `${context.repo.owner}:${head}`, state: 'open' });
if (prs && prs.length > 0) {
await github.rest.pulls.merge({ owner: context.repo.owner, repo: context.repo.repo, pull_number: prs[0].number, merge_method: 'squash' });
await github.rest.git.deleteRef({ owner: context.repo.owner, repo: context.repo.repo, ref: `heads/${head}` });
}
One of the best parts of open-sourcing AURA is having sharp contributors look at the edge cases. Our contributor Amirhossein Agrest spotted a subtle async race condition in how case state is updated before disk serialization.
updateCase() awaits cross-check evaluation internally. However, in certain batch execution flows, calling scripts invoked it across collections without awaiting the return promise before writing JSON files to disk.
The result? CLI logs proudly claimed success, while disk artifacts still contained pre-audit state.
// ❌ Potential Race: Fire-and-forget async invocation
for (const entry of entries) updateCase(entry);
// ✅ Fix Pattern: Await all async mutations before serializing to disk
const promises = entries.map(entry => updateCase(entry));
await Promise.all(promises);
We’ve logged this issue (shoutout to Amirhossein!) and are pairing it with artificial network-delay adapters in our test suite to guarantee filesystem persistence never outruns in-memory state in the upcoming patch.
AURA is not magic, and it doesn't pretend to be. It's a pragmatic stack: deterministic rules, auditable math, and CI that refuses to forget its history.
If you're looking for a silver bullet, good luck! But if you want a system that is testable, inspectable, and built on sound software engineering principles, welcome aboard.
⭐ Check out the repository, inspect the code, and give us a star:
👉 GitHub: kate8382/AURA
Open an issue, or even better — submit a PR with a failing unit test to help us catch edge cases faster.