# Show HN: Psychosis Guard – Safety for long LLM conversations

> Source: <https://github.com/nwjang/psychosis-guard>
> Published: 2026-09-23 16:59:56+00:00

**Trajectory-aware safety middleware for long-running LLM conversations.**

English | [한국어](https://github.com/nwjang/psychosis-guard/blob/main/README.ko.md)

A chatbot can validate a user's delusional belief a little more on every turn
while no single message ever trips a content filter. psychosis-guard tracks the
**whole conversation**, scores where it is heading, and steps in with a graduated
intervention before that drift compounds. Open source, model-agnostic, runs as an
HTTP proxy or a Python library, and works with no API key on deterministic mocks.

| psychosis-bench metric | unguarded chatbot | safety prompt + psychosis-guard | 
|---|---|---|
| Delusion not confirmed | 42 % | **71 %** | 
| Harm not enabled | 60 % | **91 %** | 
| Safety intervention offered | 15 % | **87 %** | 

<sub>psychosis-bench · n=16 · one repetition · preliminary evaluation. Share of the
ideal score; full table, ablation and limitations in [Evaluation](#evaluation).</sub>

Conventional guardrails inspect one message at a time. psychosis-guard adds a
**Trajectory Rail**: a pipeline stage that tracks cumulative risk over the whole
conversation and escalates a graduated, clinically-informed intervention when the
conversation is heading the wrong way, even when every individual turn looks
harmless.

It runs as an HTTP middleware in front of any chatbot (OpenAI, Anthropic, any OpenAI-compatible server such as Ollama or vLLM, or a bot you already operate), or as a Python library.

**Disclaimer.** Research/educational tool. **NOT a medical device, NOT diagnosis
or crisis support.** In an emergency contact local emergency services or a crisis
line. Intervention copy and prompts are marked `ADVISER-REVIEW` in the source and
must be reviewed by a mental-health professional before use with real users.

- Python 3.10+
- Optional: an OpenAI or Anthropic API key, or any OpenAI-compatible endpoint. Without one, the whole pipeline runs on deterministic mocks.

```
pip install psychosis-guard                            # core library (mocks only)
pip install "psychosis-guard[server,openai,anthropic]" # HTTP server + real LLM adapters
```

Extras: `server` (FastAPI/uvicorn), `openai`, `anthropic`, `all`.

From source, for development (tests, lint):

```
git clone https://github.com/nwjang/psychosis-guard.git
cd psychosis-guard
pip install -e ".[dev]"
```

Every user turn passes through five rail stages. Stages 1–3 and 5 mirror the input / dialog / output / action rails of NVIDIA NeMo Guardrails; Stage 4 is the new, cumulative-state stage.

| Stage | Rail | What it does | 
|---|---|---|
| 1 | **Input Rail** | Pre-response risk estimate from the user turn plus the prior trajectory slope. HIGH short-circuits the chatbot and returns a safe response. | 
| 2 | **Dialog Rail** | Mode A: injects a graduated system prompt into the chatbot call. | 
| 3 | **Output Rail** | A judge scores the reply: reinforcement, sycophancy, pushback, escalation, help-referral. | 
| 4 | **Trajectory Rail** ★ | Folds the turn into cumulative state and computes the least-squares slope of delusion density over the conversation. | 
| 5 | **Action Rail** | Composite risk → `NONE / LOW / MEDIUM / HIGH` ; Mode B rewrites the reply. | 

`composite_risk = Σ wᵢ · signalᵢ + slope_boost · max(slope, 0)`

Only an *escalating* trajectory raises risk. A falling slope is not rewarded, so
interventions do not switch off while density is still high.

Key benefits:

- **Trajectory awareness.** Catches slow drift that turn-local filters cannot see.
- **Model-agnostic.** Works with any chatbot behind a single`respond()` interface,
or with replies your application already has (check-only mode).
- **Graduated, not binary.** A grounding question at LOW, an honest alternative
explanation at MEDIUM, de-escalation and referral at HIGH.
- **Fail-safe by construction.** Judge and rewriter failures never fail a turn;
they degrade to lexical signals and a deterministic intervention.
- **Config-driven.** Modes, thresholds and weights live in YAML, NeMo-style.

``` python
from psychosis_guard import PsychosisGuard

guard = PsychosisGuard.from_config("config.yml")     # deterministic mocks, no API key
reply = guard.send("Lately I keep noticing patterns that feel like signals meant for me.")
print(reply)
print(guard.log[-1].level.name, guard.summary()["delusion_slope"])
```

With a real model:

``` python
from psychosis_guard import PsychosisGuard
from psychosis_guard.adapters.llm import LLMChatbot, LLMJudge, LLMRewriter
from psychosis_guard.adapters.openai_adapter import OpenAICompleter
# from psychosis_guard.adapters.anthropic_adapter import AnthropicCompleter

llm = OpenAICompleter("gpt-4o-mini")                 # or base_url="http://localhost:11434/v1"
guard = PsychosisGuard(
    chatbot=LLMChatbot(llm, base_system_prompt="You are a friendly assistant."),
    judge=LLMJudge(llm),
    rewriter=LLMRewriter(llm),
    mode="combined",
)
reply = guard.send("user message")
```

Check-only, when your application already has a reply from any chatbot:

```
final = guard.send(user_message, bot_reply=draft_reply)   # always show `final`, not the draft
```

Any object with `respond(history, system_prompt)` is a chatbot, any object with
`score(history, reply)` is a judge, any object with `rewrite(...)` is a rewriter.
See `src/psychosis_guard/interfaces.py`.

```
export PG_CHAT_PROVIDER=openai PG_CHAT_MODEL=gpt-4o-mini OPENAI_API_KEY=sk-...
psychosis-guard check-config      # prints the resolved setup; fails loudly on mistakes
psychosis-guard serve             # http://0.0.0.0:8080, OpenAPI docs at /docs
```

Three integration shapes:

**1. Drop-in OpenAI-compatible proxy.** Point any OpenAI SDK at the server; nothing
else changes.

``` python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="<PG_AUTH_TOKEN or anything>")

raw = client.chat.completions.with_raw_response.create(
    model="guarded", messages=history, extra_headers={"X-Session-Id": session_id},
)
reply = raw.parse().choices[0].message.content
raw.headers["X-Guard-Level"]          # NONE | LOW | MEDIUM | HIGH
```

Without a session id the request is **stateless**: the trajectory is rebuilt from
the message history the client sent, so the proxy scales horizontally with no
shared state.

**2. Guarded turn.** The middleware calls the upstream chatbot for you.

```
curl -X POST localhost:8080/v1/guard/turn -H 'content-type: application/json' \
  -d '{"session_id": "abc", "message": "I keep seeing signs meant only for me"}'
```

**3. Check-only (bring your own reply).** Score and, if needed, rewrite a reply
produced by any chatbot.

```
curl -X POST localhost:8080/v1/guard/check -H 'content-type: application/json' \
  -d '{"session_id": "abc", "user_message": "...", "bot_reply": "...draft..."}'
```

Every guarded response carries the assessment:

```
{
  "reply": "...final reply the user should see...",
  "intervened": true,
  "level": "MEDIUM",
  "risk": 0.61,
  "signals": {"delusion_density": 0.5, "reinforcement": 0.0, "...": 0.0},
  "trajectory": {"turn_count": 4, "delusion_slope": 0.08, "bot_validation_count": 1},
  "trace": ["[stage1:input] ...", "[stage2:dialog] ...", "..."]
}
```

Full endpoint reference: [docs/http-api.md](https://github.com/nwjang/psychosis-guard/blob/main/docs/http-api.md).

```
cp .env.example .env              # provider, model, keys
docker compose up --build         # http://localhost:8080
curl localhost:8080/healthz
```

| Role | Providers | 
|---|---|
| Chatbot under guard | OpenAI, any OpenAI-compatible server (Ollama, vLLM, LM Studio, Groq, OpenRouter, …), Anthropic, or your own `Chatbot` implementation, or none (check-only) | 
| Judge (risk rubric) | Same set; can be a different, cheaper model than the chatbot | 
| Rewriter (Mode B) | Same set, or `none` for a deterministic appended intervention | 

Set them independently with `PG_CHAT_*`, `PG_JUDGE_*`, `PG_REWRITER_*`.

The `condition` key in `config.yml` (or `PG_CONDITION`) selects the mode. Presets
for each live in [`configs/`](https://github.com/nwjang/psychosis-guard/tree/main/configs/).

| Mode | Behaviour | 
|---|---|
| `combined` | Mode A + Mode B + Trajectory Rail. The default. | 
| `A` | System-prompt injection only (needs a steerable chatbot). | 
| `B` | Post-hoc rewrite only (fully model-agnostic). | 
| `single-turn-filter` | A + B with the Trajectory Rail off. A turn-local baseline. | 
| `detect-only` | Score and decide levels, never intervene. Shadow mode for rollout. | 
| `none` | Observe and log only. | 

Policy lives in YAML, outside code:

``` php
condition: combined
policy:
  low: 0.25          # composite-risk thresholds -> LOW / MEDIUM / HIGH
  medium: 0.50
  high: 0.75
  w_reinforcement: 0.35
  w_sycophancy: 0.20
  w_delusion: 0.20
  w_conviction: 0.15
  w_isolation: 0.10
  slope_boost: 3.0   # how strongly an escalating trajectory raises risk
  slope_window: 4    # turns the policy looks back over
```

Runtime settings are environment variables (`PG_*`, see [`.env.example`](https://github.com/nwjang/psychosis-guard/blob/main/.env.example)
and [docs/configuration.md](https://github.com/nwjang/psychosis-guard/blob/main/docs/configuration.md)).

- **Calls per guarded turn:** 1 chatbot + 1 judge, plus 1 rewriter only when the
level is above`NONE` . Use a cheap judge model and`PG_REWRITER_PROVIDER=none` for
the minimum.
- **Failure handling:** judge errors degrade to lexical signals; rewriter errors
degrade to a deterministic appended intervention. Upstream chatbot errors surface
as HTTP 502.
- **State:** sessions are process-local, TTL and LRU bounded. Run one replica, use
sticky sessions, or use the stateless proxy path.
- **Languages:** the judge rubric is language-independent and also estimates the
user-side signals, so session-keyed conversations in any language get a full
trajectory. The API-free lexicons used before the chatbot is called are English
only; prefer session ids for non-English deployments.

```
psychosis-guard serve [--host H] [--port P] [--workers N]   run the HTTP middleware
psychosis-guard check-config                                validate env + config
psychosis-guard demo [--turns N]                            offline unguarded-vs-guarded demo
```

|  | Turn-local guardrails (e.g. NeMo Guardrails) | psychosis-guard | 
|---|---|---|
| Unit of analysis | one message | the conversation trajectory | 
| Sees slow drift | no | yes, via the delusion-density slope | 
| Intervention | block / replace | graduated: ground → explain → de-escalate + refer | 
| Placement | before or after the model | before (Mode A), after (Mode B), or check-only | 

psychosis-guard is complementary: it slots behind or beside content rails rather than replacing them.

Measured on **psychosis-bench** (Dohnány et al. 2025, [arXiv:2509.10970](https://arxiv.org/abs/2509.10970)),
a public benchmark of 16 scripted 12-turn conversations that progress from
loneliness to a delusional belief to a harmful act. Every turn is scored with the
benchmark's own LLM-judge rubric: **DCS** delusion confirmation (0-2, lower is
better), **HES** harm enablement (0-2, lower is better), **SIS** whether a
safety intervention was offered (0-1, higher is better). All targets use the same
chatbot model (`gpt-4o-mini`, temperature 0.7), judge (` gpt-4o-mini`) and user
script, so the contrasts between rows are like-for-like. n = 16 cases, one
repetition, mean ± 95 % CI. Run 2026-09-07.

| target | DCS ↓ | HES ↓ | SIS ↑ | 
|---|---|---|---|
| unguarded chatbot | 1.17 ± 0.23 | 0.79 ± 0.18 | 0.15 ± 0.13 | 
| + one-paragraph safety system prompt | 0.74 ± 0.15 | 0.33 ± 0.18 | 0.76 ± 0.18 | 
| psychosis-guard, Trajectory Rail **off** (turn-local) | 0.98 ± 0.13 | 0.72 ± 0.17 | 0.20 ± 0.17 | 
| psychosis-guard, `B` (rail + rewrite) | 0.82 ± 0.10 | 0.39 ± 0.17 | **0.89 ± 0.14** | 
| psychosis-guard, `combined` | 0.85 ± 0.09 | 0.40 ± 0.16 | 0.74 ± 0.21 | 
| safety system prompt **+** psychosis-guard`combined` | **0.58 ± 0.18** | **0.19 ± 0.15** | 0.87 ± 0.11 | 

Paired Wilcoxon on the 16 matched cases, Holm-corrected:

- **vs the unguarded chatbot** ,`B` and`combined` improve all three metrics
(d = 0.8-2.1, all p < .02) and eliminate every full-validation (DCS = 2) and
full-compliance (HES = 2) turn.
- **Trajectory Rail ablation.**`B` vs the same pipeline with the rail off
differs only in the rail; the rail accounts for DCS −0.16, HES −0.33, SIS
+0.69 (all p < .05). Turn-local scoring under-calls risk on a slowly
escalating script, so the rewriter fires at the wrong level.
- **vs a safety system prompt alone** , the middleware is statistically
indistinguishable: parity, obtained without access to the chatbot's prompt.
Stacking the two is the best row on every metric (significant vs`combined` ; directionally better than the prompt alone, not significant at
n = 16).
- **Utility.** 0 interventions in 520 turns of benign control conversations.

**What this does not show.** These are scores on the chatbot's replies to a
fixed script. In a separate reactive simulation, where an LLM-played user
adjusts their next message to the reply, the middleware's referral and
pushback rates rise just as here, but the simulated user's delusion density
and conviction do not improve (`combined` ≈ unguarded); interventions that
insert the most safety language, the post-hoc rewriter alone and the safety
system prompt, make that simulated user *worse*. The intervention text is
real; the framing around it is what still needs work (the `ADVISER-REVIEW`
prompts). That simulator is unvalidated and the judge is an LLM checked only
against another LLM (κ ≈ 0.5), so treat the table above as a bot-side
benchmark, not evidence of user outcomes. Runner, scripts and per-turn
transcripts are in the research repository; three repetitions and human judge
labels are the planned next step.

- [Architecture](https://github.com/nwjang/psychosis-guard/blob/main/docs/architecture.md) (Korean:[docs/architecture.ko.md](https://github.com/nwjang/psychosis-guard/blob/main/docs/architecture.ko.md) )
- [HTTP API reference](https://github.com/nwjang/psychosis-guard/blob/main/docs/http-api.md)
- [Configuration reference](https://github.com/nwjang/psychosis-guard/blob/main/docs/configuration.md)
- [Examples](https://github.com/nwjang/psychosis-guard/tree/main/examples/) :`quickstart_mock.py` (no key),`quickstart_openai.py` ,`client_openai_sdk.py`
- [Changelog](https://github.com/nwjang/psychosis-guard/blob/main/CHANGELOG.md)

Contributions are welcome. Please read [CONTRIBUTING.md](https://github.com/nwjang/psychosis-guard/blob/main/CONTRIBUTING.md) and the
[Code of Conduct](https://github.com/nwjang/psychosis-guard/blob/main/CODE_OF_CONDUCT.md). Changes to any text marked `ADVISER-REVIEW`
(intervention copy, judge and rewriter prompts) need sign-off from a mental-health
professional before they are merged.

Apache License 2.0. See [LICENSE](https://github.com/nwjang/psychosis-guard/blob/main/LICENSE) and [NOTICE](https://github.com/nwjang/psychosis-guard/blob/main/NOTICE). This is an
independent clean-room implementation; it is architecturally inspired by NVIDIA
NeMo Guardrails but contains no NeMo source code.