{"slug": "quantdinger-s-jev-trade-filter-fail-open-never-backtested", "title": "QuantDinger's Jev trade filter: fail-open, never backtested", "summary": "A developer's source-code review of QuantDinger's optional Jev AI trade filter (commit 96cec2d) found the gate fails open: when the LLM's confidence on key questions falls below 0.55 it defers to an uncalibrated fallback model, and if that is unconfigured or credits run out, orders pass unchecked. The filter runs only on live strategies with no backtesting support, and its prompt is biased toward 'pass', so it blocks only when the model is confident something is wrong.", "body_md": "**How does QuantDinger use Jev?** As a gate in front of entry orders. A strategy decides to buy or sell, then Jev answers six questions and the code decides whether the order goes out. It's a safer design than the bots in [part 1](https://moonthetrain.com/lab/jev-trading-projects) that let Jev decide everything: exits never go through the AI, and the decision rules live in code, not in a prompt. But the gate lets orders through whenever it can't decide, and there's no way to backtest it. That last point is what made me run my own test.\n\n[Part 2](https://moonthetrain.com/lab/quantdinger-review) covered QuantDinger as a platform. Jev is an optional feature on top of it, switched on per strategy or for manual \"Quick Trade\" orders.\n\nEverything below comes from reading the source at commit `96cec2d` (21 Sep 2026), mainly `ai_decision_filter.py` and `ai_decision_context.py`. I didn't run the platform; it needs Docker, Postgres and Redis, and I only wanted to know how the gate decides.\n\nIn this order:\n\nIt only runs on live strategies, not in backtests or paper trading.\n\nSix multiple-choice questions, sent together:\n\n| Question | Choices | \n|---|---|\n| Is the data good enough to decide? | sufficient / partial / insufficient | \n| Does the signal agree across timeframes? | aligned / mixed / conflict / insufficient | \n| Does the market regime suit this entry? | favorable / neutral / adverse / insufficient | \n| Size, leverage, drawdown, losing streak | clear / caution / block / insufficient | \n| Fresh price, valid SL/TP? | clear / caution / block / insufficient | \n| Final decision | pass / reject | \n\nThe order goes out only when the final answer is `pass`, neither the risk nor the execution check says `block`, and the timeframes don't show `conflict` in an `adverse` regime at the same time. The state it sends is compact: a summary of up to 120 bars on two or three timeframes (returns, MAs, RSI, MACD, ATR, support and resistance, data age), open positions, and the strategy's recent results.\n\nFour places, in the order I'd fix them.\n\n**It lets trades through when unsure.** If Jev's confidence on the key questions is below 0.55 (the default), the platform treats that as an error and asks a different, uncalibrated LLM. If that isn't configured, the trade goes through. So the moments when the AI is least sure are exactly the moments the filter stops filtering.\n\n**Running out of credits switches it off.** With no credits left, orders pass without any check. A user can believe the filter is on when it isn't doing anything.\n\n**The prompt leans towards \"pass\".** It tells the model that missing evidence alone must not reject and to reject only on concrete evidence. Together with the fail-open paths, the gate only blocks when the model is confident something is wrong.\n\n**You can't backtest it.** The filter runs only live. There's no way to replay history and see whether it would have helped or hurt, and the README gives no numbers either.\n\nTwo smaller ones. The data-quality answer is collected but never used in the decision. And `caution` on risk or execution doesn't reduce position size; it's treated the same as `clear`.\n\nEverything above assumes Jev reads the market the right way round. The gate can only help if, when Jev says `adverse` or `conflict`, the trade really is worse than average. If Jev reads the evidence backwards, the gate blocks good entries and passes bad ones, and because it only runs live, nobody would see that until the P&L did.\n\nHow much damage that would do depends on how often the gate says no, and the prompt leans towards `pass`. That's also something you can't measure without a backtest.\n\nSo I stopped reading code and asked the simpler question directly: given clean indicators on data it has never seen, can Jev tell which way a market is about to move? That's the experiment in [part 5](https://moonthetrain.com/lab/can-jev-predict-bitcoin).\n\nIf I put an AI gate in front of an MT5 EA, I'd keep QuantDinger's two best ideas (exits never go through the model, and the rule lives in code) and change three things:\n\n`caution` cuts the lot size.\nNext in this series: the experiment. I gave Jev 12 indicators on 1,500 Bitcoin candles it had never seen and compared it with a plain model.\n\nOriginally published at [moonthetrain.com](https://moonthetrain.com/lab/quantdinger-jev-trade-filter).", "url": "https://wpnews.pro/news/quantdinger-s-jev-trade-filter-fail-open-never-backtested", "canonical_source": "https://dev.to/moonthetrain/quantdingers-jev-trade-filter-fail-open-never-backtested-5fji", "published_at": "2026-10-02 01:39:10+00:00", "updated_at": "2026-10-02 01:44:25.765844+00:00", "lang": "en", "topics": ["ai-agents", "ai-products", "ai-tools"], "entities": ["QuantDinger", "Jev", "MT5"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/quantdinger-s-jev-trade-filter-fail-open-never-backtested", "markdown": "https://wpnews.pro/news/quantdinger-s-jev-trade-filter-fail-open-never-backtested.md", "text": "https://wpnews.pro/news/quantdinger-s-jev-trade-filter-fail-open-never-backtested.txt", "jsonld": "https://wpnews.pro/news/quantdinger-s-jev-trade-filter-fail-open-never-backtested.jsonld"}}