{"slug": "an-open-source-candle-only-crypto-pump-detector-looking-for-ml-contributors", "title": "An open-source, candle-only crypto pump detector–looking for ML contributors", "summary": "An open-source, candle-only crypto pump detector, the pump-dump-crypto-screener, is seeking machine learning contributors to improve its detection algorithms. The TypeScript monorepo, available on GitHub, pulls 1-minute and 5-minute market statistics and bulk kline archives from Binance and Bybit, detects pump/dump regimes, and alerts via Telegram, with a live bot at @pumpdumpscreenerautobot. The project requires Node.js 20+, pnpm 9+, and pm2, and includes a manual review workflow to build a human-labeled dataset for evaluating and improving the screener.", "body_md": "TypeScript pnpm monorepo for pulling 1m/5m market statistics and bulk kline archives from Binance and Bybit, detecting pump/dump regimes, and alerting via Telegram.\n\n**Try the live bot:** [Open @pumpdumpscreenerautobot on Telegram](https://t.me/pumpdumpscreenerautobot)\n\n**Requirements:** Node.js 20+, pnpm 9+, [pm2](https://pm2.keymetrics.io/) (process manager for production).\n\n**No exchange API keys.** Binance and Bybit data comes from public bulk archives and public REST endpoints.\n\nYou need two free accounts (~2 minutes each):\n\n| Service | Why | Setup |\n|---|---|---|\n|\nStores pumps, monitor runs, and Telegram subscribers |\n`turso db create screener` → copy URL + auth token into `config.js` |\n\n**Telegram**`/stats`\n\n/ `/runs`\n\n/ `/about`\n\nbot[@BotFather](https://t.me/BotFather)→`/newbot`\n\n→ see [docs/telegram_setup.md](/aleks-ent/pump-dump-crypto-screener/blob/main/docs/telegram_setup.md)\n\n```\npnpm install\ncp config.example.js config.js   # fill in Turso + Telegram (see table above)\n```\n\nEdit `config.js`\n\n— at minimum, configure `database`\n\n, `telegramBotToken`\n\n, and your private `classifierTelegramChatId`\n\n. Anyone who sends `/start`\n\nis automatically subscribed to alerts and can use `/stats`\n\n, `/runs`\n\n, and `/about`\n\n; only the configured chat can classify alerts. Pump lookback and scan settings live under `pump`\n\n(defaults in [ config.example.js](/aleks-ent/pump-dump-crypto-screener/blob/main/config.example.js)):\n\n```\ndatabase: {\n  url: \"libsql://screener-....turso.io\",\n  authToken: \"...\",\n},\ntelegramBotToken: \"123456789:ABC...\",\nclassifierTelegramChatId: \"36772199\",\nweb: {\n  port: 3000,    // local app server; nginx terminates public HTTP/HTTPS\n  host: \"127.0.0.1\",\n},\npump: {\n  days: 5,        // lookback calendar days for download + scan\n  minScore: 80,\n  scanCache: true,\n  requireCalmPrePump: false, // feature flag: require a calm 2h period before pumps\n},\npnpm build\npnpm db:bootstrap\n```\n\nRun `pnpm build`\n\nagain after every code change — PM2 does not rebuild for you.\n\nProduction runs are defined in [ ecosystem.config.cjs](/aleks-ent/pump-dump-crypto-screener/blob/main/ecosystem.config.cjs) at the repo root. It starts three processes:\n\n| PM2 name | What it runs | Role |\n|---|---|---|\n`pump-monitor` |\n`pnpm pump:monitor` |\nDownload → scan → persist pumps → Telegram alerts |\n`pump-bot` |\n`pnpm pump:bot` |\n`/stats` , `/runs` , `/about` , and Pump | Dump | None button clicks |\n`pump-web` |\n`pnpm pump:web` |\nPlain HTTP page showing the last 10 stored pumps |\n\n```\npm2 start ecosystem.config.cjs\n```\n\nPM2 keeps all processes alive. When `pump-monitor`\n\nfinishes a pipeline run it exits; PM2 immediately starts the next run (`autorestart: true`\n\n). That replaces a manual cron loop.\n\n**First run** downloads `pump.days`\n\nof candles into `data/market_stats/`\n\n— network + disk required; can take hours. Market data is not in the repository.\n\nThe web page is served by `pump-web`\n\non `127.0.0.1:3000`\n\nby default. Put nginx in front of it for public HTTP/HTTPS; see [docs/nginx_letsencrypt.md](/aleks-ent/pump-dump-crypto-screener/blob/main/docs/nginx_letsencrypt.md).\n\nPersist PM2 across reboots:\n\n```\npm2 save\npm2 startup   # follow the printed command (once per server)\n```\n\nThe manual reviewer at `/review`\n\nturns detected pumps into a human-labeled dataset for\nevaluating and improving the screener. It is designed for a fast, keyboard-friendly\nworkflow:\n\n- Browse and filter stored pump events by status, category, exchange, symbol, and date.\n- Inspect Telegram subscriber votes alongside a 1m or 5m OHLCV chart centered on the screener's detection time. Charts use local history when available and otherwise load the four-hour window from the event's public Binance or Bybit API.\n- Classify each event as a wick spike, weak pump, sustained move, volume only, illiquid noise, or unclear; optionally record confidence and a comment.\n- Save and advance with keyboard shortcuts, revisit existing labels, track review progress, and export labeled datasets as JSON or CSV.\n\nHuman annotations are stored separately from the original pump records, so review does\nnot modify detector output. The workspace uses the existing Turso database, makes no\nbrowser-to-exchange requests, and can be protected with simple HTTP Basic authentication.\nSee the [pump review operator guide](/aleks-ent/pump-dump-crypto-screener/blob/main/docs/pump-event-review/implementation.md) for setup,\naccess control, deployment, and release checks.\n\n```\npm2 status\npm2 logs                          # all apps\npm2 logs pump-monitor             # download + scan pipeline\npm2 logs pump-bot                 # Telegram bot\npm2 logs pump-web                 # HTTP page\n\npm2 restart ecosystem.config.cjs  # after config.js or code changes (rebuild first)\npm2 restart pump-monitor          # restart pipeline only\npm2 restart pump-bot              # restart bot only\npm2 restart pump-web              # restart HTTP page only\n\npm2 stop ecosystem.config.cjs\npm2 delete ecosystem.config.cjs\n```\n\nAfter editing `config.js`\n\n, restart the affected process (`pm2 restart pump-monitor`\n\n, `pump-bot`\n\n, or `pump-web`\n\n). No PM2 reload is needed for config-only changes if you restart.\n\n**Classification buttons** need `pump-bot`\n\nrunning — it is included in `ecosystem.config.cjs`\n\n, not optional in production.\n\nEach PM2-driven run is an end-to-end pipeline:\n\n**Download**— last`pump.days`\n\nof 1m/5m candles from**Binance and Bybit**(archives + REST fallback). See[docs/fetch_all.md](/aleks-ent/pump-dump-crypto-screener/blob/main/docs/fetch_all.md).** Scan**— pump/dump detection; output`data/market_stats/reports/pump_events.ndjson`\n\n.**Persist + alert**— upsert pump episodes to Turso; Telegram message per** new, current**pump (`coin|pump_start_utc`\n\n). Episodes ending before the previous successful monitor cycle began are historical backfill: they are stored for`/review`\n\nbut are not broadcast as fresh alerts.\n\nThe exchange symbol universe is re-discovered when\n`data/market_stats/reports/symbol_universe.json`\n\nreaches\n`pump.universeRefreshDays`\n\nold (default: 4 days). This adds new listings and removes\ndelisted instruments from subsequent fetches and scans.\n\nOn disk: `data/market_stats/`\n\n(`archives/`\n\n, `api_fallback/raw/`\n\n, `reports/`\n\n). Cached series are skipped on repeat runs. Nothing is ever pruned automatically — see [Disk usage and retention](#disk-usage-and-retention).\n\nAlerts sent to `classifierTelegramChatId`\n\nhave **Pump | Dump | None** buttons. `pump-bot`\n\nverifies the callback came from that chat before writing `pumps.classification`\n\n; other subscribers receive the same alerts without classification buttons.\n\nTurso bootstrap (one-time):\n\n```\nturso db create screener\nturso db show screener --url\nturso db tokens create screener\npnpm db:bootstrap\n```\n\nTelegram details: [docs/telegram_setup.md](/aleks-ent/pump-dump-crypto-screener/blob/main/docs/telegram_setup.md).\n\nPer-coin results under `data/market_stats/reports/scan_cache/`\n\n(override with `--cache-dir`\n\non the underlying CLI). Entries are reused when detector version, scan params, window start, and loaded candle identity per exchange are unchanged — not file mtimes. New 5m bars trigger an **incremental tail rescan** (~400 bars); UTC midnight window rolls trigger a full rescan.\n\nDisable cache: set `pump.scanCache: false`\n\nin `config.js`\n\n, or delete `scan_cache/`\n\n, then `pm2 restart pump-monitor`\n\n.\n\nScanning uses a **worker thread pool** (auto-detected CPU cores). Production scan step uses compiled `dist/cli.js`\n\nwith native worker threads (required on Linux/VDS).\n\nNothing in the pipeline deletes anything — every run only appends. On a long-lived VDS `data/market_stats/`\n\ngrows without bound (100 GB+ over a couple of months is normal).\n\n| Path | What it holds | Grows with | Read back? |\n|---|---|---|---|\n`archives/` |\nBinance bulk archive downloads, `symbol=X/date=YYYY-MM-DD` |\ndays × symbols | yes |\n`api_fallback/raw/` |\nREST fallback candles, `date=YYYY-MM-DD/symbol=X` |\ndays × symbols | yes |\n`extracted/` |\narchives unpacked to NDJSON, `date=YYYY-MM-DD/symbol=X` |\ndays × symbols | yes — derived, regenerable from `archives/` |\n`reports/pump_detector/<runId>/` |\norchestrator log + two files per coin | every run (~50 MB each) |\nno |\n`reports/scan_cache/` |\nper-coin scan results | symbol count only (~6 MB) | yes |\n\n`reports/pump_detector/`\n\nis usually the bulk of it. A fresh run directory is created on every scan, and PM2 restarts `pump-monitor`\n\nthe moment it exits, so this is several GB/day of debug output that no code path reads.\n\nScanning itself only needs `pump.days`\n\nof candles. Longer retention exists for the `/review`\n\nUI, which reads `archives/`\n\n, `api_fallback/`\n\n, and `extracted/`\n\nto draw charts for past episodes — below the retention horizon those charts fall back to TradingView, while the stored pump rows in Turso are unaffected.\n\n[ scripts/prune-market-data.sh](/aleks-ent/pump-dump-crypto-screener/blob/main/scripts/prune-market-data.sh) prunes both by age. It is dry-run by default:\n\n```\n./scripts/prune-market-data.sh            # preview: matched directories and size\n./scripts/prune-market-data.sh --apply    # delete\n```\n\nDefaults keep 60 days of candles and 2 days of run directories; override with `RETAIN_DAYS`\n\n, `RUN_RETAIN_DAYS`\n\n, `DATA_DIR`\n\n. Safe to run while `pump-monitor`\n\nis live — the run-directory floor keeps the in-flight run, and the pipeline only writes today's date partitions.\n\nKeep it from coming back with a daily cron:\n\n```\n(crontab -l 2>/dev/null; echo \"17 4 * * * cd /path/to/repo && ./scripts/prune-market-data.sh --apply >> /tmp/prune-market-data.log 2>&1\") | crontab -\n```\n\nPM2's own logs accumulate separately from `data/`\n\n. Check with `du -sh ~/.pm2/logs`\n\n; `pm2 flush`\n\nclears them and `pm2 install pm2-logrotate`\n\nprevents recurrence.\n\nThe pipeline CLIs accept flags if you need a single manual run (debugging only):\n\n```\npnpm pump:monitor -- --no-telegram --cache-dir /path/to/cache\n```\n\nNormal operation should stay on PM2.\n\n**Local cache coverage** — how complete on-disk data is for the lookback window:\n\n```\npnpm report:coverage\npnpm report:coverage -- --exchanges binance --days 7\npnpm report:coverage -- --start 2026-06-03T00:00:00Z --end 2026-06-04T00:00:00Z --json\n```\n\nExample output:\n\n```\nWindow: 2026-06-03 → 2026-06-04 (1 days), intervals 5m\nFor all coins present on exchange: Binance 82.9% cached (1133/1367), Bybit 80.0% cached (...)\n```\n\nUseful flags: `--days`\n\n, `--exchanges`\n\n, `--quote-currencies`\n\n, `--discover`\n\n, `--json`\n\n.\n\n`ecosystem.config.cjs`\n\n— PM2 process definitions (`pump-monitor`\n\n,`pump-bot`\n\n,`pump-web`\n\n)`config.js`\n\n— Turso, Telegram bot token,`pump.*`\n\n,`fetch.intervals`\n\n(copy from`config.example.js`\n\n)`scripts/prune-market-data.sh`\n\n— age-based cleanup for`data/market_stats/`\n\n`packages/core`\n\n— HTTP client, config, pull window`packages/exchanges`\n\n— exchange adapters`packages/storage`\n\n— NDJSON, gaps, manifest I/O`packages/universe`\n\n— symbol universe and task resolution`packages/archive`\n\n— archive planners and download runner`apps/fetch-market-archives`\n\n— universe archive pull CLI (+ coverage reports)`packages/db`\n\n— screener database (Turso/libSQL)`packages/pump-detector`\n\n— pump regime detection ([PUMP_DETECTION_RULES.md](/aleks-ent/pump-dump-crypto-screener/blob/main/PUMP_DETECTION_RULES.md))`apps/run-pump-detector`\n\n— scan worker pool and`pump_events.ndjson`\n\n`apps/pump-monitor`\n\n— fetch + scan + Turso + Telegram alerts + HTTP pump page`apps/fetch-market-stats`\n\n— REST candle pull CLI (standalone; not used by`pump-monitor`\n\n)\n\nThis project is open source under the [MIT License](/aleks-ent/pump-dump-crypto-screener/blob/main/LICENSE).", "url": "https://wpnews.pro/news/an-open-source-candle-only-crypto-pump-detector-looking-for-ml-contributors", "canonical_source": "https://github.com/aleks-ent/pump-dump-crypto-screener", "published_at": "2026-08-16 20:16:30+00:00", "updated_at": "2026-08-16 20:40:38.518523+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools"], "entities": ["Binance", "Bybit", "Telegram", "pump-dump-crypto-screener", "GitHub", "Node.js", "pnpm", "pm2"], "alternates": {"html": "https://wpnews.pro/news/an-open-source-candle-only-crypto-pump-detector-looking-for-ml-contributors", "markdown": "https://wpnews.pro/news/an-open-source-candle-only-crypto-pump-detector-looking-for-ml-contributors.md", "text": "https://wpnews.pro/news/an-open-source-candle-only-crypto-pump-detector-looking-for-ml-contributors.txt", "jsonld": "https://wpnews.pro/news/an-open-source-candle-only-crypto-pump-detector-looking-for-ml-contributors.jsonld"}}