{"slug": "show-hn-classify-mechanical-faults-using-contrastive-language-audio-pretraining", "title": "Show HN: Classify mechanical faults using Contrastive Language-Audio Pretraining", "summary": "A developer released cardiag, an open-source audio-ML pipeline that uses Contrastive Language-Audio Pretraining (CLAP) to classify mechanical faults from phone recordings. The tool achieves 0.79 AUROC for fault detection and provides calibrated triage, returning 'uncertain' when confidence is low. The project is available as a CLI and web app.", "body_md": "`cardiag`\n\nis an end-to-end audio-ML pipeline. It scrapes fault-sound clips from\nYouTube/TikTok, cleans the audio (isolating the mechanical sound from speech, music,\nand noise), embeds it with a frozen CLAP model, and trains small linear heads to\ntriage the fault. It is exposed as a CLI and a live web app.\n\n## cardiag-demo.mp4\n\nThis is a proof of concept, and honest about what that means. Diagnosing a car fault\nfrom a phone recording is genuinely hard, so `cardiag`\n\nis built as a calibrated\ntriage aid rather than a diagnoser: it tells you whether something sounds wrong,\nroughly *where in the car* it is, and a ranked shortlist of likely parts. When the\naudio won't support a call, it says \"uncertain\" instead of bluffing.\n\nThe real contribution is the cleaning + honest-training recipe, which is reusable on other audio datasets. The modest accuracy here reflects how hard the problem is from crude phone audio (we hit the literature ceiling); the\n\nsamemethod reaches 0.93 AUROC on clean engine audio. See[docs/DEFENSE.md].\n\nTwo pages visualize the first two stages of the pipeline:\n\n[Isolating the engine audio](https://adamsohn.com/separate/)— an interactive look at the`clean()`\n\ncascade pulling a short mechanical span out of noisy YouTube audio (speech, music, road noise).[CLAP, visualized](https://adamsohn.com/clap/)— how the frozen CLAP model turns those spans into the 512-d embedding the linear heads classify.\n\nMeasured out-of-sample, leakage-safe (by-video grouped CV over 1,031 video groups;\npermutation **p = 0.0005**). These are honest numbers, not a leaderboard.\n\n| Capability | Result | vs. chance |\n|---|---|---|\n| Is something wrong? (fault/normal) | AUROC 0.79 [0.76, 0.83] |\n0.50 |\n| Where in the car? (6 zones) | right zone in top-3 ≈ 75% |\n2× |\n| Which part? (12+ families) | right part in top-3 ≈ 45–65% |\n3–4× |\n| Knows when it doesn't know | calibrated (ECE ≈ 0.04), returns `UNCERTAIN` |\n— |\n\nFull details, and the one head we *demoted* for failing out-of-sample (knock), are\nin [docs/MODEL_CARD.md](/adam-s/car-diagnosis/blob/main/docs/MODEL_CARD.md).\n\nA fresh clone is immediately usable. A small pre-trained model ships in `models/`\n\n,\nand a synthetic demo clip is bundled, so nothing needs to be downloaded or scraped.\n\n```\ngit clone <this-repo> && cd car-diagnosis\nuv venv && source .venv/bin/activate\nuv pip install -e \".[scrape,web,dev,viz]\"     # Python 3.11\n\ncardiag doctor                 # preflight: what's installed\ncardiag train --fixtures       # a working model offline in ~2s (no scrape, no 2 GB download)\ncardiag diagnose <clip.wav>    # verdict + where-in-the-car + ranked parts\ncardiag serve --model models   # live web app: drop a clip / paste a link, \"explain why\"\n```\n\nVerify the whole thing end-to-end in an isolated worktree: `bash scripts/clone_verify.sh`\n\n.\n\n```\naudio ──► clean() cascade ──► CLAP embedding ──► linear heads ──► Diagnosis\n          (isolate spans)     (frozen, 512-d)    (fault/region/    (calibrated,\n                                                  part/knock)       UNCERTAIN-aware)\n```\n\nThere is **one segmentation path**. Scraped clips, your own recordings (`cardiag ingest`\n\n, any length), and uploads at inference all flow through the same `clean()`\n\ncascade that isolates short mechanical spans. Spans over ~10 s are split into windows\nso CLAP never silently truncates them. Training and serving share one embedding\ncontract, so there is no train/serve skew.\n\n```\ncardiag diagnose clip.wav            # full model: verdict + region + ranked parts\ncardiag triage   clip.wav            # calibrated engine-vs-running-gear\ncardiag clean    clip.wav            # isolate the mechanical sound (no model needed)\ncardiag inspect  clip.wav -o r.html  # SEE/HEAR the pipeline: spans, spectrograms, scores\ncardiag ingest   ./my_audio --kind fault --cause wheel_bearing   # bring your own audio\ncardiag scrape   youtube|tiktok      # build a corpus (Reddit is deprecated — too noisy)\ncardiag train                        # train on your corpus\n```\n\nAdd `--json`\n\nto any inference command for machine-readable output.\n\n[docs/DEFENSE.md](/adam-s/car-diagnosis/blob/main/docs/DEFENSE.md)— the honest case that a deliberately crude method earns a real triage result.[docs/MODEL_CARD.md](/adam-s/car-diagnosis/blob/main/docs/MODEL_CARD.md)— per-head metrics, intended use, limitations.[docs/architecture.md](/adam-s/car-diagnosis/blob/main/docs/architecture.md)— pipeline diagrams.[docs/scraping-guide.md](/adam-s/car-diagnosis/blob/main/docs/scraping-guide.md)— start-to-finish corpus building.\n\nValid for social-style / targeted-upload audio (YouTube, TikTok, or a phone clip a\nuser records deliberately). It is **not** a safety-critical or standalone diagnostic.\nIt is a triage assistant that narrows where to look and is honest about its\nuncertainty. Model files are joblib artifacts: load only ones you trust.\n\nLicense: see [LICENSE](/adam-s/car-diagnosis/blob/main/LICENSE).", "url": "https://wpnews.pro/news/show-hn-classify-mechanical-faults-using-contrastive-language-audio-pretraining", "canonical_source": "https://github.com/adam-s/car-diagnosis", "published_at": "2026-07-01 16:57:20+00:00", "updated_at": "2026-07-04 07:32:22.019492+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools"], "entities": ["CLAP", "YouTube", "TikTok", "cardiag"], "alternates": {"html": "https://wpnews.pro/news/show-hn-classify-mechanical-faults-using-contrastive-language-audio-pretraining", "markdown": "https://wpnews.pro/news/show-hn-classify-mechanical-faults-using-contrastive-language-audio-pretraining.md", "text": "https://wpnews.pro/news/show-hn-classify-mechanical-faults-using-contrastive-language-audio-pretraining.txt", "jsonld": "https://wpnews.pro/news/show-hn-classify-mechanical-faults-using-contrastive-language-audio-pretraining.jsonld"}}