{"slug": "tagging-environmental-sounds-with-yamnet-tflite-an-esc-50-evaluation-over-2000", "title": "Tagging Environmental Sounds with YAMNet TFLite: An ESC-50 Evaluation over 2,000 Clips", "summary": "A developer evaluated the YAMNet TFLite model on the ESC-50 environmental sound dataset, achieving 60.45% top-1 accuracy across 2,000 clips. The experiment used a pretrained model without fine-tuning, with higher accuracy of 68.94% on 40 categories where label mappings were more direct.", "body_md": "Hello, everyone.\n\nToday is Tanabata in Japan. Events like Tanabata, summer dance festivals, and Christmas all seem to have their own sound images. They also feel like composites of multiple sound categories, such as music, effects, voices, and ambient noise, forming one overall impression.\n\nToday, I am testing how well the YAMNet TFLite model can tag the ESC-50 environmental sound dataset.\n\n[YAMNet](https://www.tensorflow.org/hub/tutorials/yamnet) is an audio tagging model that predicts 521 AudioSet sound event classes. In this experiment, I run the TFLite version published on TensorFlow Hub from Python and map its labels to the 50 categories in [ESC-50](https://github.com/karolpiczak/ESC-50).\n\nThe main questions are:\n\nThe complete code and reproducible environment are available in the [yamnet-esc50-audio-tagging lab in kiarina/labs](https://github.com/kiarina/labs/tree/main/2026/07/07/yamnet-esc50-audio-tagging).\n\nYou will need the following tools, an internet connection for the first run, and a few hundred MB of free space:\n\nThe following commands fetch only this lab and run it:\n\n```\ngit clone --depth 1 --filter=blob:none --sparse \\\n  https://github.com/kiarina/labs.git\ncd labs\ngit sparse-checkout set .gitignore .mise/tasks Makefile mise.toml \\\n  2026/07/07/yamnet-esc50-audio-tagging\nmise -C 2026/07/07/yamnet-esc50-audio-tagging run\n```\n\nOn the first run, the task downloads YAMNet TFLite, the class map, and the ESC-50 archive. Later runs verify the downloaded files by SHA-256 and reuse them.\n\nFor a short smoke test, run this inside the lab directory:\n\n```\nuv run python main.py --folds 1 --limit-per-class 1\n```\n\nThe model is the TFLite version of YAMNet from TensorFlow Hub.\n\n```\nmodel: https://tfhub.dev/google/lite-model/yamnet/classification/tflite/1?lite-format=tflite\nmodel SHA-256: 10c95ea3eb9a7bb4cb8bddf6feb023250381008177ac162ce169694d05c317de\nclass map revision: 5c597f85268743140854f0e670f2175e8668553a\nclass map SHA-256: cdf24d193e196d9e95912a2667051ae203e92a2ba09449218ccb40ef787c6df2\ninput: 1-D float32 waveform, 16 kHz mono\noutput: frame scores [frames, 521]\n```\n\nEach clip is converted with FFmpeg to 16 kHz mono float32 PCM and passed to the TFLite model. The frame scores are averaged into clip-level scores.\n\nThe input dataset is all 2,000 clips from ESC-50. ESC-50 contains five-second environmental recordings across 50 classes. Those 50 classes are also grouped into five coarse categories:\n\nThe dataset is pinned to revision `33c8ce9eb2cf0b1c2f8bcf322eb349b6be34dbb6`\n\n. The full ESC-50 dataset is distributed under CC BY-NC, so the noncommercial restriction applies to this experiment, which uses all 50 classes.\n\nYAMNet and ESC-50 do not use the same label set. I therefore manually assigned plausible YAMNet labels to each ESC-50 category.\n\nSome examples:\n\n| ESC-50 category | Mapped YAMNet labels |\n|---|---|\n`dog` |\n`Dog` , `Bark`\n|\n`cat` |\n`Cat` , `Meow` , `Purr`\n|\n`keyboard_typing` |\n`Typing` , `Computer keyboard`\n|\n`chainsaw` |\n`Chainsaw` |\n`washing_machine` |\n`Mechanisms` , `Water` , `Environmental noise`\n|\n\nFor each clip, the category score is the maximum score among the mapped YAMNet labels for that ESC-50 category. The category with the highest score becomes the prediction. I then report:\n\nThere is no training and no cross-validation. This is one pass over all 2,000 clips with a fixed pretrained model.\n\nOne run on a Mac Studio without a warm-up produced:\n\n| Scope | count | fine@1 | fine@3 | fine@5 | coarse@1 |\n|---|---|---|---|---|---|\n| all categories | 2,000 | 60.45% | 79.15% | 85.45% | 78.70% |\n| direct label mapping categories | 1,600 | 68.94% | 85.00% | 90.50% | 83.00% |\n\n`direct label mapping categories`\n\ncovers the 40 categories where I judged the ESC-50 category and YAMNet labels to be relatively direct matches. It includes categories such as `dog`\n\n, `cat`\n\n, `rain`\n\n, `keyboard_typing`\n\n, and `chainsaw`\n\n, and excludes categories such as `insects`\n\n, `drinking_sipping`\n\n, `can_opening`\n\n, and `washing_machine`\n\n.\n\nThe elapsed time was:\n\n```\n2000 clips elapsed: 57.985 s\nseconds per clip:   0.029 s\n```\n\nThis includes FFmpeg decoding, TFLite inference, mean aggregation of frame scores, and ESC-50 category scoring for all 2,000 files. It excludes model and dataset downloads.\n\nThe strongest fine-grained categories were:\n\n| category | fine@1 | fine@3 | coarse@1 |\n|---|---|---|---|\n`church_bells` |\n100.0% | 100.0% | 100.0% |\n`dog` |\n97.5% | 97.5% | 97.5% |\n`keyboard_typing` |\n97.5% | 97.5% | 97.5% |\n`train` |\n95.0% | 100.0% | 97.5% |\n`frog` |\n92.5% | 95.0% | 97.5% |\n\nThe hardest categories were:\n\n| category | fine@1 | fine@3 | coarse@1 |\n|---|---|---|---|\n`drinking_sipping` |\n0.0% | 2.5% | 20.0% |\n`hen` |\n0.0% | 92.5% | 92.5% |\n`mouse_click` |\n2.5% | 37.5% | 82.5% |\n`can_opening` |\n5.0% | 20.0% | 57.5% |\n`washing_machine` |\n5.0% | 25.0% | 12.5% |\n\n`hen`\n\nhad 0% fine@1, but 92.5% fine@3 and 92.5% [coarse@1](mailto:coarse@1). Most errors were predicted as `rooster`\n\n. The exact name was wrong, but the model still placed the clip near a very similar sound.\n\nThe most common confusions were:\n\n| actual | predicted | count |\n|---|---|---|\n`hen` |\n`rooster` |\n35 |\n`snoring` |\n`breathing` |\n28 |\n`pouring_water` |\n`water_drops` |\n27 |\n`mouse_click` |\n`keyboard_typing` |\n22 |\n`siren` |\n`clock_alarm` |\n20 |\n\nCoarse-category accuracy was:\n\n| coarse category | correct / count | coarse@1 |\n|---|---|---|\n| animals | 369 / 400 | 92.25% |\n| exterior and urban | 349 / 400 | 87.25% |\n| human non-speech | 293 / 400 | 73.25% |\n| natural soundscapes and water | 284 / 400 | 71.00% |\n| interior and domestic | 279 / 400 | 69.75% |\n\nFirst, YAMNet TFLite ran cleanly from Python. Processing 2,000 five-second clips in just under a minute makes it practical as a lightweight audio tagging step.\n\nThe result is not strong enough if you need one exact label among 50 categories every time. Overall fine@1 was 60.45%. However, the correct category appeared in the top five for 85.45% of all clips, and for 90.50% of the direct-mapping subset. That looks useful for generating candidate labels, feeding a downstream classifier, or initializing audio search metadata.\n\nCoarse accuracy rose to 78.70%. The animals and exterior/urban groups were especially stable. The interior/domestic group was weaker. Everyday indoor sounds often share broad labels such as `click`\n\n, `water`\n\n, or `mechanisms`\n\n, so there may not be enough information to separate the fine categories cleanly.\n\nThere are important caveats. This evaluation depends on a manual mapping from 521 YAMNet labels to 50 ESC-50 categories. If YAMNet assigns a high score to a nearby label that is not in my mapping, the result can be counted as wrong even when the acoustic interpretation is reasonable. The reverse can also happen: broad labels may accidentally raise a category score.\n\nThe ESC-50 repository also documents possible information leakage from class-dependent preprocessing of the original source recordings. These numbers should therefore be read as observations under this fixed setup, not as an absolute model benchmark.\n\nThe verification environment was:\n\nYAMNet feels different from a model like CLAP, which is more about semantic distance between sounds. It is easier to understand as a tool that directly attaches labels to what is currently audible. The top-line fine@1 number is in the 60% range, but the top candidates make it much more useful.\n\nErrors such as `hen → rooster`\n\n, `snoring → breathing`\n\n, and `pouring_water → water_drops`\n\nare also close enough that they do not feel random. If an application treats the output as candidates rather than final truth, the result looks like a reasonable coarse understanding of the sound.\n\nEveryday domestic sounds such as `washing_machine`\n\nand `drinking_sipping`\n\nlook harder. Categories that contain multiple mixed sounds, or categories without a clean YAMNet label, depend heavily on how the mapping table is written.\n\nStill, it is interesting to see sounds that create one impression from multiple combined categories broken back into top candidates. Rather than expecting YAMNet alone to be perfect, I would treat it as a lightweight entry point for audio tagging.", "url": "https://wpnews.pro/news/tagging-environmental-sounds-with-yamnet-tflite-an-esc-50-evaluation-over-2000", "canonical_source": "https://dev.to/kiarina/tagging-environmental-sounds-with-yamnet-tflite-an-esc-50-evaluation-over-2000-clips-1alb", "published_at": "2026-07-07 05:23:34+00:00", "updated_at": "2026-07-07 05:58:16.898886+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence"], "entities": ["YAMNet", "TFLite", "ESC-50", "TensorFlow Hub", "FFmpeg", "Mac Studio"], "alternates": {"html": "https://wpnews.pro/news/tagging-environmental-sounds-with-yamnet-tflite-an-esc-50-evaluation-over-2000", "markdown": "https://wpnews.pro/news/tagging-environmental-sounds-with-yamnet-tflite-an-esc-50-evaluation-over-2000.md", "text": "https://wpnews.pro/news/tagging-environmental-sounds-with-yamnet-tflite-an-esc-50-evaluation-over-2000.txt", "jsonld": "https://wpnews.pro/news/tagging-environmental-sounds-with-yamnet-tflite-an-esc-50-evaluation-over-2000.jsonld"}}