{"slug": "i-tried-to-build-an-ai-anime-on-a-16gb-macbook-here-is-exactly-where-it-broke", "title": "I tried to build an AI anime on a 16GB MacBook. Here is exactly where it broke", "summary": "A developer testing AI animation on a 16GB MacBook M1 Pro found that text-to-video generation with Wan 2.1 failed to complete, while image generation with Animagine XL 4.0 took about five minutes per image and struggled with hands. The developer also discovered that text-to-speech misread Japanese period vocabulary, highlighting the importance of checking TTS output for proper nouns.", "body_md": "I wanted to find out whether a laptop can make an animated short with AI. Not \"can a\n\nmodel generate a picture\" — the whole thing: music, voices, character art that stays\n\nthe same person across shots, and motion.\n\nThe machine is an M1 Pro with 16GB of unified memory. Everything below was measured on\n\nit over three days. Two parts worked better than I expected. One part did not work at\n\nall, and that is the part worth writing down, because \"it does not run on this\n\nhardware\" is the answer people actually need before they spend a weekend on it.\n\nI did not want to pay for Suno or Udio, so I wrote a synthesiser. Plucked strings are\n\nKarplus-Strong: fill a buffer the length of one period with noise, then walk it,\n\naveraging each sample with its neighbour. The averaging is a low-pass filter applied\n\nonce per period, so the high harmonics die first and you get a string.\n\nThe first version was unlistenable and I could not say why. So I measured it against a\n\nreference track — FFT, energy per band, stereo correlation, dynamic range:\n\n| Metric | Reference | My v1 | My v2 |\n|---|---|---|---|\n| bass 60-250Hz | 34.3% | 85.2% |\n32.4% |\n| mid 500-2kHz (melody) | 42.6% | 1.5% |\n49.4% |\n| simultaneous partials | 6 | 2 | 8 |\n| L/R correlation | 0.47 | 1.00 |\n0.10 |\n| dynamic range | 2.7dB | 9.8dB | 4.8dB |\n\nTwo things jump out. **85.2% of the energy was in the bass and 1.5% was in the band\nwhere melody lives** — the tune was not quiet, it was absent. And the L/R correlation\n\n48 seconds of finished audio synthesises in 3.5 seconds.\n\nFour attempts, in order: edge-tts (two Japanese voices total, so you cannot cast a\n\nscene), VOICEVOX (43 speakers, genuinely good, but built for utility narration),\n\n[Style-Bert-VITS2](https://github.com/litagin02/Style-Bert-VITS2) with an emotional corpus (seven emotions as a continuous weight), and\n\nfinally a higher-fidelity model.\n\nI spent that whole ladder assuming the flatness was a model-quality problem. It was\n\nnot. The script contained period vocabulary, and the TTS was reading it wrong:\n\n| Written | What the TTS said | Correct |\n|---|---|---|\n| 明智日向守 (a title: \"Akechi, Governor of Hyūga\") | Akechi Hyūga Mamoru\n|\nAkechi Hyūga no Kami\n|\n| 濃姫 (a name: \"Nōhime\") | No*o*hime |\nNohime |\n\nIt parsed 守 — the \"governor\" in a court title — as the given name *Mamoru*. That was in\n\nthe most dramatic line in the script. No amount of model swapping fixes a wrong reading.\n\nVOICEVOX's `/audio_query`\n\nendpoint returns the kana and accent position it is about to\n\nuse, so you can check every proper noun before you synthesise anything. I had never\n\nlooked at it.\n\nPractical notes if you go down this path: Style-Bert-VITS2 needs Python 3.11 (pyopenjtalk\n\ndoes not build on 3.13), and the distributed BERT weights are fp16 while the synthesis\n\nside runs fp32, so CPU inference dies with `Input type (c10::Half) and bias type (float)`\n\nuntil you cast it.\n\nshould be the same\n\n[Animagine XL 4.0](https://huggingface.co/cagliostrolab/animagine-xl-4.0), 832x1216, 28 steps: **about 5 minutes per image** on MPS.\n\nCharacter consistency held. Same seed plus the character's appearance written out\n\nidentically in every prompt, and the same person appears in a rain-lit corridor and at a\n\nbanquet. That was the risk I expected to sink the project, and it did not.\n\nWhat broke was hands. A shot described as \"pouring sake into a cup\" produced **three\nhands**. I had chosen close-ups of hands deliberately — the source material has no male\n\nAlso, without negative prompts for it, a 1560 Japanese castle grows roses, and a naginata\n\nbecomes a katana.\n\n[Wan 2.1](https://github.com/Wan-Video/Wan2.1) T2V 1.3B, Apache-2.0, through [ComfyUI](https://github.com/comfyanonymous/ComfyUI). 832x480, 33 frames — about two seconds of\n\nfootage — at 20 steps.\n\n| Progress after 90 minutes | 9 of 20 steps |\n| Process CPU | 10.4% |\n| Swap in use | 23.3GB of 24.5GB |\n| System memory free | 19% |\n\nIt never finished. The CPU figure is the tell: the process was not computing, it was\n\nwaiting on disk. The text encoder alone is 6.7GB; add the diffusion weights, the VAE and\n\nthe intermediate tensors and you exceed 16GB of unified memory, and the overflow goes to\n\nSSD. Swap cannot be read at the speed a sampler wants it.\n\nSo: not \"slow\". Not running. If you want generated motion on this class of machine, the\n\nhonest options are a hosted model or a rented GPU.\n\nI downloaded the same model twice, in two different ways, and did not notice until\n\nsomeone asked why the disk was filling up.\n\n`hf_hub_download`\n\nputs the file in a cache and returns the path; copying it to your\n\nmodels directory leaves **two full copies** — 21GB in my case. Then, holding\n\nComfyUI-format weights already, I called `diffusers`\n\n' `from_pretrained()`\n\n, which\n\ndownloaded the entire Diffusers-format repository of the same model — another 5.2GB.\n\nMove the file instead of copying it, delete the cache entry after, and if you already\n\nhave ComfyUI-format weights, drive ComfyUI rather than handing the name to diffusers.\n\nSynthesis and stills are yours for free and they are good. Prosody is a data problem\n\nbefore it is a model problem — check what your TTS thinks the words are. Composition\n\nchoices decide whether generation breaks, so avoid hands. And measure the video step\n\nbefore you plan around it: an hour and a half of swap thrashing for nine steps is the\n\nkind of number that changes a project, and it takes one run to find.\n\n*Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.*", "url": "https://wpnews.pro/news/i-tried-to-build-an-ai-anime-on-a-16gb-macbook-here-is-exactly-where-it-broke", "canonical_source": "https://dev.to/morinaga/i-tried-to-build-an-ai-anime-on-a-16gb-macbook-here-is-exactly-where-it-broke-18g3", "published_at": "2026-09-02 10:05:42+00:00", "updated_at": "2026-09-02 10:23:36.177138+00:00", "lang": "en", "topics": ["generative-ai", "ai-tools", "ai-infrastructure"], "entities": ["MacBook M1 Pro", "Wan 2.1", "Animagine XL 4.0", "ComfyUI", "Style-Bert-VITS2", "VOICEVOX", "edge-tts"], "alternates": {"html": "https://wpnews.pro/news/i-tried-to-build-an-ai-anime-on-a-16gb-macbook-here-is-exactly-where-it-broke", "markdown": "https://wpnews.pro/news/i-tried-to-build-an-ai-anime-on-a-16gb-macbook-here-is-exactly-where-it-broke.md", "text": "https://wpnews.pro/news/i-tried-to-build-an-ai-anime-on-a-16gb-macbook-here-is-exactly-where-it-broke.txt", "jsonld": "https://wpnews.pro/news/i-tried-to-build-an-ai-anime-on-a-16gb-macbook-here-is-exactly-where-it-broke.jsonld"}}