{"slug": "running-apple-s-on-device-ai-from-flutter-with-no-api-key-and-no-server", "title": "Running Apple's On-Device AI From Flutter, With No API Key and No Server", "summary": "A developer has created three Flutter packages that enable on-device AI features from Apple's iOS 26 and macOS 26, including a language model, image generator, and LiDAR room scanner, without requiring API keys or servers. The packages support schema-constrained structured output and tool calling, ensuring user data remains on the device. The developer notes the on-device model is not a frontier model but is effective for summarization, extraction, and classification tasks.", "body_md": "Every \"add AI to your Flutter app\" tutorial ends the same way: get an API key,\n\nadd a billing card, send the user's text to somebody else's server, and hope the\n\nlatency is tolerable.\n\nOn iOS 26 and macOS 26 there is a different option that most Flutter developers\n\nhave not tried, because the plugins did not exist. Apple ships a language model,\n\nan image generator and a LiDAR room scanner **on the device**. No key, no\n\naccount, no per-token cost, no network — and nothing the user types leaves their\n\nphone.\n\nI built three packages to reach them from Flutter. This is what each one can\n\ngenuinely do, and where each one stops.\n\n```\nfinal session = await LanguageModelSession.create(\n  instructions: 'You summarise text in one sentence.',\n);\nprint(await session.respond(article));\nawait session.dispose();\n```\n\nThat runs entirely on the device. It streams, it supports tool calling, and —\n\nthe part that matters most in practice — it does **schema-constrained structured\noutput**. You describe the shape you want and the model is constrained to produce\n\nIf you have ever written a retry loop around `jsonDecode`\n\nbecause a hosted model\n\nreturned prose instead of JSON, that is the problem this removes at the source.\n\nIt is worth dwelling on this one, because it is the difference between a demo\n\nand something you can put in a product.\n\nThe usual pattern with a hosted model is: write a prompt asking politely for\n\nJSON, get back something that is *usually* JSON, wrap `jsonDecode`\n\nin a\n\ntry/catch, add a retry, and accept a failure rate you cannot drive to zero. Every\n\nproduction LLM integration has this scar tissue.\n\nSchema-constrained generation removes the problem rather than mitigating it. The\n\nmodel is constrained *during sampling* to tokens that keep the output valid\n\nagainst your schema. It is not being asked to produce the right shape; it is\n\nunable to produce a wrong one.\n\nFor anything where the model's output feeds code rather than a human — extract\n\nthese fields, classify into these categories, pull the dates out of this text —\n\nthat is the difference between a feature you ship and a feature you keep\n\napologising for.\n\nThe model can also call functions you expose. The pattern is the familiar one —\n\nyou describe the tools, the model decides when to call them, you run them and\n\nhand back results — but with the round trip happening entirely on the device.\n\nThe practical use is less \"agents\" and more \"give the model access to the user's\n\nown data without that data going anywhere\". A tool that reads the user's local\n\nnotes or transactions lets the model answer questions about them, while the notes\n\nthemselves never leave the phone. That is not achievable with a hosted model at\n\nany price.\n\n**Where it stops.** This is a small model. It is very good at summarising,\n\nextracting, classifying, rewriting and answering questions about text you give\n\nit. It is not a frontier model and it will not reason its way through a hard\n\nnovel problem. Treat it as a fast, free, private text-processing engine and it\n\nis excellent. Treat it as a substitute for a large hosted model and it will\n\ndisappoint you.\n\n[ apple_foundation_models](https://devshakib.jumyn.com/packages/apple_foundation_models) — iOS and macOS.\n\n``` js\nawait for (final image in ImageCreator.generate(prompt)) {\n  setState(() => _preview = image);\n}\n```\n\nImage Playground generates on the device too, and the API streams **partial\nimages as they are made**. That is not a nicety, it is the entire user\n\n| first image | 6.8s |\n| second image | 10.3s |\n| size | ~4 MB PNG each |\n\nSeven to ten seconds of a spinner feels broken. Seven to ten seconds of an image\n\nvisibly resolving in front of you feels like something is being made for you.\n\nSame wait, completely different experience — which is why the package streams by\n\ndefault rather than offering it as an option.\n\nThe same package also wraps the native text field that carries **Writing Tools**\n\nand **Genmoji**, so users get the system rewrite, proofread and summarise\n\naffordances they already know from Notes and Mail, in your app, without you\n\nbuilding any of it.\n\nLook at the timings again and think about what the alternative looks like.\n\nA modal spinner for 6.8 seconds is a long time. Long enough that a meaningful\n\nshare of users will assume it has hung and back out. Long enough that on the\n\nsecond image, at 10.3 seconds, some will kill the app.\n\nThe same 10.3 seconds spent watching an image emerge from noise into shape is not\n\na wait at all — it is the product. The user is watching their idea being made.\n\nNobody backs out of that, because there is visible progress on every frame.\n\nThis is the general lesson for on-device generation, not a quirk of this API. The\n\ncompute is slower than a datacentre GPU. You cannot fix that. What you can do is\n\nmake the latency *visible and interesting* rather than hidden behind a spinner,\n\nand the perceived experience inverts.\n\nBudget around **4 MB per PNG**, which matters if you are storing or syncing them.\n\n[ apple_intelligence](https://devshakib.jumyn.com/packages/apple_intelligence) — iOS and macOS.\n\n```\nfinal controller = RoomScanController();\n```\n\nRoomPlan uses the LiDAR sensor to scan a room and hand back structured geometry:\n\n`walls` , `floors`\n|\nsurfaces with metre dimensions and a transform |\n`doors` , `windows` , `openings`\n|\nfound within the walls |\n`objects` |\nrecognised furniture — chair, table, bed, storage… |\n`usdzPath` |\na USDZ model, ready for AR Quick Look |\n`raw` |\nRoomPlan's own encoding, untouched |\n\nReal dimensions in metres, not a point cloud you have to interpret. For anyone\n\nbuilding for interiors, property, removals, insurance or furniture retail, that\n\nis the difference between a demo and a product.\n\nThe package deliberately hosts **Apple's own scanning UI** rather than\n\nreimplementing it. Users have seen that interface in Apple's Measure app; the\n\ncoaching overlays that tell you to move slower or point at the wall are tuned by\n\npeople with the sensor data. Replacing that with a custom camera view would be\n\nworse in every way that matters.\n\n[ roomplan](https://devshakib.jumyn.com/packages/roomplan) — iOS, LiDAR required.\n\nThe USDZ output gets the attention because it drops straight into AR Quick Look.\n\nThe structured geometry is the more valuable half.\n\nBecause you get walls with metre dimensions, doors and windows located within\n\nthem, and recognised furniture, you can answer questions rather than just display\n\na model: *will this sofa fit through that door?* *how much paint for these\nwalls?*\n\nThat is the difference between a scanning feature and a product. Estate agents,\n\nremovals firms, insurers, interior retail and trades all have a version of that\n\nquestion, and all of them currently answer it with a tape measure and a phone\n\ncall.\n\nThese are Apple frameworks, so the constraints are Apple's:\n\nWhich is why the honest framing is not \"replace your hosted model\". It is: for\n\nthe substantial set of tasks these handle, you get them for free, instantly,\n\noffline, and without any user data leaving the device — and you fall back to\n\nwhatever you use today when the device cannot help.\n\nThat fallback path is not a workaround. It is the design.\n\nThere are three separate questions and they have three different answers, which\n\nis why a single `isSupported`\n\nboolean is not enough:\n\nOnly the third can change while your app is running, and it is the one people\n\nforget. Query availability at the point of use rather than caching a result from\n\nlaunch, and design the UI so an unavailable feature is *absent* rather than\n\npresent-and-failing. A greyed-out button with a tooltip explaining an OS\n\nrequirement is worse than not showing the button, because it advertises\n\nsomething the user cannot act on.\n\nThree reasons that survive contact with a real product.\n\n**Cost.** Per-token pricing turns every active user into a running bill.\n\nOn-device inference costs nothing and does not scale with success.\n\n**Latency.** No round trip. For summarise-as-you-type or classify-on-paste, the\n\ndifference between 40ms and 900ms is the difference between a feature people use\n\nand one they wait for.\n\n**Privacy you can state simply.** \"Your notes never leave your phone\" is a\n\nsentence you can put in an App Store listing and defend. It is also the only\n\nversion of that promise that is true without a legal team.\n\nTo be clear about when *not* to reach for these:\n\n| On-device | Hosted | |\n|---|---|---|\n| Cost per call | none | per token |\n| Latency | ~40ms to first token | 300ms–2s round trip |\n| Works offline | yes | no |\n| Data leaves device | no | yes |\n| Reasoning depth | limited | frontier |\n| Platform | iOS/macOS 26+ | anywhere |\n| Availability | user must enable | always |\n\nThe right architecture for most apps is both: on-device for the high-frequency,\n\nlow-complexity, privacy-sensitive work, hosted for the occasional hard question,\n\nand a capability check deciding which one runs. That is not a compromise, it is\n\njust using the cheaper tool for the easier job.\n\nAll three packages are MIT licensed with public source, and all three hold a\n\nperfect 160/160 on pub.dev. Start with whichever problem you actually have —\n\n[the full list is here](https://devshakib.jumyn.com/packages).\n\n*Originally published at devshakib.jumyn.com. I write about Flutter, Dart and the parts of shipping that are genuinely awkward — and publish the packages that came out of them at pub.dev/publishers/jumyn.com.*", "url": "https://wpnews.pro/news/running-apple-s-on-device-ai-from-flutter-with-no-api-key-and-no-server", "canonical_source": "https://dev.to/devshakib/running-apples-on-device-ai-from-flutter-with-no-api-key-and-no-server-1o78", "published_at": "2026-09-02 19:08:58+00:00", "updated_at": "2026-09-02 19:24:57.734588+00:00", "lang": "en", "topics": ["developer-tools", "ai-products", "artificial-intelligence", "large-language-models", "generative-ai"], "entities": ["Apple", "Flutter"], "alternates": {"html": "https://wpnews.pro/news/running-apple-s-on-device-ai-from-flutter-with-no-api-key-and-no-server", "markdown": "https://wpnews.pro/news/running-apple-s-on-device-ai-from-flutter-with-no-api-key-and-no-server.md", "text": "https://wpnews.pro/news/running-apple-s-on-device-ai-from-flutter-with-no-api-key-and-no-server.txt", "jsonld": "https://wpnews.pro/news/running-apple-s-on-device-ai-from-flutter-with-no-api-key-and-no-server.jsonld"}}