I put a 45M-parameter LLM on a phone. It was 100% confident about a number it invented.

The model returned 1230.0

. The input said 560.00

. Confidence: 1.0000.

Not 0.6. Not "uncertain." One point zero, with a written rationale explaining how it got there — a rationale that quoted the correct number and then produced a different one.

That's the single most useful thing I learned building react-native-needle, a React Native binding for

Needle 2 isn't a chatbot. Give it a tool schema and it emits a schema-conforming call or declines. That's a narrower job than "generate text," and a much better fit for phones: no server, no API key, no per-token cost, no network, and nothing leaves the device.

The C API is four functions. That's the whole thing:

int  needle_load(const unsigned char* cact, unsigned long long n);
int  needle_init(const char* system_prompt, const char* tools_json, const char* tool_index_path);
int  needle_complete(const char* input, int max_new_tokens, char* out, int out_capacity);
void needle_reset(void);

I'd budgeted days for the native work. Four functions behind a JNI bridge and a CMake target is an afternoon.

arm64-v8a

linked cleanly. armeabi-v7a

did not:

undefined symbol: std::__ndk1::__hash_memory

Cactus's 32-bit archive is built against a newer libc++ than NDK 27 exports. No flag fixes that — it's an ABI mismatch in a prebuilt binary. So the package is arm64-only, with a needleSupported

flag so 32-bit devices degrade instead of crashing with an UnsatisfiedLinkError

.

Worth knowing before you plan around "it's only 14 MB": in the APK it's 28 MB — 14.5 MB of engine plus 13.7 MB of weights.

I ran eight measured cases on an arm64 emulator. Clean, single-intent commands are excellent:

That last row matters as much as the others. Asked something outside its tools, it returns an empty call list and a reason rather than inventing an answer.

For device control and voice-command routing, this is genuinely usable. 140 ms round trips, offline, free.

1. A leading unrelated clause kills the whole request.

Preheat to 200 C and bake 25 minutes, serves 4. Add 250 grams of flour to the list.

With an add_item

tool available, it declined:

{ "function_calls": [], "reasoning": "No tool available for preheating or baking." }

The second sentence on its own works fine. It latches onto the first instruction and gives up instead of scanning for the part it can serve. Segment multi-intent input yourself.

2. String fields over-capture. In a denser message, tracking

came back as "AB4471, ref 522119876543"

— the code plus the next field glued on.

3. Numbers in crowded strings get fabricated. The 1230.0

case above. Reproducible. Tightening the schema didn't help. Adding "copy verbatim, never calculate" to the system prompt produced byte-identical output.

Needle tells you when it's making things up:

"validation": { "ungrounded": ["record_item.amount"], "negation": false }

It flagged the exact field it had fabricated — in the same response where it reported confidence 1.0.

So: the confidence score was useless, and the self-report was correct. If you build on any model that exposes something like this, gate on the grounding flag and ignore the confidence number. I'd have shipped a wrong value if I'd trusted the metric that looked like the trustworthy one.

Documented memory is ~28 MB per session. Measured peak RSS: 500–530 MB. That's the real deployment constraint, not the model file size, and it's the thing that decides whether you can ship this on a low-RAM device.

I also published my own wrong numbers first. My initial notes said "9–15 s per completion." Re-measuring across eight runs gave 140 ms – 1.6 s. The original figure came from one cold run that included initialisation, and I'd written it down as steady-state. Measure more than once before you publish a benchmark — including in your own README.

I wrote the response parser from what I assumed the envelope looked like:

return parsed?.arguments ?? parsed?.parameters ?? parsed;

The real output nests one level deeper:

{"type":"call","function_calls":[{"name":"set_brightness","arguments":{...}}]}

So extract()

handed callers the whole envelope instead of their fields, and a decline came back as a truthy object — indistinguishable from a real answer. That shipped in 0.1.0 and was live for about half an hour.

The fix wasn't the interesting part. Rewriting the tests was: they now run against real captured device output, not my idea of the shape. Every one of my original hand-written tests passed against a format the model never emits.

files

in package.json

is an allowlist, and it beats .npmignore. My

.npmignore

excluded the 20 MB engine and 14 MB weights. npm ignored it and cut a !

negations inside files

:

"files": ["build", "android", "!android/libs", "!android/src/main/assets/*.cact"]

With that, 18.6 kB. There's now a prepack

check that fails the pack if those negations ever go missing, because the failure is invisible until you actually inspect the tarball.

The binding works and is on npm. The stock model is good at what it was built for — tool calling and device control — and unreliable at dense free-text numeric extraction, which is arguably off-label use.

The interesting next step is a LoRA fine-tune on domain data, which cactus-needle

supports. A 14 MB model that's actually good at one narrow job is a far more useful thing than a general one that's occasionally, confidently wrong.

npm install react-native-needle

Android/arm64, Expo SDK 51+. MIT; the engine is Apache-2.0 from Cactus Compute.