{"slug": "turn-text-into-actions-with-needle-14mb", "title": "Turn Text into Actions with Needle, 14Mb", "summary": "Cactus Compute released Needle 2, a 14MB function-calling LLM that runs entirely on a Raspberry Pi 5's CPU and selected the correct tool for the prompt \"Turn the LED on\" in 78 milliseconds. The model, installed via the cactus-needle Python package, uses the @needle.tool decorator to build tool schemas from function names, docstrings and type annotations, and its native session stays around 28MB while the full Python demo process peaked between 43MB and 46.4MB. Needle 2 is not a chatbot: asked \"What is the capital of France?\" it returned an empty function_calls list in 92ms, which the team calls the correct response for an action model.", "body_md": "# Turn text input into actions with Needle, a 14MB function-calling LLM\n\n**In this post, our friends from Cactus Compute show how Needle 2, a 14MB function-calling model, turns plain English into local actions on a Raspberry Pi 5, using the CPU alone.**\n\nType “Turn the LED on” and 78 milliseconds later, the LED on our Raspberry Pi 5 comes on. That is Needle 2 running on the CPU alone – without a dedicated AI HAT.\n\nLet’s start with what Needle is *not*. It is not a chatbot. Instead, the team at [Cactus Compute](https://www.cactuscompute.com) trained it for one task only: reliable, structured on-device actions.\n\n“Needle 2 is rather excellent” \n\n– Eben Upton, CEO, Raspberry Pi\n\nThe setup is simple – you declare Python functions, then Needle picks which one to call and fills in the arguments. We used it to write local notes, query `vcgencmd`, interact with the LEDs, and more on a CPU-only Raspberry Pi 5.\n\n```\npython3 -m venv needle-env\n\nsource needle-env/bin/activate\n\npython -m pip install cactus-needle\n```\n\nAfter the first download, Needle runs locally without a cloud API or network connection. This first example needs no extra hardware.\n\n``` python\nimport needle\n\nfrom pathlib import Path\n\nimport subprocess\n\nnotes_path = Path(\"needle-notes.txt\")\n```\n\nThe `Needle` class accepts functions decorated with `@needle.tool`. The decorator uses each function’s name, docstring, and type annotations to build its tool schema. In `save_note` below, the annotation tells Needle that `text` must be a string.\n\n``` python\n@needle.tool\n\ndef save_note(text: str):\n\n    \"\"\"Append a note to a local text file.\"\"\"\n\n    with notes_path.open(\"a\", encoding=\"utf-8\") as notes:\n\n        notes.write(text + \"\\n\")\n\n    return {\"text\": text, \"path\": str(notes_path)}\n\n@needle.tool\n\ndef get_temperature():\n\n    \"\"\"Return the current CPU temperature of this Raspberry Pi in Celsius.\"\"\"\n\n    out = subprocess.check_output([\"vcgencmd\", \"measure_temp\"], text=True)\n\n    return {\"temperature_c\": float(out.split(\"=\")[1].split(\"'\")[0])}\n\nagent = needle.Needle(tools=[save_note, get_temperature])\n\nresponse = agent.run(\"Save a note that says the cooler is working.\")\n```\n\nInside `run()`, the model first produces this tool selection (output trimmed for clarity):\n\n```\n{\n\n  \"type\": \"call\",\n\n  \"function_calls\": [\n\n    {\n\n      \"name\": \"save_note\",\n\n      \"arguments\": { \"text\": \"the cooler is working\" }\n\n    }\n\n  ]\n\n}\n```\n\n`run()` executes the function and includes its return value in `response[\"results\"]`:\n\n```\n[\n\n  {\n\n    \"text\": \"the cooler is working\",\n\n    \"path\": \"needle-notes.txt\"\n\n  }\n\n]\n```\n\nIn this run, the initial `complete()` inside `run()` selected the function and produced its arguments in 107ms.\n\nWe can also ask:\n\n```\nagent.reset()\n\nresponse = agent.run(\"How hot is this Raspberry Pi?\")\n```\n\nNeedle calls:\n\n```\n{\n\n  \"type\": \"call\",\n\n  \"function_calls\": [\n\n    {\"name\": \"get_temperature\", \"arguments\": {}}\n\n  ]\n\n}\n```\n\n`run()` executes `vcgencmd` and includes the reading in its results:\n\n```\n{ \"temperature_c\": 49.9 }\n```\n\nWe also declared `set_led(on)`, `blink_led(times)`, and `take_photo()` via `rpicam-still` to test tool selection. Needle chooses the tool; ordinary Python code decides what happens next. You can similarly decorate any Python function that has a clear name, type annotations, and description with `@needle.tool`.\n\nNeedle is deliberately narrow. Ask “What is the capital of France?” and it returns `{ \"function_calls\": [] }` in 92ms because none of the tools we gave it can answer that question. For an action model, refusing an unrelated request is the correct response.\n\nNeedle’s native session stays around 28MB. In this Python demo, the complete process peaked between 43MB and 46.4MB, including the interpreter. Here are a few more runs:\n\n| **Prompt** | **Call** | **Prefill (tok/s)** | **Decode (tok/s)** | **Time taken (ms)** | \n| Turn the LED on. | `set_led` | 488 | 296 | 78 | \n| How hot is this Raspberry Pi? | `get_temperature` | 487 | 303 | 149 | \n| Blink the LED 2 times. | `blink_led` | 475 | 314 | 83 | \n| Take a photo. | `take_photo` | 461 | 248 | 76 | \n| Save a note that says the cooler is working. | `save_note` | 475 | 305 | 107 | \n| What is the capital of France? | *(none)* | 470 | 297 | 92 | \n\nThese examples were run on a Raspberry Pi 5, 8GB, Raspberry Pi OS, CPU only, `cactus-needle` 2.0.7. **Prefill** and **decode** are Needle’s session counters. Each latency in the **Time taken** column is wall-clock time for one `complete()` call, before the selected Python function runs.\n\nIf you already write GPIO Zero functions, you can decorate them and pass them in. The same goes for other application logic. Needle can also be fine-tuned locally on a laptop for a particular set of tools.\n\nYou’ll find the weights and code at [huggingface.co/Cactus-Compute/needle2](https://huggingface.co/Cactus-Compute/needle2), [github.com/cactus-compute/needle](https://github.com/cactus-compute/needle). Both are released under Apache 2.0.", "url": "https://wpnews.pro/news/turn-text-into-actions-with-needle-14mb", "canonical_source": "https://www.raspberrypi.com/news/turn-text-input-into-actions-with-needle-a-14mb-function-calling-llm/", "published_at": "2026-09-24 05:40:23+00:00", "updated_at": "2026-09-24 06:01:33.693346+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-tools", "ai-products", "developer-tools"], "entities": ["Cactus Compute", "Needle 2", "Raspberry Pi 5", "Eben Upton", "Raspberry Pi", "cactus-needle", "vcgencmd", "rpicam-still"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/turn-text-into-actions-with-needle-14mb", "markdown": "https://wpnews.pro/news/turn-text-into-actions-with-needle-14mb.md", "text": "https://wpnews.pro/news/turn-text-into-actions-with-needle-14mb.txt", "jsonld": "https://wpnews.pro/news/turn-text-into-actions-with-needle-14mb.jsonld"}}