Needle 2 Hands-On: 14MB Model Runs Local AI Agent, No GPU Required

When the entire AI industry chases hundred-billion-parameter giants, a company called Cactus Compute has taken the exact opposite path — their newly released Needle 2 is a model with just 45M parameters and 14MB in size, yet it runs a full AI Agent at 500 tokens per second on a Raspberry Pi.

This isn’t a lab concept. Needle 2 is already commercially deployed in the Pebble Index 01 smart ring, where users speak to the ring and trigger device actions without any internet connection.

In this article, we deep-dive into this “counter-intuitive” model: what can it actually do? What can’t it do? And where does it sit in the 2026 edge AI landscape?

What Is Needle 2?

Needle 2 is an open-source edge Agent model developed by Cactus Compute. Their core philosophy: not every AI task needs a large model — in many scenarios, a sufficiently small and fast specialized model is actually better.

Needle 2’s positioning is crystal clear — it doesn’t chat, write articles, or answer questions. It does exactly three things:

1. Tool Calling: Maps natural language instructions to specific function calls
2. Device Use: Executes UI action sequences on mobile devices
3. Structured Extraction: Extracts schema-conforming structured data from text

In one sentence: Needle 2 is not a “conversational AI” — it’s an “action AI.”

Core Specifications

Metric	Needle 2	Notes
Parameters	45M	45 million parameters
Model Size	14 MB	Single binary file
Runtime RAM	28 MB	Peak session RAM
GPU Required	None	Pure CPU inference
Quantization	CQ2-bit	Proprietary 2-bit quantization
Context Window	256 tokens	Sliding window, tool declarations pinned
Decode Speed (RPi 5)	500+ tok/s	CPU-only
Decode Speed (Quest 3S)	400-1,500 tok/s	VR device
Decode Speed (Samsung A-Series)	300-700 tok/s	Sub-$200 phone
Pretraining Data	115B tokens	Proprietary corpus
Post-training Data	38B tokens	With compact reasoning traces
License	MIT	Fully open source

What does 28MB RAM mean? An ESP32-S3 microcontroller has 8MB PSRAM, and newer microcontrollers have enough memory to run Needle 2. This means a few-dollar chip can run an AI Agent.

Architecture: Simple Attention Network

The reason Needle 2 can be this small and still work lies in its architecture — the Simple Attention Network (SAN). This isn’t a tweak to Transformer; it’s a ground-up redesign.

Key Innovations

1. Hadamard MLP replaces traditional FFN

Traditional Transformer feed-forward networks are parameter hogs. Needle 2 replaces them with a fixed Walsh-Hadamard transform plus learned diagonal matrices — channel mixing at almost zero parameter cost.

2. Engram Memory System

World knowledge lives not in model weights but in hashed n-gram tables. Only a few rows are read per token, nearly zero cost at decode time. Critical for embedded devices where flash reads are both power and latency expensive.

3. Multi-Lane Residual Streams

A 27-layer, 512-wide network gains routing flexibility far beyond its width through multi-lane design, at the cost of just a few dot products per layer.

4. Bounded Memory Attention

A 256-token sliding window ensures the KV cache is always bounded. Tool declarations are pinned as permanent “sinks” — a tool-calling model must never forget its tools, and this makes it structurally impossible.

5. Train = Quantize

The most elegant design. Needle 2 isn’t quantized after training (that would destroy a small model) — it’s trained at CQ2-bit precision from pretraining through post-training: weights, activations, and KV cache are all 2-bit. The 2-bit model you deploy IS the model that was trained, with no accuracy loss.

Needle 1 → Needle 2: Key Improvements

Dimension	Needle 1	Needle 2	Improvement
Parameters	26M	45M	+73%
Model Size	~10 MB	14 MB	Slight increase
Capabilities	Tool calling	Tool calling + device use + extraction	Major expansion
Pretraining Data	Undisclosed	115B tokens	First disclosure
Post-training	Undisclosed	38B tokens (with reasoning)	First disclosure
Tool Retrieval	None	Built-in retrieval head, Top-5 per turn	New
Confidence Score	None	Calibrated confidence per response	New
Fine-tuning	Full	LoRA + merge export	More practical
Browser Playground	None	Yes, WebAssembly	New

Comparison: Needle 2 vs Same-Tier Models

Model	Parameters	Size	RAM	Tool Call	Device Use	Extraction	Open Source
Needle 2	45M	14 MB	28 MB	✅	✅	✅	MIT
FunctionGemma 270M	270M	~540 MB	~1 GB	✅	❌	❌	✅
LFM2.5 230M	230M	~460 MB	~800 MB	✅	❌	❌	✅
Apple Foundation Model	Undisclosed	~hundreds MB	~hundreds MB	✅	✅	Partial	❌
Meta Muse Glimmer 30B	30B	~18 GB	~20 GB	✅	✅	✅	✅

Key insight: Needle 2 trades wins with FunctionGemma 270M and LFM2.5 230M on Mobile-Actions benchmarks — but at 5× to 70× fewer parameters, running at 2 bits against their f16.

Deployment Quick Start

Python

pip install cactus-needle

import needle

@needle.tool
def get_weather(city: str):
    """Get current weather for a city."""
    return {"city": city, "temp_c": 27, "sky": "clear"}

agent = needle.Needle(tools=[get_weather])
print(agent.run("What's it like in Lagos?")["results"])

Structured Extraction

from pydantic import BaseModel

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

invoice = needle.extract("Invoice from Acme Corp, $1,200.00, due 2026-09-01", Invoice)
print(invoice.vendor, invoice.total)  # -> Acme Corp 1200.0

Fine-tuning

needle generate-data --tools my_tools.json --num-samples 500 --output data.jsonl
needle finetune data.jsonl --epochs 10
needle build checkpoints/needle2.pkl --lora checkpoints/needle_lora.pkl --out my_needle.cact

Use Cases and Limitations

✅ Best For

• Smart home voice control
• Wearable devices (watches, rings)
• Offline devices needing local inference
• IoT devices (21B+ devices, most with < 100MB RAM)
• Privacy-sensitive scenarios
• Budget phones under $200
• Browser-based AI via WebAssembly

❌ Not For

• Open-domain conversation
• Long document understanding
• World knowledge Q&A
• Complex reasoning
• Multimodal tasks

Bottom line: If your task is “map natural language to predefined function calls,” Needle 2 may be the most efficient choice. If you need a model to “understand” the world and generate open text, it’s the wrong tool entirely.

Final Verdict

Dimension	Rating	Notes
Technical Innovation	⭐⭐⭐⭐⭐	SAN architecture + train-time quantization
Practicality	⭐⭐⭐⭐	Accurate tool calling, trivial deployment, but limited scope
Ecosystem Maturity	⭐⭐⭐	Python package + Playground + fine-tuning chain available
Open Source Commitment	⭐⭐⭐⭐⭐	MIT license, everything open
Value	⭐⭐⭐⭐⭐	14MB + 28MB RAM, runs on a $35 Raspberry Pi

Needle 2 isn’t trying to replace the large model on your computer — it’s about giving devices that never had AI the ability to be intelligent for the first time. When 2.1 billion IoT devices can all run local Agents, that’s when AI is truly everywhere.

---

FAQ

What’s the difference between Needle 2 and Needle 1?

Needle 2 increases parameters from 26M to 45M, adds device use and structured extraction (Needle 1 only did tool calling), and adds tool retrieval and confidence gating.

Can Needle 2 run on a phone?

Yes. On Samsung A-Series phones under $200, decode speed is 300-700 tok/s. Pebble has integrated Needle 2 in the Index 01 smart ring, running fully offline.

Does Needle 2 need a GPU?

Not at all. Pure CPU inference, 500+ tok/s on a Raspberry Pi 5. The entire model is 14MB and needs only 28MB RAM.

What can and can’t Needle 2 do?

Excels at: tool calling, device control, structured data extraction. Not suited for: open-domain chat, long document understanding, world knowledge Q&A. It’s an “action AI,” not a “conversational AI.”

How do I fine-tune Needle 2 for my own tools?

Install with pip install cactus-needle, then LoRA fine-tune: prepare JSONL data → needle finetune → needle build to export a single .cact file. Fine-tuning runs on Mac/PC in minutes to hours.