Needle2

The 14MB Bet: Why Smaller Is Smarter for Edge AI

When most of the AI industry is racing to build ever-larger models, Cactus Compute has gone the other direction. Needle2 is an open 45M-parameter model that fits into a single 14MB binary, designed for tool calling, device use, and structured extraction on edge devices. The bet is simple: turning on a light or locking a door doesn't require a frontier model with billions of parameters. It requires a model that can map a messy sentence to a function call with the right arguments—a task that needs no world knowledge or open-ended prose. That smaller formulation is what makes Needle2 possible, and it's a bet that challenges the industry's assumption that bigger is always better.

The company's own numbers put this in perspective: there are more than 21 billion IoT devices against roughly 1.5 billion PCs, and most phones in emerging markets ship under $200. These are the devices Needle2 targets—no GPU, no NPU, just a few dozen MB of RAM. By focusing on this underserved segment, Cactus Compute is staking out a position that's both practical and contrarian.

Inside the Simple Attention Network: Architecture Without the Fat

Needle2's architecture is a departure from conventional transformers. The Simple Attention Network (SAN) replaces dense MLP projections with a fixed Walsh-Hadamard transform, reducing parameter count while maintaining routing flexibility. The model also uses an 'engram'—hashed n-gram tables that store world knowledge outside the main stack, read a few rows per token. This design cuts compute per token to 70 MFLOPs, compared to 164 for a same-shape transformer and 460 for LFM2.5 230M.

The memory system is built for fixed-RAM devices: a 256-token sliding window bounds the KV cache, and system prompts and tool declarations are pinned as permanent sinks, ensuring the model never forgets its tools. This isn't just a clever trick; it's a structural guarantee that matters for real-world deployment.

Benchmarking Against Giants: How Needle2 Trades Wins

On public benchmarks like Mobile Actions and DroidCall, Needle2 holds its own against models 5 to 70 times larger. On Mobile Actions, it scores 63.7% accuracy (ordered strict exact match) versus 69.1% for LFM2.5 230M and 64.0% for FunctionGemma 270M—both running at f16, while Needle2 runs at CQ2-bit. On DroidCall, it beats LFM2.5 (17.0% vs 11.0%) and nearly matches FunctionGemma (17.5%).

The company is transparent about the asymmetries: baselines run at f16 because post-hoc quantization collapses models not trained for it, and Needle2 is specialized for tool calling while baselines are general models. These trade-offs favor Needle2 in some ways and the baselines in others, but the results still show that a 45M model can compete in the narrow domain of function calling.

Deployment Realities: From Raspberry Pi to ESP32

Needle2's speed and memory footprint make it practical for hardware that was previously too constrained for AI. On a Raspberry Pi 5, it hits 500 tokens/sec decode; on VR devices like Meta Quest 3S, 400–1,500 tokens/sec; on sub-$200 phones, 300–700 tokens/sec. Session RAM peaks at 28MB, which fits on microcontrollers like ESP32-S3 with external PSRAM.

The engine is a single dependency-free C++ binary that probes the CPU at startup and selects kernels—SDOT, NEON, AVX2, RISC-V vectors, wasm SIMD, or scalar. It runs from Cortex-M to x86 to WebAssembly, and the grammar compiler prevents malformed JSON. This isn't a research toy; it's a production-ready artifact.

The 'Needle' Name: Precision in a Haystack of Models

The name 'Needle' evokes precision—finding the right tool in a haystack of possibilities. It's a fitting metaphor for a model that excels at mapping user intent to specific function calls. The tagline, 'Agentic LLM for tiny devices,' reinforces the focus on edge hardware. The domain, cactuscompute.com/needle, ties it to the parent company, Cactus, which also offers hybrid cloud and engine products. The name is short, memorable, and suggests a sharp, focused tool rather than a general-purpose assistant.

This naming choice positions Needle2 as a specialist, not a generalist. It's a deliberate contrast to models like LFM2.5 or FunctionGemma, which sound more generic. In a market crowded with 'GPT' and 'Llama' clones, 'Needle' stands out for its clarity and purpose.

Open Questions and the Road Ahead

Needle2 is an impressive technical achievement, but questions remain. The company doesn't disclose funding or detailed customer metrics beyond the Pebble partnership. The evaluation methodology, while transparent, relies on benchmarks that may not capture real-world variability. And the model's specialization means it won't handle chat or open-ended tasks—a limitation that's by design but could limit adoption.

Still, for developers building smart home devices, wearables, or robots, Needle2 offers a compelling option: a small, fast, private, and offline-capable model that can be fine-tuned on a Mac or PC in minutes. The open-source release and the sandbox demo suggest a commitment to community adoption. Whether it can carve out a sustainable niche remains to be seen, but the bet on sub-$200 devices is a smart one in a world where edge AI is often synonymous with premium hardware.