MaruthLabs
Madhuram on Device: Mobile Inference Benchmarks
Madhuram-v0.6 was measured on a mid-range Android phone against the three closest open models, using the same device, the same llama.cpp build and the same quantisation for all four. It generates tokens faster than any of them, from the smallest file on disk.
- Published
- July 2026
- Decode
- 119 tokens per second
- Prefill
- 452 tokens per second
- On disk
- 100 MB
- Device
- Nothing Phone (3a)
The Numbers
Decode speed is what a user actually feels: it sets how fast words appear once the model starts replying. Prefill is how quickly the prompt is read before generation begins, and matters most with long inputs. Size on disk decides whether the model can ship inside an app at all.
Nothing Phone (3a), Snapdragon 7s Gen 3, 4-bit quantisation, 4 threads.
| Model | Decode (tok/s) | Prefill (tok/s) | Size on disk |
|---|---|---|---|
| Madhuram-v0.6 (151M) | 119 | 452 | 100 MB |
| SmolLM2-135M | 117 | 230 | 104 MB |
| LFM2.5-230M | 86 | 645 | 151 MB |
| Gemma-3-270M | 54 | 293 | 247 MB |
Medians of six runs. Prefill measured on a 2048-token prompt. Same device, build and quantisation for all four models. The best figure in each column is set in bold.
What Stands Out
Madhuram leads decode at 119 tokens per second, ahead of a model with fewer parameters and more than twice the rate of Gemma 3 270M. It does so from a 100 MB file, the smallest of the four, which is what makes bundling it inside an app realistic.
LFM2.5 leads prefill by a wide margin. That is a genuine advantage on long inputs, and worth knowing if your workload is dominated by reading rather than writing. For interactive use, where the model produces more text than it consumes, decode is the number that governs how responsive it feels.
Why It Matters
- Nothing leaves the device. No network round trip, no server logs, no data in transit.
- It works offline. Aeroplane mode, weak coverage, or an air-gapped deployment all behave the same.
- No inference bill. Compute belongs to the device, so cost does not scale with usage.
Getting It On Your Hardware
On-device builds are delivered as a custom engagement. We handle the conversion, quantisation, and integration for your target hardware, and tune the trade-off between size and quality to suit the device.