Madhuram-v0.6 was measured on a mid-range Android phone against the three closest open models, using the same device, the same llama.cpp build and the same quantisation for all four. It generates tokens faster than any of them, from the smallest file on disk.
Decode tok/s
Prefill tok/s
On disk
Decode speed is what a user actually feels: it sets how fast words appear once the model starts replying. Prefill is how quickly the prompt is read before generation begins, and matters most with long inputs. Size on disk decides whether the model can ship inside an app at all.
On-device inference
Nothing Phone (3a) · Snapdragon 7s Gen 3 · 4-bit · 4 threads
Higher is better. Hover a bar for the exact figure.
| Model | Decode (tok/s) | Prefill (tok/s) | Size on disk |
|---|---|---|---|
| Madhuram-v0.6 (151M) | 119 | 452 | 100 MB |
| SmolLM2-135M | 117 | 230 | 104 MB |
| LFM2.5-230M | 86 | 645 | 151 MB |
| Gemma-3-270M | 54 | 293 | 247 MB |
Medians of six runs. Prefill measured on a 2048-token prompt. Same device, build and quantisation for all four models. Best figure in each column in green.
Madhuram leads decode at 119 tokens per second, ahead of a model with fewer parameters and more than twice the rate of Gemma 3 270M. It does so from a 100 MB file, the smallest of the four, which is what makes bundling it inside an app realistic.
LFM2.5 leads prefill by a wide margin. That is a genuine advantage on long inputs, and worth knowing if your workload is dominated by reading rather than writing. For interactive use, where the model produces more text than it consumes, decode is the number that governs how responsive it feels.
On-device builds are delivered as a custom engagement. We handle the conversion, quantisation, and integration for your target hardware, and tune the trade-off between size and quality to suit the device. If you want to evaluate the model first, the hosted API is free.