Madhuram-v0.6: Technical Overview

Madhuram-v0.6 is a 151 million parameter model built on the Perseus architecture and trained on 600 billion tokens of English text. It runs a 16,384-token context window, four times that of v0.5, and is designed to hold that window on consumer hardware rather than only in a datacentre.

A 16K window is roughly 12,000 words of input: a full research paper, a long support thread, several source files, or an afternoon of conversation. The model accepts all of it in a single pass, on the same weights that run on a phone.

Retrieval quality across that window is measured directly rather than asserted. The results are in the long-context section below, reported at every context length from 2K to 16K and set against three comparable models.

All four models were run through the same harness with identical settings rather than quoting self-reported figures. Results are zero-shot on base checkpoints. Across the commonsense suite the four models sit within 1.07 points of each other.

Zero-shot, evaluated using lm-evaluation-harness. Best scores in each category are set in bold.

The knowledge results are where the training-efficiency gap is clearest. Madhuram leads GPQA Diamond and SciQ outright, and beats both SmolLM2 and Gemma 3 on MMLU by five to seven points, despite a three to tenfold data disadvantage. LFM2.5 takes MMLU, CommonSenseQA and TriviaQA, broad factual recall being precisely where nineteen trillion tokens pay off.

Zero-shot on base checkpoints. Best scores in each category are set in bold.

After supervised finetuning, Madhuram leads its size class on graduate-level science reasoning and multi-step reasoning, and roughly a third ahead of both size-class peers on instruction following. LFM2.5 holds substantial advantages on instruction following and mathematics, the return on nineteen trillion pretraining tokens combined with multi-stage reinforcement learning, and precisely the ceiling that additional data and compute raise.

BBH 3-shot, GSM8K 5-shot reported as the mean of the flexible-extract and strict-match parsers, GPQA and IFEval zero-shot. Best scores in each category are set in bold.

The 16K window is validated with RULER needle-in-a-haystack evaluations, averaged across eight retrieval task variants at context lengths from 2K to 16K. Madhuram outperforms both size-class peers at every context length, and by a widening margin as context grows. SmolLM2-135M-Instruct cannot run 16K context at all. LFM2.5 leads throughout, holding above 90% recall across the full range.

• Madhuram-v0.6
• SmolLM2-135M-Instruct
• Gemma-3-270M-IT
• LFM2.5-230M

Post-trained checkpoints. The vertical axis starts at 40. SmolLM2 has no 16K point because its maximum context is 8K.

Average recall (%) across the eight RULER NIAH tasks , single 1 to 3, multikey 1 to 3, multiquery and multivalue, on post-trained checkpoints. SmolLM2 maximum context is 8K. Best scores in each category are set in bold.

Converted to GGUF and run through llama.cpp on a Nothing Phone (3a), Madhuram delivers the fastest decode of the four models tested while occupying the least storage. LFM2.5 leads on prefill; Madhuram leads on token generation, which is what dominates perceived latency in interactive use.

Nothing Phone (3a), Snapdragon 7s Gen 3. 4-bit quantisation, 4 threads, medians of 6 runs. Prefill measured on a 2048-token prompt. Same device, build and quantisation for all four models.

The model serves as a versatile baseline for distributed intelligence, targeting environments where low latency and privacy are paramount:

• On-Device: Converts to GGUF for llama.cpp, validated on consumer smartphone hardware.
• Embedded Systems: Functional on resource-constrained IoT hardware.
• Private Cloud: Deployable as a high-speed agent within secure enterprise firewalls.

Work in progress includes multilingual expansion, better long-context, and scaling-law studies to inform a larger model. For inquiries regarding specific evaluation results or research collaborations, please contact the lab.