Madhuram-v0.6: Technical Overview

Madhuram-v0.6 pairs a 16,384-token context window with a 600 billion token training budget at 151 million parameters. Evaluated against models trained on two to nineteen trillion tokens, it leads its size class on knowledge benchmarks.

Core Specifications

Madhuram-v0.6 is a 151 million parameter model built on the Perseus architecture and trained on 600 billion tokens of English text. It runs a 16,384-token context window, four times that of v0.5, and is designed to hold that window on consumer hardware rather than only in a datacentre.

Working With Long Context

A 16K window is roughly 12,000 words of input: a full research paper, a long support thread, several source files, or an afternoon of conversation. The model accepts all of it in a single pass, on the same weights that run on a phone.

Retrieval quality across that window is measured directly rather than asserted. The results are in the long-context section below, reported at every context length from 2K to 16K and set against three comparable models.

Commonsense Reasoning

All four models were run through the same harness with identical settings rather than quoting self-reported figures. Results are zero-shot on base checkpoints. Across the commonsense suite the four models sit within 1.07 points of each other.

Zero-shot, evaluated using lm-evaluation-harness. Best scores in each category highlighted in green.

Knowledge Benchmarks

The knowledge results are where the training-efficiency gap is clearest. Madhuram leads GPQA Diamond and SciQ outright, and beats both SmolLM2 and Gemma 3 on MMLU by five to seven points, despite a three to tenfold data disadvantage. LFM2.5 takes MMLU, CommonSenseQA and TriviaQA, broad factual recall being precisely where nineteen trillion tokens pay off.

Zero-shot on base checkpoints. Best scores in each category highlighted in green.

Instruction-Tuned Performance

After supervised finetuning, Madhuram leads its size class on graduate-level science reasoning and multi-step reasoning, and roughly a third ahead of both size-class peers on instruction following. LFM2.5 holds substantial advantages on instruction following and mathematics, the return on nineteen trillion pretraining tokens combined with multi-stage reinforcement learning, and precisely the ceiling that additional data and compute raise.

BBH 3-shot, GSM8K 5-shot reported as the mean of the flexible-extract and strict-match parsers, GPQA and IFEval zero-shot. Best scores in each category highlighted in green.

Long-Context Retrieval

The 16K window is validated with RULER needle-in-a-haystack evaluations, averaged across eight retrieval task variants at context lengths from 2K to 16K. Madhuram outperforms both size-class peers at every context length, and by a widening margin as context grows. SmolLM2-135M-Instruct cannot run 16K context at all. LFM2.5 leads throughout, holding above 90% recall across the full range.

Average recall (%) across the eight RULER NIAH tasks , single 1 to 3, multikey 1 to 3, multiquery and multivalue, on post-trained checkpoints. SmolLM2 maximum context is 8K. Best scores in each category highlighted in green.

On-Device Inference

Converted to GGUF and run through llama.cpp on a Nothing Phone (3a), Madhuram delivers the fastest decode of the four models tested while occupying the least storage. LFM2.5 leads on prefill; Madhuram leads on token generation, which is what dominates perceived latency in interactive use.

Nothing Phone (3a), Snapdragon 7s Gen 3. 4-bit quantisation, 4 threads, medians of 6 runs. Prefill measured on a 2048-token prompt. Same device, build and quantisation for all four models.

Deployment

The model serves as a versatile baseline for distributed intelligence, targeting environments where low latency and privacy are paramount:

• Hosted: Available through the chat interface and an OpenAI-compatible API for free.
• On-Device: Converts to GGUF for llama.cpp, validated on consumer smartphone hardware.
• Embedded Systems: Functional on resource-constrained IoT hardware.
• Private Cloud: Deployable as a high-speed agent within secure enterprise firewalls.

Availability

• Chat: chat.maruthlabs.com
• API: API documentation
• Edge: Edge deployment guide

Next Steps

Work in progress includes multilingual expansion, better long-context, and scaling-law studies to inform a larger model. For inquiries regarding specific evaluation results or research collaborations, please contact the lab.