Codex spent 500,000 tokens reading my logs. Then it forgot what it read.

Frontier models are extraordinary reasoners. The catch is that most of what we ask them to do isn’t reasoning — it’s reading. Opening the repository, scanning logs, checking documentation, reading four files to find the three that matter. A reasoning model deliberates before every step, so a single investigation can consume hundreds of thousands of tokens — half a million on a heavy run — and then, at the context limit, compress the whole session into a summary and quietly drop the exact evidence it just gathered. You pay premium rates for reading, and the reading doesn’t even stick.

The fix is architectural: stop paying a reasoning model to read. Separate the two jobs. I put a purpose-built local model on an NVIDIA DGX Spark to do the reading and retrieval, and reserved the frontier model for the reasoning it’s exceptional at. The result is faster, dramatically cheaper, and — because the reading happens on hardware I control — private by design.

The constraint that actually matters

On a box like the Spark, the number that decides performance isn’t the advertised petaflop of compute — it’s memory bandwidth. With 128 GB of unified memory at 273 GB/s, and generation that moves a share of the model’s weights through memory for every token, the winning move is to move less weight per token. That points directly at the right class of model: one where only a fraction of the parameters are active per token, stored in a compact precision, served by an engine that reuses context instead of reprocessing it.

I evaluated the field and the fit was clear: NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 — 30B total parameters, roughly 3B active per token via mixture-of-experts routing, NVFP4 weights, a hybrid Mamba/Transformer architecture, paired with NVIDIA’s speculative-decoding draft checkpoint that advances several tokens per expensive step. For a single interactive agent, it turns the Spark into a fast, private reader.

vllm serve --model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 \
  --moe-backend marlin \
  --kv-cache-dtype fp8 \
  --enable-prefix-caching \
  --gpu-memory-utilization 0.85 \
  --mamba-backend flashinfer \
  --mamba-cache-mode align \
  --reasoning-parser nemotron_v3 \
  --tool-call-parser qwen3_coder \
  --enable-auto-tool-choice \
  --speculative_config.model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark \
  --speculative_config.num_speculative_tokens 3

Every flag earns its place. Prefix caching removes the prefill an agent would otherwise repay every turn. The reasoning and tool-call parsers make the model usable by a real agent harness, not just a chat window.

The result

Reading and research that took tens of minutes now completes in one to two. The entire reading phase leaves the paid meter. And the sensitive material — the logs, the code, the documents — never leaves the network.

The architecture

Two tiers, each doing what it’s best at. Local: a self-hosted model reads, searches, and returns a structured evidence package — a timeline, exact file and line references, a likely root cause. Frontier: a reasoning model receives that package and spends its full context on judgment, arriving at the hard problem already briefed instead of spending three-quarters of its window discovering it.

This is how agentic AI becomes economical and private at the same time: put the reading where it’s cheap and controlled, and the reasoning where it’s worth paying for. It’s the pattern I build for teams that need AI to run on hardware they own.