Field Reports
Engineering writeups of production AI systems we build: the method, the measurements, and the cost.
An RL/Eval Proof on Prime Intellect
We built a deterministic decision-grading loop around BearingBrain's production query stream, benchmarked four models against it, then ran hosted RL training on Prime Intellect and re-ran the same gate: 0.954 → 0.962 overall.
Findings
- A deterministic grader is the precondition for RL training — a reward function that is itself an LLM is a moving target.
- The labels were the weak link, not the model: 6 of 11 low-scoring traces were production-label errors the model was right about.
- Any evaluation you can grade deterministically, you can RL-train on Prime.
Agent-Generated GPU Kernels on a Real Hot Path
We took BearingBrain's ISO 281 bearing-life calculation — the compute behind every selection query — and fed it to Makora's agentic kernel generator. The winning fused CUDA kernel runs the 100,000-candidate sweep in 0.013 ms on an H100 (21.6× faster than torch.compile) and was independently verified on an RTX A3000: 34.8× faster at 100k candidates, 67× at 1M.
Findings
- The branchy, table-driven load-factor math is the hard part — fusion must handle per-bearing-type lookup, not just arithmetic.
- Kernel generation is a search, not a guarantee: 40 of 84 candidates failed validation before the winner emerged.
- An owned workstation GPU is a free, independent verification lane — no rental required.
The Small Model Question
Do you need a hosted model, or is one on your own hardware good enough? We ran DeepSeek-V4-Flash, Qwen3-8B, and Llama-3.2-1B through the same 449-query gate with real latency measurements: the local 8B scored within 3 points of hosted at the same speed, for $0.
Findings
- An 8B local model is a credible production option for structured extraction — within 3 points of the best hosted model at the same latency.
- Size selection is workload-specific: the same 1B model that collapses on structured parsing (0.188 on decision events) is fine at chat (0.888).
- Local inference means production data never leaves the building — a hard requirement for many manufacturers.
What you get from us
These reports are the standard we hold client work to: a defined before/after, reproducible measurements, and an honest account of constraints. If a process in your business has numbers attached — a runtime, a hit rate, a cost per query — we can measure it, improve it, and prove the improvement.