B.E.A.M.
Light and modular inference engine, built on MLX.
Robust prefix cache Expert streaming No bloat
How to install
uv tool install mlx-beam
beam doctorbeam doctor reports the Python, MLX, device and memory it sees.
What sets it apart
- Robust prefix cache
- RAM and SSD tiers, checkpoints for hybrid models. Survives model swaps and restarts.
- Expert streaming
- Mixture-of-experts models larger than memory. Residency configurable, from minimal RAM to fully resident.
- No bloat
- The core is the token path. Vision, audio, conversion, structured output and tool-call repair are optional extras.
- Batched MTP
- Multi-token prediction stays on with many requests at once.
- Batched vision
- Images go through the same scheduler; no request waits behind a picture.
- No stalls
- A short request beside a long prefill answers in seconds.
- Mixed-precision KV cache
- Bits per layer, set at conversion.
- Thinking budget
- A hard cap on the reasoning trace, per request.
- Responses API
- Next to chat completions, stateless.
Why it exists
Existing MLX servers either stop at the basics or grow things that have no place in an inference engine: a built-in game, a cloud path that arrives with an update. The ones we ran daily also had bugs where it matters most: prefix cache, batching under load, vision. B.E.A.M. keeps the core to the token path and fixes those paths at the source. Everything else is an extra you choose to install; nothing ever ships in the core that you did not ask for.
Where it stands
| CLI, packaging, CI | skeleton |
| Vendored mlx-lm base | planned |
| Prefix cache with recurrent-state checkpoints | planned |
| Multi-token prediction in the batch | planned |
| Expert streaming from SSD | planned |
Measured numbers are published as they are measured, with machine, model and date.