B.E.A.M.
Light and modular inference engine, built on MLX.
Robust prefix cache Expert streaming No bloat
How to install
uv tool install mlx-beam
beam serve --model p4ik/Qwen3.8-27B-MLX-OptiQ-5bit --draft-model bundledThe server answers on port 8000; --draft-model bundled uses the draft head the checkpoint ships. beam doctor reports the Python, MLX, device and memory it sees.
Features
Everything the engine does, with where each piece stands. Extras are packages or optional installs beside the core; the core runs without any of them.
Server and APIs
5 of 5 built- OpenAI APIbuiltChat completions, completions, Responses and models, with streaming.
- Anthropic APIbuiltMessages with thinking blocks and tool use, on the same token path.
- Health and metricsbuilt/health shows each capability with evidence; /metrics serves counters.
- Access controlbuiltAn API key is required off loopback; Host check and CORS allowlist.
- CLIbuiltbeam serve takes mlx-lm's flag names; beam doctor checks the machine.
Engine
4 of 7 built- Continuous batchingbuiltShort requests are answered beside a long prefill, capped by a valve.
- Prefix cachebuiltThe KV of a shared prefix is reused, for recurrent and window layers too.
- Prefix cache SSD tierplannedA second tier on SSD lets entries outlive eviction from RAM and restarts.
- Mixed-precision KV cachebuiltBits per layer come from the conversion; quantized after or during prefill.
- Multi-token predictionbuiltAn own draft head predicts ahead, verified exactly, one request at a time.
- Batched MTPplannedSeveral requests speculate at once.
- Expert streamingplannedModels larger than memory run with experts streamed from SSD.
Request control
3 of 3 built- Reasoning controlbuiltThinking on or off, a budget per request, effort levels, marker families.
- Tool callingbuiltParsers per model family and a repair ladder that reports every step.
- Sampling controlsbuiltTemperature, top-p/k, min-p, penalties, logit bias, seeds and defaults.
Extras
1 of 6 built- VisionbuiltImage input through the mlx-beam-vision package; families on the models page.
- AudioplannedAudio input for models that take it, as its own package.
- ConvertplannedQuantizes into the package layout: mixed-precision weights, KV profile, draft head and tower.
- StructuredplannedJSON schema and grammar-constrained decoding.
- GGUFplannedGGUF checkpoints load behind a guard.
- ImagesplannedImage generation and editing through the Images API.
Measured numbers are published as they are measured, with machine, model and date.
Why it exists
Existing MLX servers either stop at the basics or grow things that have no place in an inference engine, and the ones we ran daily had bugs where it matters most: prefix cache, batching under load, vision. B.E.A.M. keeps the core to the token path and fixes those paths at the source; everything else is a package or an extra you choose to install.