Narwhal
Disaggregated LLM inference framework that hot-swaps prefill and decode roles without reloading weights.
Tested on Linux
Use it
Linux x86_64
You need first
- Runtimes:
python>=3.11 - A GPU is required
Install narwhal-inference with uv tool install
uv tool install narwhal-inference
Where the README and reality differ
- README: narwhal dev init then narwhal dev status
What worked: Even after creating ~/.cache/huggingface and ~/.cache/huggingface/hub, `narwhal dev init` still exits 2 with FileNotFound for `~/.cache/huggingface/hub/models--Qwen--Qwen3.5-0.8B`; `narwhal dev status` without a prior successful init exits 2 with missing runs/dev/instance.json.
Check that it works
narwhal-serve --versionThe output should contain “narwhal-inference 0.8.0”.
What it does
These come from the README; pickfit didn’t check them one by one.
- Switch prefill and decode roles without reloadingNot tested
- Serve completion and chat requestsNot tested
- Profile engines and run benchmarksNot tested
- Export metrics to Prometheus and GrafanaNot tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- Not used yet
- Works offline
Good for
- serve completion and chat requests streamed or buffered
- reassign prefill and decode roles across GPU fleet
- profile engines and run ordered benchmark points
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
Installs and starts
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026
- Version 0.8.0