Lumen
Rust binary that runs Qwen LLMs GPU-resident locally and serves OpenAI-compatible API.
Tested on LinuxKnown vulnerabilities
Use it
Before you install: known vulnerabilities
pickfit found 1 known high or critical vulnerability in the packages a fresh install brings (OSV.dev, 3 Nov 2026). Weigh this before you install it.
crates:rustls-webpki 0.103.10rustls-webpki: Denial of service via panic on malformed CRL BIT STRING (GHSA-82j2-j2ch-gfr8, high, fixed in 0.103.13)
Linux x86_64
You need first
- A GPU is optional
- Needs the network
download install script
curl -fsSL -o /tmp/install.sh https://servelumen.com/install.shinstall lumen binaries via script
bash /tmp/install.sh
Check that it works
lumen --versionThe output should contain “lumen v0.51.0”.
First thing to try
lumen pull qwen3.5-9b:q4_0 --yeslumen-server qwen3.5-9b:q4_0 --port 8000curl -fsSL http://127.0.0.1:8000/v1/modelsWhat it does
Good for
- generate text from a prompt with a local Qwen model
- serve OpenAI-compatible chat completions with streaming
- generate an image from a text prompt
An alternative to llama.cpp.
How far pickfit got
Linux x86_64
- Installs
- Starts
- Produces output
- Works offline
macOS: the project says it runs there; pickfit hasn’t tried it.
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
It produces model output
- Replayed from scratch
- GPU: Tesla T4
- Works offline
- Checked on 1 Nov 2026
- Version v0.51.0
Differences the test machine hit
- README: curl -fsSL https://servelumen.com/install.sh | bash
In the test: download with curl -fsSL -o /tmp/install.sh, read /tmp/install.sh fully, then run bash /tmp/install.sh - README: no CUDA SDK needed
In the test: installer auto-fetched NVIDIA NVRTC + cuBLAS 12.9 (670 MB) into ~/.local/lib/lumen because the box has a driver but no CUDA toolkit - README: lumen run qwen3.5-9b:q8_0
In the test: used qwen3.5-9b:q4_0 (~5.5 GB LBC) instead of README q8_0; Q4_0 is the smallest quant for the 9B and is verified on T4 cc 7.5