Flyweight
Serves GGUF/safetensors LLMs from a native C++/CUDA runtime via OpenAI/Anthropic APIs.
Tested on Linux
Use it
Linux x86_64
You need first
- Runtimes:
Python 3.11+ - A GPU is optional
Install flyweight via uv tool
uv tool install flyweight-llm
Check that it works
flyweight --version 2>&1The output should contain “flyweight 0.25.0”.
First thing to try
flyweight doctor 2>&1Run flyweight doctor first to check your setup.
What it does
- Serve language models through compatible APIsNot tested
- Chat with a model in the built-in web UINot tested
- Send images to a model as inputNot tested
- Generate images from text promptsNot tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- 0 of 4 features work
- Works offline
Windows: the project says it runs there; pickfit hasn’t tried it.
Good for
- serve a GGUF model with chat UI and HTTP API
- chat with a local model in the bundled chat UI
- generate one response from a model and exit
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
Its main function works
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026
- Version 0.25.0
Differences the test machine hit
- README: flyweight serve model.gguf
In the test: serve with --backend cpu and a model from a supported family (tested LiquidAI LFM2-350M Q4_0, 209 MB); the suggested stories260K test model is refused at load - README: CPU backend serves without a GPU
In the test: generation of 16 tokens from LFM2-350M on the CPU backend did not finish within 300 s; /v1/models, /health and /props answer instantly