superfluid
Serving daemon providing one scheduler and APIs for local LLM inference across runtimes and machines.
Tested on Linux
Use it
Linux x86_64
install superfluid via npm global
npm install --global @basecompute/superfluid
Check that it works
superfluid --versionThe output should contain “superfluid 0.1.6”.
First thing to try
superfluid runtimes 2>&1
mkdir -p ~/models && curl -fL -o ~/models/stories260K.gguf https://huggingface.co/ggml-org/test-model-stories260K/resolve/main/stories260K-f32.gguf && ls -lh ~/models/superfluid runtime install llamacpp 2>&1superfluid serve ~/models/stories260K.gguf --port 8453 2>&1curl -s http://127.0.0.1:8453/v1/modelsRun superfluid runtime install llamacpp to add a runtime, then superfluid serve <model.gguf> --port 8453 to start serving a model.
What it does
- Serve local LLM inference across runtimesTested
How to use it
- Install a runtime with
superfluid runtime install llamacpp. - Start serving a model with
superfluid serve <model.gguf> --port 8453. - Send requests to the API, for example
curl -s http://127.0.0.1:8453/v1/completions -H 'content-type: application/json' -d '{"model":"<model>","prompt":"<your text>","max_tokens":16}'.
- Install a runtime with
- Serve OpenAI, Anthropic and Ollama clientsNot shown working
- Run coding agents on local modelsNot tested
- Add machines to share the serving workNot tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- 1 of 4 features work
- Works offline
macOS: the project says it runs there; pickfit hasn’t tried it.
Good for
- serve local LLM inference via OpenAI, Anthropic or Ollama API
- run a coding agent on local models
- distribute serving across machines on local network
An alternative to llama-server, Ollama and mlx_lm.server.
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
It produces model output
- Replayed from scratch
- Works offline
- Checked on 1 Nov 2026
- Version 0.1.6
Differences the test machine hit
- README: runtime installed on first use from its own project's releases
In the test: explicit `superfluid runtime install llamacpp` was run before serve to ensure CPU build present; serve log shows ready runtime - README: model has chat template for OpenAI/Anthropic/Ollama examples
In the test: stories260K GGUF has no chat template - chat routes return 'codec has no chat dialect'; used /v1/completions for inference which works, and verified Ollama/Anthropic endpoints exist but require chat-template model