Frink
Pure-Rust engine that loads GGUF checkpoints and runs quantized dense and MoE inference.
Tested on LinuxKnown vulnerabilities
Use it
Before you install: known vulnerabilities
pickfit found 1 known high or critical vulnerability in the packages a fresh install brings (OSV.dev, 3 Nov 2026). Weigh this before you install it.
npm:zod 4.4.3Zod can exhaust memory while validating oversized constrained collections (GHSA-mrhx-c7xc-m3w5, high, fixed in 4.6.6)
Linux x86_64
Download install script for reading
curl -fsSL https://raw.githubusercontent.com/antonellof/frink/main/scripts/install.sh -o /tmp/frink-install.shInstall frink via release script
bash /tmp/frink-install.sh
Check that it works
frink --versionThe output should contain “0.50.0”.
First thing to try
mkdir -p ~/models && curl -fL -o ~/models/stories260K.gguf https://huggingface.co/ggml-org/test-model-stories260K/resolve/main/stories260K-f32.gguf && ls -l ~/models/stories260K.ggufFRINK_EXPERT_CACHE_BYTES=536870912 frink serve -m ~/models/stories260K.ggufcurl -s http://127.0.0.1:8383/v1/modelsStart the server with a GGUF model file using frink serve -m <model.gguf>, then send requests to http://127.0.0.1:8383.
What it does
- Serve an OpenAI-compatible APITested
How to use it
- Start the server with
frink serve -m <model.gguf>. - List the loaded model with
curl -s http://127.0.0.1:8383/v1/models. - Request a chat completion with `curl -s -X POST http://127.0.0.1:8383/v1/chat/completions -H 'content-type: application/json' -d '{"model":"llama","messages":[{"role":"user","content":"<your text>"}],
- Start the server with
- Serve embeddings and rerankingTested
How to use it
- Start the server with
frink serve -m <model.gguf>. - Get an embedding with
curl -s -X POST http://127.0.0.1:8383/v1/embeddings -H 'content-type: application/json' -d '{"model":"llama","input":"<your text>"}'.
- Start the server with
- Run inference on GGUF checkpointsNot tested
- Run Mixture-of-Experts modelsNot tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- 2 of 4 features work
- Works offline
macOS: the project says it runs there; pickfit hasn’t tried it.
Good for
- serve a GGUF model via OpenAI-compatible API
- run a GGUF checkpoint for text completion
- benchmark inference against llama.cpp on same host
An alternative to llama.cpp.
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
It produces model output
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026
- Version 0.50.0
Differences the test machine hit
- README: curl -fsSL .../install.sh | bash
In the test: download to /tmp/frink-install.sh, read it, then run bash /tmp/frink-install.sh - README: frink -m <model> -p ... -n ... runs directly
In the test: run with FRINK_EXPERT_CACHE_BYTES=536870912 on this 2GiB box - README: frink -m ... -dev metal -ngl all
In the test: omit -dev metal -ngl all on Linux CPU; run without device flags