goinfer
Pure-Go library and binaries for running open-weight LLMs locally from safetensors or GGUF.
Tested on macOS and Linux
Use it
macOS arm64
You need first
- Runtimes:
Go>=1.27
download goinfer-serve darwin-arm64 binary and make executable
curl -fsSL -o goinfer-serve https://github.com/townsendmerino/goinfer/releases/latest/download/goinfer-serve-darwin-arm64 && chmod +x goinfer-serve && ls -lh goinfer-serve && ./goinfer-serve --version
Check that it works
curl -fsSL -o goinfer-serve https://github.com/townsendmerino/goinfer/releases/latest/download/goinfer-serve-darwin-arm64 && chmod +x goinfer-serve && ls -lh goinfer-serve && ./goinfer-serve --versionThe output should contain “goinfer-serve v0.23.0”.
Start a local server with goinfer-serve -model <model.gguf>, or run goinfer-chat models to see available models.
Linux x86_64
You need first
- Runtimes:
Go>=1.27
Download linux-amd64 serve and chat binaries to ~/.local/bin
mkdir -p ~/.local/bin ~/models && curl -fL -o ~/.local/bin/goinfer-serve https://github.com/townsendmerino/goinfer/releases/latest/download/goinfer-serve-linux-amd64 && curl -fL -o ~/.local/bin/goinfer-chat https://github.com/townsendmerino/goinfer/releases/latest/download/goinfer-chat-linux-amd64 && chmod +x ~/.local/bin/goinfer-serve ~/.local/bin/goinfer-chat
Where the README and reality differ
- README: curl ... goinfer-serve-darwin-arm64 (macOS only)
What worked: curl -fL -o ~/.local/bin/goinfer-serve .../download/goinfer-serve-linux-amd64 (and goinfer-chat-linux-amd64), chmod +x
Check that it works
goinfer-serve --versionThe output should contain “backends: cpu cuda”.
First thing to try
goinfer-chat modelsStart a local server with goinfer-serve -model <model.gguf>, or run goinfer-chat models to see available models.
What it does
- Serve models through OpenAI and Anthropic APIsTested
How to use it
- Start the server with
goinfer-serve -model <model.gguf>. - List loaded models with
curl -fsSL http://127.0.0.1:8080/v1/models. - Send chat requests to
http://127.0.0.1:8080/v1/chat/completions(OpenAI style) orhttp://127.0.0.1:8080/v1/messages(Anthropic style).
- Start the server with
- Limit output to a Go struct or JSON SchemaNot tested
- Download and run GGUF models from Hugging FaceNot shown working
How far pickfit got
macOS arm64
- Installs
- Starts
- 0 of 3 features work
- Needs the network
Linux x86_64
- Installs
- Starts
- 1 of 3 features work
- Works offline
Other systems
Windows: the project says it runs there; pickfit hasn’t tried it.
Good for
- constrain generation to a Go struct or JSON Schema
- serve a local model to a coding agent via OpenAI API
- extract a scanned invoice image into a Go struct
An alternative to Ollama, vLLM and llama.cpp.
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
macOS arm64
Installs and starts
- Replayed from scratch
- Needed the network
- Checked on 1 Nov 2026
- Version v0.23.0
Differences the test machine hit
- README: curl -fsSL -o goinfer-serve https://github.com/townsendmerino/goinfer/releases/latest/download/goinfer-serve-darwin-arm64
In the test: also chmod +x goinfer-serve before ./goinfer-serve --version - README: flag examples use goinfer-serve -model and -web (single dash)
In the test: both -model/--model work but listen addr is -addr / --addr default 127.0.0.1:8080 not --listen - README: goinfer-chat models lists checkpoints
In the test: goinfer-serve binary does not have 'models' subcommand; only pull, check, connect and --version - README: install step 1 runs goinfer-serve
In the test: nothing to change; a note: the box ran goinfer-serve, downloaded, without reading it first, so pickfit did not inspect it
Linux x86_64
It produces model output
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026
- Version v0.23.0
Differences the test machine hit
- README: go get / go install with Go>=1.27
In the test: skipped go get/go install candidates (box has no Go and installing a toolchain is forbidden); used release binaries instead