Lattice
Pure Rust inference engine for running transformer models locally on Apple Silicon and Linux.
Tested on Linux
Use it
Linux x86_64
You need first
- Runtimes:
Rust 1.93+ - A GPU is optional
- At least 8 GB of RAM
Download prebuilt lattice binary release
mkdir -p ~/.local/bin /tmp/lattice-dl && cd /tmp/lattice-dl && curl -fLO "https://github.com/ohdearquant/lattice/releases/download/v0.9.0/lattice-0.9.0-x86_64-unknown-linux-gnu.tar.gz" && curl -fLO "https://github.com/ohdearquant/lattice/releases/download/v0.9.0/lattice-0.9.0-x86_64-unknown-linux-gnu.tar.gz.sha256"Verify checksum and list tarball
cd /tmp/lattice-dl && sha256sum -c lattice-0.9.0-x86_64-unknown-linux-gnu.tar.gz.sha256Clean reinstall ending in help check
cd /tmp/lattice-dl && tar -xzf lattice-0.9.0-x86_64-unknown-linux-gnu.tar.gz && install -m 0755 lattice-0.9.0-x86_64-unknown-linux-gnu/lattice ~/.local/bin/lattice && lattice --help
Check that it works
lattice --helpThe output should contain “Usage: lattice <COMMAND>”.
Run lattice --help to see the available commands.
What it does
- Chat with a local modelNot tested
- Serve an OpenAI-compatible chat APINot tested
- Generate embeddings with 9 supported modelsNot tested
- Quantize models to Q4 or Q8Not tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- 0 of 4 features work
- Works offline
macOS: the project says it runs there; pickfit hasn’t tried it.
Good for
- run interactive chat with a local model
- serve OpenAI-compatible chat completions locally
- quantize models to Q4 or QuaRot
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
Installs and starts
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026
Differences the test machine hit
- README: cargo install lattice-serve --bin lattice --features f16 (requires Rust 1.93+)
In the test: used prebuilt release tarball lattice-0.9.0-x86_64-unknown-linux-gnu.tar.gz (verified sha256, installed to ~/.local/bin) - README: GitHub releases ship lattice-<version>-<target>.tar.gz
In the test: only v0.9.0 has prebuilt Linux x86_64 assets; releases up to v0.14.0 ship none, so the binary lags current crate versions - README: README documents chat/serve usage without stating version-flag support
In the test: top-level `lattice --version` errors (exit 2); `lattice --help` works and lists chat/serve/doctor/prune-score