hk
From-scratch Zig runtime and format for local LLM inference with mmap loading and CPU/CUDA/Vulkan engines.
Tested on Linux
Use it
Linux x86_64
You need first
- Runtimes:
Zig 0.17 - A GPU is optional
Isolated install of README pip package via uv tool install hknt
uv tool install hkntDownload native hk-x86_64-linux release binary next to Python package and make executable
curl -fL -o "$HOME"/.local/share/uv/tools/hknt/lib/python3.12/site-packages/hk/hk-x86_64-linux https://github.com/harshitkhandelwal208/hk/releases/download/v2.0.0/hk-x86_64-linux && chmod +x "$HOME"/.local/share/uv/tools/hknt/lib/python3.12/site-packages/hk/hk-x86_64-linux
Where the README and reality differ
- README: pip install hknt (bundles the native library and CLI)
What worked: Download hk-x86_64-linux from releases v2.0.0 into the installed hk package dir and chmod +x so the `hk` shim finds it
Verified in pickfit: the commands exactly as its sandbox ran them
For the record only: they name the sandbox's home, /home/sbx.
uv tool install hknt 2>&1curl -fL -o /home/sbx/.local/share/uv/tools/hknt/lib/python3.12/site-packages/hk/hk-x86_64-linux https://github.com/harshitkhandelwal208/hk/releases/download/v2.0.0/hk-x86_64-linux && chmod +x /home/sbx/.local/share/uv/tools/hknt/lib/python3.12/site-packages/hk/hk-x86_64-linuxhk --helphk listCheck that it works
hk --helpThe output should contain “hardware-profile”.
First thing to try
hk listRun hk list to see the models available on your machine.
What it does
These come from the README; pickfit didn’t check them one by one.
- Run models loaded with zero-copy memory mappingNot tested
- Serve models through an OpenAI-compatible serverNot tested
- Run inference on CPU, CUDA or VulkanNot tested
- Convert models while they downloadNot tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- Produces output
- Works offline
macOS and Windows: the project says it runs there; pickfit hasn’t tried it.
Good for
- run a prompt on a local model via CLI
- chat interactively with a local model
- serve a local model via OpenAI-compatible API
- convert a GGUF file to .hk format
- widen a model or grow vocabulary without retraining from zero
An alternative to llama.cpp and Hugging Face Transformers.
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
It produces model output
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026
- Version 2.0.0
Differences the test machine hit
- README: --version or --help works
In the test: Use `hk --help`; native binary has no `--version` flag (exits 1 with BadUsage)