pickfit

BrowseLocal AI tools

jitllm

JIT inference engine that adapts to hardware to run LLMs faster, even when they exceed memory.

Tested on macOS and Linux

Use it

macOS arm64

You need first

  • Runtimes: Go 1.26 or newer
  1. brew tap and install jitllm via Homebrew

    brew tap jitllm/jitllm 2>&1
    brew trust jitllm/jitllm 2>&1
    brew install jitllm 2>&1
    
Where the README and reality differ
  • README: jitllm convert qwen3-30b-a3b (and flags after file)
    What worked: Go flag parser requires flags BEFORE file: jitllm convert -o DIR <file> and jitllm run -n 16 <file> "prompt"; placing -o or -n after file errors

Check that it works

which jitllm && which jitllmd && jitllm --help 2>&1 | head -n 100; echo ---VERSION---; jitllm --version 2>&1; jitllm version 2>&1; jitllmd --help 2>&1

The output should contain “jitllm convert”.

First thing to try

mkdir -p ~/models && curl -fL -o ~/models/stories260K.gguf https://huggingface.co/ggml-org/test-model-stories260K/resolve/main/stories260K-f32.gguf && ls -lh ~/models/stories260K.gguf
jitllm convert -o ~/models/stories260K.jlm ~/models/stories260K.gguf 2>&1

Start by converting a model to the local format with jitllm convert -o <output.jlm> <model.gguf>.

Linux x86_64

You need first

  • Runtimes: Go 1.26 or newer
  1. Download install script for inspection

    curl -fsSL https://jitllm.org/install.sh -o /tmp/jitllm-install.sh
  2. Install jitllm and jitllmd from release script

    sh /tmp/jitllm-install.sh
Where the README and reality differ
  • README: curl -fsSL https://jitllm.org/install.sh | sh
    What worked: Download to /tmp/jitllm-install.sh, read it, then run sh /tmp/jitllm-install.sh; installer verifies checksums.txt and installs to ~/.local/bin
  • README: jitllm --version or --help works
    What worked: jitllm version subcommand prints 0.2.6; jitllm --version and jitllm --help exit 2 with usage on stderr
  • README: On Linux the install script starts jitllmd as a systemd user service
    What worked: Installer prints 'no systemd user session here: start the server with jitllmd serve' and skips service install where no systemd user session exists

Check that it works

jitllm version

The output should contain “0.2.6”.

First thing to try

jitllm version

Start by converting a model to the local format with jitllm convert -o <output.jlm> <model.gguf>.

What it does

These come from the README; pickfit didn’t check them one by one.

  • Generate optimized kernels for your hardware
    Not tested
  • Run models larger than memory using paging
    Not tested
  • Serve chat and completions through compatible APIs
    Not tested

How far pickfit got

macOS arm64

  1. Installs
  2. Starts
  3. 0 of 3 features work
  4. Works offline

Linux x86_64

  1. Installs
  2. Starts
  3. Answers requests
  4. Works offline

Windows: the project says it runs there; pickfit hasn’t tried it.

Good for

  • convert catalog models to local format
  • chat with a local model
  • serve chat and embeddings via API
  • call tools with constrained decoding

An alternative to llama.cpp, vLLM and mistral.rs.

How pickfit tested it

Each run starts from a clean machine and follows the steps above. How pickfit verifies

macOS arm64

Its main function works

  • Replayed from scratch
  • Works offline
  • Checked on 1 Nov 2026
  • Version 0.2.6

Differences the test machine hit

  • README: jitllm convert -o output writes file directly
    In the test: -o is a directory; convert creates DIR/basename.jlm inside it, nested when -o already looks like *.jlm; server only lists *.jlm files directly under -models, so copy to ~/models/tiny.jlm for discovery
  • README: curl http://localhost:4242/v1/chat/completions with model qwen3-30b-a3b
    In the test: base model without chat template is rejected for chat; use /v1/completions or a chat-template model for chat completions

Linux x86_64

Its service answers requests

  • Replayed from scratch
  • Works offline
  • Checked on 28 Oct 2026
  • Version 0.2.6

How to read a tool card

Stitching
pickfit installed this tool itself. Everything under the stitches comes from that run. A tool pickfit has not run never gets stitches.
Verified on your OS
The install steps ran on the system you chose. “Ran on Linux only” means it has not been run on yours yet.
macOS, Linux and WindowsWhere it ran
Green OS marks: pickfit ran the install steps there. Grey ones: the README says it runs there, and pickfit has not tried.
How far the run got
Installs and starts, its main function works, its service answers, it produces model output. A window means a desktop app opened; a plug, an MCP client used it. The mark sits beside the run’s result, under “Why it fits and how we know” and in “How pickfit tested it”.
Visual check
pickfit looked at screenshots of the app. Green: it looks right. Amber with a number: things that look wrong; open it to see them. Grey and dashed: not checked yet.
Known vulnerabilities
Known high or critical vulnerabilities (OSV.dev) in what it installs. Green: none known. Red with a number: how many; open it for the list, and read it before you install. Grey and dashed: not checked, which does not mean safe.
About the run
The day it ran, and the version that ran.
Source of a reason
Every reason a pick fits rests on one profile field. Open the reason to see the field and the value it relies on.
Hidden gem
High quality, little known.
Quality and popularity
Two scores out of 100. Quality: how well it installed and worked when pickfit tried it, and how good its install docs and upkeep are. Popularity: from stars, downloads and Hacker News.
Features checked
Of the main features the README advertises, how many the run actually exercised (green), and how many failed (red). Grey ones were not checked, and the card says why.
Not verified
A well-known tool Fitz mentions from general knowledge. No run record, no steps, no score.