quantpicker
Measures GGUF quants on your GPU for speed and quality loss to pick the fastest acceptable one.
Tested on macOS and Linux
Use it
macOS arm64
You need first
- A GPU is optional
- Needs the network
- Keys or accounts: HF_TOKEN
install quantpicker via uv tool install
uv tool install quantpicker
Check that it works
quantpicker search qwen3 8b 2>&1The output should contain “Qwen/Qwen3-8B-GGUF”.
First thing to try
quantpicker search qwen3 8b 2>&1Run quantpicker search <model name> to find GGUF repos on Hugging Face.
Linux x86_64
You need first
- A GPU is optional
- Needs the network
- Keys or accounts: HF_TOKEN
install quantpicker via uv tool
uv tool install quantpicker
Check that it works
quantpicker --help 2>&1The output should contain “Measures which GGUF quant”.
First thing to try
quantpicker search qwen3 8b 2>&1Run quantpicker search <model name> to find GGUF repos on Hugging Face.
What it does
- Measure GGUF quants and pick the fastest oneTested
How to use it
- Run
quantpicker <repo> --quants Q4_K_M,Q5_K_M --chunks 5 --reps 1with a Hugging Face GGUF repo. - Compare the tok/s speed and mean KLD shown for each quant.
- Run
- Search Hugging Face for GGUF reposTested
How to use it
- Run
quantpicker search <your search words>. - Read the table of repos with their downloads and likes.
- Run
- List quants and see which fit your GPUTested
How to use it
- Run
quantpicker list <repo>. - Check the size and fits columns, and which quants are measured by default.
- Run
How far pickfit got
macOS arm64
- Installs
- Starts
- 2 of 3 features work
- Needs the network
Linux x86_64
- Installs
- Starts
- 3 of 3 features work
- Needs the network
Other systems
Windows: the project says it runs there; pickfit hasn’t tried it.
Good for
- measure GGUF quants and pick fastest within quality limit
- find GGUF repos on Hugging Face
- list quants and check what fits GPU memory without downloading
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
macOS arm64
Its main function works
- Replayed from scratch
- Needed the network
- Checked on 1 Nov 2026
- Version 0.1.0
Linux x86_64
Its main function works
- Replayed from scratch
- GPU: Tesla T4
- Needed the network
- Checked on 1 Nov 2026
- Version 0.1.0
Differences the test machine hit
- README: pip install quantpicker
In the test: uv tool install quantpicker (isolated install avoids PEP 668 read-only system) - README: quantpicker unsloth/Qwen3-8B-GGUF
In the test: ran measure on bartowski/SmolLM2-135M-Instruct-GGUF with --quants Q4_K_M,Q5_K_M --chunks 5 --reps 1 to fit box time/disk; README example uses unsloth/Qwen3-8B-GGUF (~3-8 GiB per quant)