MFQ
MFQ quantizes large models and runs them locally with native CUDA and Metal runtimes.
Tested on LinuxKnown vulnerabilities
Use it
Before you install: known vulnerabilities
pickfit found 10 known high or critical vulnerabilities in the packages a fresh install brings (OSV.dev, 3 Nov 2026). Weigh this before you install it.
crates:zip 8.6.0zip: Critical Parser Differential & Security Control Bypass via Dual-Header Inconsistencies (Zip Cloaking) Leading to Arbitrary File Overwrite & RCE (GHSA-m665-v7pm-7grj, critical, fixed in 9.0.5)npm:dompurify 3.4.16DOMPurify SAFE_FOR_TEMPLATES bypass: split template expressions survive in attached shadow root content on DOM output modes (GHSA-cg2g-xh64-v4q7, critical, fixed in 3.4.17)
See 8 more
crates:zip 8.6.0— RUSTSEC-2026-0369 (critical), fixed in 9.0.0crates:quinn-proto 0.11.16— RUSTSEC-2026-0340 (high), fixed in 0.11.18crates:quinn-proto 0.11.16— RUSTSEC-2026-0341 (high), fixed in 0.11.17crates:zip 8.6.0— GHSA-9hj8-73fh-9963 (high), fixed in 9.0.0crates:zip 8.6.0— GHSA-hjqx-j4rw-g7xv (high), fixed in 9.0.0crates:zip 8.6.0— GHSA-q7v9-x93c-xqv3 (high), fixed in 9.0.0crates:zip 8.6.0— RUSTSEC-2026-0367 (high), fixed in 9.0.0npm:dompurify 3.4.16— GHSA-mrv2-3pqr-j5jr (high), fixed in 3.4.17
Linux x86_64
git clone MFQ
git clone https://github.com/Tylogi/MFQ.git MFQuv sync base deps
cd MFQ && uv syncuv sync --extra train to get torch for quantize
cd MFQ && uv sync --extra train
Check that it works
cd MFQ && uv run mfq --version 2>&1The output should contain “mfq 0.1.0”.
First thing to try
cd MFQ && uv run python -c "
import json, os, pathlib
import torch
from safetensors.torch import save_file
# create tiny hf model
out='/tmp/tiny_hf'
os.makedirs(out, exist_ok=True)
config={
'model_type':'qwen3',
'hidden_size':128,
'num_hidden_layers':2,
'num_attention_heads':4,
'num_key_value_heads':4,
'intermediate_size':256,
'vocab_size':32000,
'max_position_embeddings':512,
'hidden_act':'silu',
'rms_norm_eps':1e-05
}
open(os.path.join(out,'config.json'),'w').write(json.dumps(config))
# create dummy weights matching qwen3?
# Look at expected keys: model.embed_tokens, model.layers.0 etc
sd={}
def rand(shape): return torch.randn(shape, dtype=torch.float16)
sd['model.embed_tokens.weight']=rand((32000,128))
for i in range(2):
p=f'model.layers.{i}'
sd[f'{p}.self_attn.q_proj.weight']=rand((128,128))
sd[f'{p}.self_attn.k_proj.weight']=rand((128,128))
sd[f'{p}.self_attn.v_proj.weight']=rand((128,128))
sd[f'{p}.self_attn.o_proj.weight']=rand((128,128))
sd[f'{p}.mlp.gate_proj.weight']=rand((256,128))
sd[f'{p}.mlp.up_proj.weight']=rand((256,128))
sd[f'{p}.mlp.down_proj.weight']=rand((128,256))
sd[f'{p}.input_layernorm.weight']=rand((128,))
sd[f'{p}.post_attention_layernorm.weight']=rand((128,))
sd['model.norm.weight']=rand((128,))
sd['lm_head.weight']=rand((32000,128))
save_file(sd, os.path.join(out,'model.safetensors'))
print('done', os.listdir(out))
" 2>&1cd MFQ && uv run mfq quantize /tmp/tiny_hf /tmp/tiny.mfq --preset S4-M --backend cpu --dry-run 2>&1From the MFQ folder, run uv run mfq quantize <model_dir> <output.mfq> --preset S4-M --backend cpu --dry-run to preview a quantization.
What it does
- Convert Hugging Face checkpoints to MFQ formatTested
How to use it
- Preview the job with
uv run mfq quantize <model_dir> <output.mfq> --preset S4-M --backend cpu --dry-run. - Quantize with
uv run mfq quantize <model_dir> <output.mfq> --preset S4-M --backend cpu --overwrite. - Look for "status": "ok" and the output path in the result.
- Preview the job with
- Run large models on your own hardwareNot tested
- Download and run models in MFQ StudioNot tested
- Serve models through an OpenAI-compatible APINot tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- 1 of 4 features work
- Works offline
Good for
- run large models on local hardware
- download and run models in MFQ Studio
- serve models via OpenAI-compatible API
An alternative to Strata, llama.cpp and oMLX.
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
Its main function works
- Replayed from scratch
- Works offline
- Checked on 1 Nov 2026
- Version 0.1.0
Differences the test machine hit
- README: uv sync + uv run mfq build --backend auto works on any host
In the test: build requires CUDA toolkit nvcc/CMake or Apple Silicon Metal; on plain Linux x86 without accelerator it fails with 'no supported inference accelerator was detected' - README: uv run mfq serve starts server
In the test: serve also auto-detects backend and fails with same BuildError when no accelerator present, so F3 cannot be tested in this sandbox - README: quantize from HF safetensors
In the test: quantize --backend cpu works with torch+extra train deps, verified with tiny synthetic Qwen3 checkpoint; README example uses --backend auto which would require accelerator