FreeToken Kai
Unofficial FreeToken fork serving large MoE models on consumer GPUs with auto-tuning benchmark.
Not verified yetKnown vulnerabilities
How far pickfit got
pickfit hasn’t verified FreeToken Kai yet, so there are no tested install steps.
What follows comes from its README only.
Use it
Before you install: known vulnerabilities
pickfit found 28 known high or critical vulnerabilities in the packages a fresh install brings (OSV.dev, 3 Nov 2026). Weigh this before you install it.
pypi:pillow 12.3.0Pillow: Heap buffer overflow in _tiffWriteProc when saving TIFF to BytesIO (GHSA-332p-7hrc-92wf, critical, fixed in 13.0.0)
See 9 more
pypi:pillow 12.3.0— GHSA-5x8w-8jj5-q52h (critical), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-8g5g-8vmr-9782 (critical), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-v9pv-5gc8-w2m6 (critical), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-w889-685w-c8fh (critical), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-22hg-r758-rrw9 (high), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-27h8-gf7j-v5hc (high), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-3j72-p6qg-6xp7 (high), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-58r4-6rv3-gfh8 (high), fixed in 13.0.0pypi:pillow 12.3.0— GHSA-5xm2-6676-qcp9 (high), fixed in 13.0.0
18 more findings not listed here.
You need first
- A GPU is required
What it does
Good for
- run gpt-oss-120b on one RTX 3060 12 GB
- run 125B MoE on two RTX 3060s or one
- run 35B MoE on RTX 2060 6 GB
- serve vision models with image input on small card
- serve one model split across two consumer GPUs
An alternative to FreeToken.