BigMoeOnEdge
Runs larger-than-RAM Mixture-of-Experts models locally by streaming experts from flash.
Tested on Linux
Use it
Linux x86_64
You need first
- System packages:
WebKitGTK 4.1
download install.sh from releases
curl -fsSL https://github.com/Helldez/BigMoeOnEdge/releases/latest/download/install.sh -o /tmp/install.shrun install.sh (installs bmoe-cli/bmoe-server to ~/.local/bin)
sh /tmp/install.sh
Check that it works
bmoe-cli --versionThe output should contain “0.39.0”.
First thing to try
mkdir -p ~/models && curl -fL -o ~/models/SmolLM2-135M-Instruct-Q8_0.gguf https://huggingface.co/bartowski/SmolLM2-135M-Instruct-GGUF/resolve/main/SmolLM2-135M-Instruct-Q8_0.gguf && ls -lh ~/models/SmolLM2*.ggufbmoe-cli -m ~/models/SmolLM2-135M-Instruct-Q8_0.gguf --planWhat it does
- Auto-configure cache and execution planTested
- Serve OpenAI-compatible API for coding agentsTested
- Run MoE models larger than memoryNot shown working
- Run file-editing agent in folderNot tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- 2 of 4 features work
- Works offline
macOS and Windows: the project says it runs there; pickfit hasn’t tried it.
Good for
- Run 21GB MoE model on 12GB phone
- Serve local OpenAI-compatible chat API
- Run agent to read and edit folder files
- Connect coding agents to local server
An alternative to Strata, ds4, colibri and KTransformers.
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
It produces model output
- Replayed from scratch
- Works offline
- Checked on 2 Nov 2026
- Version 0.39.0
Differences the test machine hit
- README: bmoe-cli -m model.gguf --auto -p "Explain MoE routing." works with any model
In the test: stories260K.gguf fails with 'failed to create context' due to K cache q8_0 block size; SmolLM2-135M-Instruct-Q8_0.gguf works and was used for verification - README: README examples use trailing pipes for brevity
In the test: this box rejects trailing pipes (| head) for smoke tests because exit code is hidden; re-ran checks without pipes