Ducklab
Self-hosted harness that runs full dev cycle with multiple LLMs under test gates.
Tested on LinuxKnown vulnerabilities
Use it
Before you install: known vulnerabilities
pickfit found 1 known high or critical vulnerability in the packages a fresh install brings (OSV.dev, 3 Nov 2026). Weigh this before you install it.
go:golang.org/x/image v0.40.0golang.org/x/image/tiff has excessive resource consumption in PackBits decompression (GHSA-q675-qj96-32m9, high, fixed in 0.41.0)
Linux x86_64
You need first
- Runtimes:
Go 1.25+,Node 22+ - System packages:
git,libgtk-3-dev libwebkit2gtk-4.1-dev
These steps ran from a folder named work in the home folder, which already existed. Make it and run them from there:
mkdir -p "$HOME/work" && cd "$HOME/work"git clone the ducklab repo and go build the ducklab CLI, ducklab-engine and ducklab binaries into ~/.local/bin (candidate 3; candidates 1 and 2 skipped: apt needs root, GTK dev headers absent)
cd "$HOME"/work && git clone --depth 1 https://github.com/jrullan/ducklab && cd ducklab && mkdir -p ~/.local/bin && go build -o ~/.local/bin/ducklab-cli ./cmd/ducklab && go build -o ~/.local/bin/ducklab ./cmd/ducklab && go build -o ~/.local/bin/ducklab-engine ./cmd/ducklab-engine
Where the README and reality differ
- README: go build -o ducklab-cli ./cmd/ducklab (then ducklab commands used as on PATH)
What worked: The CLI auto-starts the engine by looking for ducklab-engine on PATH, so ducklab-engine must be built and installed too (make install does this). Built all three binaries into ~/.local/bin. - README: ducklab duckling scorecard
What worked: No such verb: 'unknown duckling command: scorecard'. Duckling verbs are list, add, probe, test, remove, show, set, suggest. Seat suggestions with pass rates come from 'ducklab roster suggest'.
Verified in pickfit: the commands exactly as its sandbox ran them
For the record only: they name the sandbox's home, /home/sbx.
cd /home/sbx/work && git clone --depth 1 https://github.com/jrullan/ducklab && cd ducklab && mkdir -p ~/.local/bin && go build -o ~/.local/bin/ducklab-cli ./cmd/ducklab && go build -o ~/.local/bin/ducklab ./cmd/ducklab && go build -o ~/.local/bin/ducklab-engine ./cmd/ducklab-engineducklab --versioncat > /home/sbx/work/mcp_init.jsonl <<'EOF'
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"pickfit-check","version":"0.0.1"}}}
{"jsonrpc":"2.0","method":"notifications/initialized"}
{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}
EOFducklab mcp serve < /home/sbx/work/mcp_init.jsonlCheck that it works
ducklab --versionThe output should contain “ducklab dev”.
First thing to try
cat > "$HOME"/work/mcp_init.jsonl <<'EOF'
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"pickfit-check","version":"0.0.1"}}}
{"jsonrpc":"2.0","method":"notifications/initialized"}
{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}
EOFducklab mcp serve < "$HOME"/work/mcp_init.jsonlWhat it does
- Generate requirements, spec and plan from a briefNot tested
- Run tasks in solo, pair, tournament, split, council modesNot tested
- Assign model seats from measured scorecardsNot shown working
- Operate the loop from another model via MCPNot shown working
How far pickfit got
Linux x86_64
- Installs
- Starts
- 0 of 4 features work
- Works offline
macOS: the project says it runs there; pickfit hasn’t tried it.
Good for
- write requirements, spec and plan from a brief
- build development tasks with one or several models
- run project test gate for verification
- review task commit
- plan release notes of what shipped
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
Its main function works
- Replayed from scratch
- Works offline
- Checked on 16 Nov 2026
- Version dev
Differences the test machine hit
- README: sudo apt install libgtk-3-dev libwebkit2gtk-4.1-dev; make desktop && make install
In the test: Desktop needs root for apt and GTK/WebKit dev headers (pkg-config gtk+-3.0 missing on this box). Not attempted; only the CLI and engine were built. - README: ducklab roster suggest / cycle commands run from a project
In the test: Commands need a ducklab project: 'no ducklab project at ... or any parent' outside one. Create it with git init and ducklab project init --name ... - README: Needs Go 1.25+
In the test: Built with the box's Go 1.27.4 with no problems; the module downloads needed network during the build step.