PegaInfer
Rust and CUDA engine serving LLMs via OpenAI-compatible API.
Not verified yetKnown vulnerabilities
How far pickfit got
pickfit hasn’t verified PegaInfer yet, so there are no tested install steps.
What follows comes from its README only.
Use it
Before you install: known vulnerabilities
pickfit found 2 known high or critical vulnerabilities in the packages a fresh install brings (OSV.dev, 3 Nov 2026). Weigh this before you install it.
crates:pulldown-cmark 0.13.3Exponential event growth with `ENABLE_WIKILINKS` (RUSTSEC-2026-0414, high, fixed in 0.13.5)
See 1 more
crates:pulldown-cmark 0.13.3— RUSTSEC-2026-0433 (high), fixed in 0.13.6
You need first
- System packages:
CUDA Toolkit with nvcc and cuBLAS - A GPU is required
What it does
pickfit hasn’t run it yet, so these come from the README only.
- Serve LLMs via OpenAI-compatible APINot tested
- Serve completions and chat completionsNot tested
- Tensor parallel prefix cache samplingNot tested
Good for
- serve Qwen3-4B model locally
An alternative to vLLM.