InferBridge
Windows-first OpenAI-compatible local LLM server on Intel via OpenVINO GenAI with chat UI and model tools.
Tested on Linux
Use it
Linux x86_64
Clone InferBridge source
rm -rf /tmp/ib2 && git clone https://github.com/Quazmoz/InferBridge.git /tmp/ib2 && cp -r /tmp/ib2/. ./Install with pip editable dev dependencies
pip install -e .[dev]
Where the README and reality differ
- README: `pip install -e .[dev]` then `.\start_server.bat --mock` / `.\start_server.bat --check-devices`
What worked: On Linux use `OV_LLM_MOCK=1 python -m app.server --mock --host 127.0.0.1 --port 8123 --model tinyllama-1.1b-chat-fp16` (installed `inferbridge` CLI works the same); `start_server.sh --mock` requires a `.venv` created by `./setup.sh --minimal` and `start_server.bat` is Windows-only.
Check that it works
inferbridge --helpThe output should contain “usage: inferbridge”.
First thing to try
OV_LLM_MOCK=1 python -m app.server --mock --host 127.0.0.1 --port 8123 --model tinyllama-1.1b-chat-fp16curl -sf http://127.0.0.1:8123/health/readyOnce the server is running, check that it is ready with curl -sf http://127.0.0.1:8123/health/ready.
What it does
These come from the README; pickfit didn’t check them one by one.
- Serve local LLMs through an OpenAI-compatible APINot tested
- Chat with streaming replies in the browserNot tested
- Chat with local vision-language modelsNot tested
- Generate text embeddings locallyNot tested
How far pickfit got
Linux x86_64
- Installs
- Starts
- Answers requests
- Works offline
Windows: the project says it runs there; pickfit hasn’t tried it.
Good for
- chat with local LLM via OpenAI-compatible chat completions
- generate text embeddings locally
- chat with local vision-language models using images
- diagnose runtime, devices and NPU readiness via System Doctor
- run local hardware benchmarks for inference
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
Its service answers requests
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026