OMeter
Benchmarks Ollama models on local and cloud endpoints with sortable tables.
Tested on Linux
Use it
Linux x86_64
You need first
- Runtimes:
python 3.14+ - Keys or accounts: OLLAMA_CLOUD_API_KEY for benchmarking cloud models
install from GitHub with uv tool
uv tool install git+https://github.com/EndoTheDev/OMeter.git
Check that it works
ometer --version 2>&1The output should contain “ometer 0.4.0”.
First thing to try
ometer --local 2>&1Run ometer --local to start with your local Ollama models.
What it does
- Measure time to first token and tokens per secondNot tested
- List local and cloud modelsNot shown working
- View results in sortable tablesNot tested
- Export results to JSON or CSVNot shown working
How far pickfit got
Linux x86_64
- Installs
- Starts
- 0 of 4 features work
- Works offline
Good for
- list local Ollama models
- list cloud Ollama models
- benchmark time-to-first-token and tokens-per-second
- export benchmark results to JSON or CSV
- view benchmark history per model
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
Its main function works
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026
- Version 0.4.0
Differences the test machine hit
- README: python 3.14+
In the test: uv automatically downloaded and used cpython-3.15.2 to satisfy the requirement on this Python 3.12 box - README: uv tool install . from the project directory
In the test: installed with `uv tool install git+https://github.com/EndoTheDev/OMeter.git` because the work dir contains only README.md, no pyproject.toml