Throttle
CLI that proves which API traffic can move to self-hosted open-weight models and routes it with fallback.
Tested on Linux
Use it
Linux x86_64
Install throttle from own/mvp branch via pip
pip install git+https://github.com/KushagraKanaujia/throttle@own/mvp
Check that it works
throttle --helpThe output should contain “throttle demo”.
First thing to try
throttle demoRun throttle demo to see a sample report of how much traffic could move to self-hosted models.
What it does
- See which traffic can move without losing qualityTested
How to use it
- Run
throttle demo. - Read the "Share movable now" line for the share of traffic and monthly dollars that can move.
- Run
- Route API traffic to self-hosted modelsNot shown working
- Retest workloads when new models come outNot tested
- Measure cost per million tokens with 95% intervalsNot shown working
How far pickfit got
Linux x86_64
- Installs
- Starts
- 1 of 4 features work
- Works offline
Good for
- capture OpenAI or Anthropic traffic via local proxy
- replay captured requests to self-hosted model for shadow testing
- measure dollars per 1M tokens with 95% confidence intervals
How pickfit tested it
Each run starts from a clean machine and follows the steps above. How pickfit verifies
Linux x86_64
Its main function works
- Replayed from scratch
- Works offline
- Checked on 30 Oct 2026
- Version 0.7.0