Latency spikes3×
Peak hours can triple response times on a single cloud.
● ROUTING 2.4B TOKENS / DAY
One OpenAI-compatible endpoint. Northvane scores every provider on live latency, price and health, then sends each request down the fastest route.
ROUTING TRAFFIC FOR TEAMS SHIPPING AI IN PRODUCTION
A routing layer for the GPU era. Providers change prices, slow down at peak hours and go offline without warning. Northvane absorbs every one of those moments, so your product never has to.
Peak hours can triple response times on a single cloud.
Your margin moves every time their pricing does.
When one provider goes down, your product goes with it.
01 · BUILT FOR SPEED
P50 LATENCY · LLAMA-3.1-70B · LAST 5 MIN
LOWER IS BETTER
Cloud B
96 ms
Cloud A
84 ms
Cloud E
79 ms
Northvane (routed)
71 ms
02 · LIVE PROVIDER BOARD
Northvane re-ranks every provider continuously. The route you get is the best one right now, not the best one last quarter.
| # | PROVIDER | REGION | P50 LATENCY | $ / 1M TOK | UPTIME 30D | LOAD | TREND |
|---|---|---|---|---|---|---|---|
| 01 | Cloud CACTIVE ROUTE | us-west | 71 ms | $0.42 | 99.99% | ██████░░░░ | ▃▂▂▁▂▁▁▂ |
| 02 | Cloud E | us-central | 79 ms | $0.51 | 99.97% | ███████░░░ | ▂▃▂▃▂▂▃▂ |
| 03 | Cloud A | us-east | 84 ms | $0.48 | 99.95% | ████████░░ | ▃▄▃▂▄▃▄▃ |
| 04 | Cloud B | eu-west | 96 ms | $0.39 | 99.90% | █████░░░░░ | ▄▃▅▄▃▄▅▄ |
| 05 | Cloud D | ap-south | 112 ms | $0.35 | 99.80% | ████░░░░░░ | ▅▆▄▅▇▅▆▅ |
| 06 | Cloud F | eu-north | 138 ms | $0.31 | 99.70% | ███░░░░░░░ | ▆▅▇▆▅▇▆█ |
POLICY · LOWEST LATENCY UNDER $0.50 / 1M
LIVE · P50 OVER LAST 5 MIN · REFRESHED 1 S AGO
FIELD NOTES
“We cut our inference bill by a third in the first week, and stopped getting paged at 3 a.m.”
03 · DROP-IN
Keep the OpenAI SDK you already use. Point it at Northvane and set the provider to auto; the router picks the winner on every call.
route.py
01from openai import OpenAI02 03client = OpenAI(04 base_url="https://api.northvane.dev/v1",05 api_key=os.environ["NORTHVANE_KEY"],06)07 08client.chat.completions.create(09 model="llama-3.1-70b",10 extra_body={"provider":"cloud-a""auto""cloud-c"},11)04 · CAPABILITIES
Live observability. Every request traced with its provider, latency, cost and fallback path.
Automatic failover. A degraded provider is skipped before your users notice.
Smart caching. Repeated prompts are served from cache at zero provider cost.
Budget caps. Set a monthly ceiling; routing gets cheaper as you approach it.
Region pinning. Keep traffic inside the data boundary you choose.
05 · PRICING
Free
Up to 1M routed tokens a month
$0.08
per 1M routed tokens
Custom
Volume pricing and SLAs