Per-conversation USD budgets
set_session_budget("sess_42", max_usd=50). Aggregator sums spend across every SDK on that session_id. Crosses the threshold → every SDK raises BudgetExceeded at the next call boundary. No race window.
$ ts features
Detect, intervene, calibrate. The SDK detects in-process. The cloud intervenes via callback. The judge calibrates which signals you can wire to automation without false-positive fatigue.
[01] detection · 14 rules
V0 · 8 rules
Same tool, ≥3 cosine-similar calls in 60s
Agent stuck on a tool it's not getting unstuck from
Prompt-tokens-per-turn slope > 1500/turn
ReAct context window growing without bound
Exact-hash repeat embedding within session
Same vector look-up paid for repeatedly
No user-facing output for 5 min, calls firing
Background agent that nobody is reading
Classification prompt sent to a frontier model
Opus answering a yes/no — Haiku would suffice
≥5 identical calls in 30s
Provider 529 + no exponential backoff
≥30 tool defs OR ≥30KB of tool JSON
The MCP problem — 55K tokens before user types
≥3 retrieval calls with overlapping queries in 120s
RAG pipeline that keeps re-fetching
Vision · 3 rules
Same image SHA-256 (or phash distance ≤6) across calls
Same screenshot re-uploaded turn after turn
High-detail flag on low-detail-suitable image
Paying 4× for detail you don't need
Vision dollars concentrated in <5% of sessions
One agent eating the vision budget
Audio · 1 rule
Deepgram channels≥2 with multichannel=true
Per-channel billing doubled without reason
Voice · 1 rule
Same text_hash across ≥3 ElevenLabs voice_ids in 10s
A/B test scaffold leaked into production
Rerank · 1 rule
Same Cohere rerank request_hash across ≥2 calls in 30s
Reranks fired without a cache layer
Repair · 1 rule
≥2 correction-shaped user turns + similar regenerations within 10 turns
Cost-aligned proxy for hallucination/grounding failures
[02] intervention pack
set_session_budget("sess_42", max_usd=50). Aggregator sums spend across every SDK on that session_id. Crosses the threshold → every SDK raises BudgetExceeded at the next call boundary. No race window.
Absolute 100k tokens/min default. Per-customer adaptive baselines on Pro. Choose alert or block per project. Rolling 60-second window in the dashboard.
One click in the dashboard, every connected SDK halts at the next record_call within ≈4 seconds — the pull-based policy plane's worst-case lag. Works through corporate proxies that block SSE / WebSocket.
2-second poll on active sessions, 5-second on idle. Stateless. Fail-open by default. Scales linearly. Operators don't lose control when the proxy mid-tier eats long-lived connections.
[03] calibration · pro tier
01
Gray-zone events (confidence 0.5–0.8) get the recent context window forwarded to Claude Haiku 4.5. Either ratifies (→ ≈0.9) or vetoes (→ ≈0.2). A 10% sample also routes to Gemini 2.5 Flash to measure cross-family asymmetry, published weekly as the bias datasheet.
02
7 signals — tool-call distribution, model-routing, retry rate, p95 latency, error rate, prompt-token distribution, completion-token distribution — rolled into a single 0–1 index against a 7-day baseline. Slack alert at >2σ drop or per-signal threshold breach.
03
POST /api/cost/estimate for per-call gating (point estimate + p25/p75 band, 14-day lookback). PLUS the V2.1 forecast endpoint: GET /api/cost/forecast?period=week|month|quarter — velocity-based projection with p25/p75 and p05/p95 bands, plus a forecast_variance alert that fires when projected > 110% of the prior period. Cold-start falls back to a global cohort prior with low_historical_data warning.
04
Every state-changing action — project create, user invite, role change, key rotate, policy update, kill-switch toggle — written with database-level append-only enforcement. 90-day retention on Pro, up to 7 years on Enterprise. CSV export.
[04] providers
| provider | sdk | streaming | async | notes |
|---|---|---|---|---|
| Anthropic | anthropic | ✓ | ✓ | |
| OpenAI | openai | ✓ | ✓ | incl. Whisper + GPT-image |
| Google Gemini | google-genai | ✓ | ✓ | Vertex AI via vertexai=True |
| AWS Bedrock | boto3 | ✓ | — | |
| Voyage AI | voyageai | — | ✓ | |
| Cohere | cohere | ✓ | ✓ | chat + embed + rerank |
| Replicate | replicate | ✓ | ✓ | |
| Deepgram | deepgram-sdk | ✓ | ✓ | |
| ElevenLabs | elevenlabs | ✓ | ✓ | |
| Plus every OpenAI-compatible endpoint (DeepSeek, Together, Fireworks, Groq, OpenRouter, Anyscale, Mistral, Perplexity, vLLM, Ollama, TGI, LM Studio) instrumented transparently via the OpenAI wrapper. | ||||
$ next