Skip to content

improvement: show cached input prices#771

Open
JannikSt wants to merge 1 commit into
mainfrom
improvement/show-cached-input-prices
Open

improvement: show cached input prices#771
JannikSt wants to merge 1 commit into
mainfrom
improvement/show-cached-input-prices

Conversation

@JannikSt

@JannikSt JannikSt commented Jul 1, 2026

Copy link
Copy Markdown
Member

Surface the prefix-cache discount rate in the CLI now that RFT pricing carries it.

  • prime train models gains a Cached column between Input and Output
  • prime train run preview and prime train usage render an "Inference (cached input)" line/row when the model advertises a cached rate or the run consumed cached tokens

Note

Low Risk
Display and optional schema fields only; no billing logic changes, with tests for legacy backends and cost fallback behavior.

Overview
Adds optional API fields and CLI display for prefix-cache (cached) inference input pricing and usage, aligned with RFT backends that bill cached input tokens at a discounted rate.

Models & API: RunUsageBreakdown / RunPricing gain cached_input_tokens and inference_cached_input_per_mtok. RLModel gains cached_input_*, list_cached_*, and effective_cached_* fields; resolve_prices supports a virtual inference_cached_input category with the same list/effective semantics as other price types.

CLI: prime train models adds a Cached column; run confirmation Pricing shows Inference Cached Input when a rate exists; prime train usage splits inference input into non-cached vs Inference (cached input) rows (tokens and derived costs sum correctly, with fallback to the full input rate when cached tokens exist but no cached rate is sent). JSON help strings and --output json include the new fields.

Tests: Coverage for models table/JSON cached pricing and usage row visibility, absence on old backends, and rate fallback.

Backward compatible: missing fields keep prior layouts and parsing behavior.

Reviewed by Cursor Bugbot for commit 0280ab7. Bugbot is set up for automated code reviews on this repo. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant