Skip to content

Fix CI: switch to data/models.json with paid & free multipliers#195

Open
Rob Bos (rajbos) wants to merge 2 commits into
mainfrom
rajbos/fix-model-multiplier-empty-parsing
Open

Fix CI: switch to data/models.json with paid & free multipliers#195
Rob Bos (rajbos) wants to merge 2 commits into
mainfrom
rajbos/fix-model-multiplier-empty-parsing

Conversation

@rajbos

@rajbos Rob Bos (rajbos) commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Problem

The daily update-model-multipliers CI job has been failing with:

ValueError: Parsed zero models from release body.

The upstream source stopped populating the multiplier column in release markdown bodies. More importantly, GitHub Copilot has moved to usage-based billing — the old single "multiplier" concept has been replaced by separate paid-plan and free-plan values in the notifier's structured data/models.json file.

Root cause

Our script was parsing the release body markdown table, which no longer contains multiplier data. The correct data source is data/models.json in the notifier repo, which has structured fields: multiplier_paid, multiplier_free, provider, release_status.

Changes

scripts/update-model-multipliers.py

  • New data source: fetch data/models.json directly instead of parsing markdown
  • Parse both multiplier_paid (Business / Enterprise / Pro) and multiplier_free (Copilot Free)
  • Sentinel: -1 = model not available on this plan; 0 = included free; >0 = premium
  • Empty field → defaults to 1 (unknown premium, costs 1 PRU per request)
  • Still fetches the latest release tag for provenance tracking

src/lib/model-multipliers.generated.ts

  • Now exports CURRENT_MODEL_MULTIPLIERS_PAID and CURRENT_MODEL_MULTIPLIERS_FREE
  • CURRENT_MODEL_MULTIPLIERS kept as a backward-compat alias for _PAID

src/lib/model-multipliers.legacy.ts

  • Added LEGACY_MODEL_MULTIPLIERS_FREE for legacy model identifiers
  • Renamed LEGACY_MODEL_MULTIPLIERSLEGACY_MODEL_MULTIPLIERS_PAID (alias preserved)

src/lib/utils.ts

  • Exports MODEL_MULTIPLIERS_PAID and MODEL_MULTIPLIERS_FREE
  • getModelMultiplier(model, plan?) is now plan-aware (defaults to 'paid')
  • -1 sentinel is folded to 0 in cost calculations (not-available = no cost)

Verification

All 239 tests pass ✅

When GitHub re-populates multiplier_paid/multiplier_free in the docs (as their billing data stabilizes), the daily workflow will automatically pick up the real values.

With GitHub Copilot's usage-based billing, model multipliers no longer
apply. Each premium request counts as 1 PRU at \.04 regardless of model.

Changes:
- scripts/update-model-multipliers.py: treat empty multiplier cells as 1
  (no longer a fatal error; upstream data stopped publishing multipliers)
- model-multipliers.legacy.ts: flatten all non-zero multipliers to 1;
  only the free/default distinction (0 vs 1) still has meaning
- model-multipliers.generated.ts: regenerate from latest release
- utils.ts: remove '* multiplier' from partial-month cost projection
  (full-month path already omitted it); update comments
- App.tsx: remove Multiplier column from model table; update tooltip
- Tests: update model-summary-total and model-info-limits accordingly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Tagging Rob Bos (@rajbos) for notifications

Instead of parsing the markdown release body, the update script now fetches
data/models.json directly from rajbos/github-copilot-model-notifier. This
file has structured fields for both multiplier_paid and multiplier_free.

Changes:
- scripts/update-model-multipliers.py: fetch data/models.json via raw URL;
  parse multiplier_paid and multiplier_free; generate both _PAID and _FREE
  constants; keep CURRENT_MODEL_MULTIPLIERS as backward-compat alias for _PAID
- model-multipliers.generated.ts: regenerated with PAID/FREE split
- model-multipliers.legacy.ts: add LEGACY_MODEL_MULTIPLIERS_FREE;
  rename to LEGACY_MODEL_MULTIPLIERS_PAID (alias preserved for compat)
- utils.ts: import _PAID/_FREE variants; expose MODEL_MULTIPLIERS_PAID,
  MODEL_MULTIPLIERS_FREE; make getModelMultiplier() plan-aware (default 'paid');
  -1 sentinel (not available on plan) is folded to 0 for cost math

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rajbos Rob Bos (rajbos) changed the title Remove model multipliers: adapt to usage-based billing Fix CI: switch to data/models.json with paid & free multipliers Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant