Prime CLI v1#766
Open
xeophon wants to merge 21 commits into
Open
Conversation
Removes the `prime_cli/leaves/` layer: command callbacks now live in `prime_cli/commands/` with their Pydantic config schemas beside them in `*_configs.py`, consumed by the router via `Command.config_attr`. `command_registry.py` shrinks ~400 lines: `module` is derived from `path[0]` via a 3-entry alias map (`eval->evals`, `secret->secrets`, `train->rl`) plus a single `module_override` for the `train usage` exception, and `config_attr` is derived from the path. The 135 entries keep only `path`, `summary`, `section`, `run_attr`, `raw`, `positionals`. Rich group-level help panels are restored in `command_router.py` (the per-command pydantic_config panels were already present). Tests updated for Click 8.2.1 console-width pinning and whitespace-normalized assertions.
xeophon
requested review from
JannikSt,
JohannesHa,
burnpiro,
d42me,
kcoopermiller and
willccbb
as code owners
Remove the 25 separate *_configs.py files by merging each module's config schemas directly into the corresponding commands/<name>.py. Configs are placed at the top of each module (above command functions) so builtin-name shadowing (e.g. def list) does not break pydantic annotation evaluation; from __future__ import annotations is added where the merged config classes are referenced before definition. Update test imports that referenced the removed *_configs modules to import the config classes from their command modules instead. Net -309 lines (per-file boilerplate removed). Full suite green (1055 passed, 2 skipped). Amp-Thread-ID: https://ampcode.com/threads/T-019f0942-a19e-752c-be0a-0288e6e7f22f Co-authored-by: Amp <amp@ampcode.com>
Q1 - eliminate the raw run_leaf(leaf_args) forwarding pattern:
* Command now declares verifiers (a Verifiers CLI module name)
instead of raw=True + a thin run_attr forwarder.
* The router execs verifiers.v1.cli.* directly for those commands, so
gepa.py, run_eval_cmd, and env serve/validate disappear.
* env init keeps Prime-side work as a pre_exec hook
(init_preflight) that runs lab-hygiene fixes before the Verifiers
exec, replacing the old subprocess-run-then-hygiene sequence.
Q2 - stop reimplementing Verifiers' own artifact format in Prime:
* _load_eval_directory -> verifiers.read_upload_data().as_dict()
(Prime keeps only the invalid-line warning, an add-on).
* _has_eval_files/_validate_eval_path/_discover_eval_outputs
-> has_eval_artifacts/resolve_eval_artifact_dir/
discover_eval_artifact_dirs.
* convert_eval_results -> delegates to
verifiers.convert_results_for_upload (byte-identical logic).
Verifiers is now the single source of truth for its artifact format; Prime
only owns the hosted-eval upload (the add-on).
Tests updated to mock prime_cli.command_router.exec_verifiers_process
and the new registry shape. Net -206 lines. Suite green (1050 passed,
2 skipped) against verifiers 0.1.15.dev409.
Amp-Thread-ID: https://ampcode.com/threads/T-019f0942-a19e-752c-be0a-0288e6e7f22f
Co-authored-by: Amp <amp@ampcode.com>
xeophon
changed the title
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prime CLI v1 — rewrite the command surface around a declarative command registry and strict Pydantic leaf configs, with a clean Prime ↔ Verifiers ownership boundary. Supersedes #760 (this branch includes those commits).
What changed
Command architecture
command_registry+ longest-path dispatcher replaces the Typer /leaves/routing. Default-command groups, implicit routing, and Typer contexts are gone.*Configmodel and arun(config)entrypoint; the router parses Prime-owned handlers viaprime-pydantic-config.prime eval run,prime eval submit,prime train run,prime lab view,prime env install owner/environment@version.moduleandconfig_attrare derived from the path via a 3-entry module-alias map (eval→evals,secret→secrets,train→rl) plus onemodule_overridefor theprime train usageexception. Each entry keeps onlypath,summary,section,run_attr,positionals, and optionalverifiers/pre_exec.*_configs.py), consumed viaCommand.config_attr.command_router.py.Prime ↔ Verifiers boundary
eval run,env init/validate/serve, andgepa run. Each such command declaresverifiers=<name>; the router execsverifiers.v1.cli.*(eval/init/serve/validate) andverifiers.scripts.gepadirectly through the selected workspace interpreter.env initkeeps a Prime-sidepre_execpreflight hook (lab hygiene) that runs before the Verifiers exec, replacing the old subprocess-run-then-hygiene sequence.verifiers.read_upload_data/convert_results_for_upload. Prime keeps only the hosted-eval upload (the add-on) and an invalid-line warning. The eval manifest contract and the eval-run sidecar are removed.Deprecated alias
prime rlis hidden and deprecated. Using it prints[DEPRECATED] The 'rl' command is deprecated. Use 'prime train' instead.and dispatches toprime train.Platform
requires-python = ">=3.11,<3.14"(drops Python 3.10); CI and release Docker matrices are aligned.verifiers>=0.1.15.dev371; the Verifiers git pin is removed fromuv.sources.docs/verifiers-v1-cli.mddocuments the Prime vs Verifiers ownership split and native artifacts.Companion change
Verifiers CLI lifecycle surface: PrimeIntellect-ai/verifiers#1857
Verification
ruff check/ruff format --checkclean acrosspackages/prime.0.1.15.dev409): 1050 passed, 2 skipped.test_command_router.pygreen, including the command-contract test that imports every command module and resolvesrun_attr/config_attr/verifiers, and the no-typer-in-source test.