Skip to content

add bulk image push CLI#728

Open
rasdani wants to merge 8 commits into
mainfrom
codex/bulk-image-push
Open

add bulk image push CLI#728
rasdani wants to merge 8 commits into
mainfrom
codex/bulk-image-push

Conversation

@rasdani

@rasdani rasdani commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add prime images push-batch as a first CLI draft for bulk image builds.
  • Default input mode reads JSONL rows with id/task_id plus raw dockerfile text, generates deterministic short tags, and assembles a batch manifest/payload.
  • Add Harbor scanner mode for immediate task directories using each task's environment/ directory as the build context while excluding tests/ and solution/ from uploaded contexts.
  • Share build-context packaging/upload helpers with the existing single-image prime images push path.

Backend assumptions

This CLI targets a proposed future backend resource: POST /images/build-batches, optional per-item context uploads from the response, then POST /images/build-batches/{batch_id}/start. The current platform endpoint is assumed/not yet implemented, so this PR focuses on client-side validation, manifest generation, request payload assembly, and mocked tests.

Deferred

Docker compose task support is intentionally deferred. Harbor mode detects compose files (docker-compose.yaml, docker-compose.yml, compose.yaml, compose.yml) and fails clearly by default, with --skip-unsupported-compose available to continue with Dockerfile-only tasks.

Validation

  • uv run pytest packages/prime/tests/test_images_push.py -q
  • uv run ruff check packages/prime/src/prime_cli/commands/images.py packages/prime/tests/test_images_push.py
  • uv run ruff format --check packages/prime/src/prime_cli/commands/images.py packages/prime/tests/test_images_push.py
  • uv run ty check packages/prime/src/prime_cli/commands/images.py packages/prime/tests/test_images_push.py
  • uv run prime images push-batch --help
  • git diff --check

Note

Medium Risk
Large new client surface area and parallel uploads depend on a not-yet-implemented backend contract; Harbor mode intentionally omits tests/ and solution/ from contexts, which could surprise users if misconfigured.

Overview
Adds prime images push-batch to build many images against the proposed POST /images/build-batches API: validate inputs, assign deterministic tags via --tag-template, package per-item context tarballs, upload in parallel to presigned URLs, then start the batch. Supports dockerfile mode (JSONL with inline Dockerfile text) and harbor mode (scan task dirs, upload environment/ while excluding tests/ and solution/). Includes --dry-run, --manifest-output, --resume with local upload progress cache, and mid-flight presigned URL refresh on 401/403.

Also adds batch-status, batch-cancel, and batch-retry for batch lifecycle management.

prime images push is refactored to share context packaging, dockerignore handling, visibility validation, and upload helpers with the batch path. Tests in test_images_push.py cover batch flows, harbor packaging, compose rejection, resume, and expired URLs.

Reviewed by Cursor Bugbot for commit 000575b. Bugbot is set up for automated code reviews on this repo. Configure here.

@rasdani rasdani changed the title [codex] add draft bulk image push CLI sketch: add draft bulk image push CLI Jun 7, 2026
@rasdani rasdani changed the title sketch: add draft bulk image push CLI protoype add draft bulk image push CLI Jun 7, 2026
@rasdani rasdani changed the title protoype add draft bulk image push CLI protoype: add draft bulk image push CLI Jun 7, 2026
@rasdani rasdani changed the title protoype: add draft bulk image push CLI do-not-merge: add draft bulk image push CLI Jun 7, 2026
@kcoopermiller kcoopermiller changed the title do-not-merge: add draft bulk image push CLI add bulk image push CLI Jun 9, 2026
@kcoopermiller kcoopermiller marked this pull request as ready for review June 9, 2026 06:27
Comment thread packages/prime/src/prime_cli/commands/images.py Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 000575b. Configure here.

"""Upload packaged contexts in parallel over a single pooled HTTP client."""
already_uploaded = set(already_uploaded or set())
uploadable = [item for item in context_items if item.context_archive is not None]
pending = [item for item in uploadable if item.source_id not in already_uploaded]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resume skips stale uploads

High Severity

On --resume, the local upload cache records only source_id values, and pending uploads are skipped when an id is already marked uploaded. The command still repackages contexts from the current source, so if a Dockerfile or Harbor environment changed between attempts, re-upload is skipped and the batch can start with outdated blobs while the CLI shows fresh tags/manifest data.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 000575b. Configure here.

raise typer.Exit(1)
except APIError as exc:
console.print(f"[red]Error: Failed to refresh upload URLs: {exc}[/red]")
raise typer.Exit(1) from exc

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resume blocks start-only retry

Medium Severity

When resuming a batch, the CLI always calls POST .../upload-urls before it knows whether any contexts still need uploading. If every item is already in the local upload cache, a failure on that refresh call exits before POST .../start, even though no uploads are required to finish a batch whose contexts were uploaded on a prior run.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 000575b. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants