Skip to content

Support external v1 harness plugins#1828

Draft
sethkarten wants to merge 1 commit into
mainfrom
feat/external-prime-agent-headless-harness
Draft

Support external v1 harness plugins#1828
sethkarten wants to merge 1 commit into
mainfrom
feat/external-prime-agent-headless-harness

Conversation

@sethkarten

@sethkarten sethkarten commented Jun 22, 2026

Copy link
Copy Markdown

Summary

Adds generic v1 support for external harness packages without vendoring their implementation into public Verifiers.

  • allows flat external plugin modules to be selected by normalized id
  • preserves internal plugin import errors instead of masking missing dependencies as missing plugins
  • adds a Prime runtime wait fallback that polls get() if wait_for_creation() times out and continues when the sandbox is actually RUNNING
  • documents the generic external plugin contract in architecture docs
  • adds regression tests for external harness loading, custom config forwarding, exact label preservation, and the Prime wait fallback

This lets research teams install private harness packages alongside Verifiers while keeping harness source in private repositories.

Testing

uv run --isolated --no-project   --with-editable .   --with pytest   --with pytest-asyncio   --with renderers==0.1.8.dev43   python -m pytest tests/v1/test_loaders.py
uv run --isolated --no-project --with ruff ruff check   verifiers/v1/ARCHITECTURE.md   verifiers/v1/loaders.py   verifiers/v1/runtimes/prime.py   tests/v1/fixtures/external_harness_v1.py   tests/v1/test_loaders.py
git diff --check

@sethkarten sethkarten force-pushed the feat/external-prime-agent-headless-harness branch from 5954224 to ca0615b Compare June 22, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant