From 51ff901281dcd2d361c758faaa0ec947e456afde Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 18 Jun 2026 06:37:29 +0000 Subject: [PATCH 1/2] docs(skill): add reusable 'pin self-hosted runner Python env' skill + prompt Captures the diagnosis+fix for the post-reboot ModuleNotFoundError (mlx_lm) on the kakeya-mac-m4 runner: lightweight env-probe diagnosis, 3-layer fix (pin venv on the runner agent PATH via .path/.env|launchd|systemd; resolve a pinned interpreter in the workflow/executor instead of bare python3; fail-fast import gate), reboot-inclusive verification, and the Cloud-VM-vs-runner distinction (Mac-only deps belong on the runner, not the Linux Cloud Agent env). Includes a ready-to-paste setup-agent prompt; generalized for any Claude/Codex agent. Co-authored-by: FluffyAIcode --- docs/kakeyainferenceenginebuildskill.md | 1 + .../pin-selfhosted-runner-python-env-skill.md | 193 ++++++++++++++++++ 2 files changed, 194 insertions(+) create mode 100644 docs/skills/pin-selfhosted-runner-python-env-skill.md diff --git a/docs/kakeyainferenceenginebuildskill.md b/docs/kakeyainferenceenginebuildskill.md index 38a40c8..afab3e4 100644 --- a/docs/kakeyainferenceenginebuildskill.md +++ b/docs/kakeyainferenceenginebuildskill.md @@ -351,5 +351,6 @@ If any answer is "no", write the weaker, true claim. - v0.5-cuda scorecard (+ honest §5): `docs/reports/kakeya-inference-engine-v0.5-cuda.md` - Engine vs vLLM long-context journey: `docs/reports/kakeya-engine-vs-vllm-h200.md`, `docs/reports/kakeya-vs-vllm-longcontext-h200.md` - MLX port lessons: `docs/mlx-port-lessons.md` +- Self-hosted runner Python pinning (reboot-proof mlx_lm/torch/transformers): `docs/skills/pin-selfhosted-runner-python-env-skill.md` - f_θ training pipeline: `docs/design/k3-f-theta-training-pipeline.md` - Session capacity / cross-host: `docs/adr/0014-agent-connection-capacity-and-cross-host-topology-tests.md` diff --git a/docs/skills/pin-selfhosted-runner-python-env-skill.md b/docs/skills/pin-selfhosted-runner-python-env-skill.md new file mode 100644 index 0000000..2f348fd --- /dev/null +++ b/docs/skills/pin-selfhosted-runner-python-env-skill.md @@ -0,0 +1,193 @@ +# Skill: Pin a self-hosted runner's Python env (survive reboots, reproducible heavy ML deps) + +**Reusable across agents (Claude / Codex / Cursor).** Copy this file or paste the +prompt in the appendix. It is written to be repo-agnostic; the concrete examples +use a GitHub Actions self-hosted Mac runner driving MLX (`mlx_lm`/`torch`/ +`transformers`), but the pattern applies to any self-hosted runner (Mac or Linux) +that runs heavy ML/native deps from a virtualenv. + +--- + +## 1. When to use this skill + +Trigger it when **a self-hosted runner job fails on a missing module that "used to +work"**, especially after a host **reboot / OS or Python upgrade / runner +re-register**. Classic signatures: + +- `ModuleNotFoundError: No module named 'mlx_lm'` (or `torch`, `transformers`, …) + in a job that previously passed. +- The failure is **fast** (seconds) — it dies at `import`, before any real work. +- A **lightweight probe** (one that only needs stdlib + a base package) still + passes, proving the runner is *online* but pointing at the **wrong interpreter**. +- The interpreter version changed (e.g. `python=3.14.3` where it used to be + `3.13.x`), or `pkg=None` for a package that should be installed. + +Root cause is almost always: the workflow invokes a **bare `python3`**, and after +the reboot the default `python3` on `PATH` is no longer the venv that has the +deps. The venv still exists; nothing points at it. + +--- + +## 2. Diagnose first (don't guess) + +Run the **cheapest possible probe** through the same runner path to read the +interpreter + module state, instead of assuming. Example (adapt the import list): + +```bash +python3 - <<'PY' +import sys +def v(m): + try: + mod = __import__(m); return getattr(mod, "__version__", "ok") + except Exception as e: + return f"MISSING ({e.__class__.__name__})" +print("python =", sys.version.split()[0], "| exe =", sys.executable) +for m in ("mlx", "mlx_lm", "torch", "transformers"): + print(f"{m} = {v(m)}") +PY +``` + +Decision rule: +- **Runner online + probe shows wrong `python`/`exe` or `MISSING` deps** → this skill (interpreter pinning). +- **Probe itself never starts (job stuck `queued`/`pending`)** → the runner *agent* + is down; restart the agent first (different problem). + +> In CI-driven runners, route the probe through the same executor the real jobs +> use (so `PATH`/env match). A one-liner like the above, committed as a tiny +> "env-probe" job/preset, is worth keeping permanently. + +--- + +## 3. Fix — three layers (do all three; they are defense-in-depth) + +### Layer A — Pin the interpreter the runner *agent* sees (host side, durable) + +Make the venv's `bin` the first thing on the **runner agent's** `PATH`, so a bare +`python3` resolves to the venv even across reboots. Pick the mechanism for how the +agent is launched: + +- **GitHub Actions runner as a service (recommended).** The runner reads a + `.env` and a `.path` file in its install dir at start: + ```bash + cd ~/actions-runner + echo "$HOME/kakeya-venv/bin" > .path # prepended to PATH + echo "VIRTUAL_ENV=$HOME/kakeya-venv" >> .env + ./svc.sh stop && ./svc.sh start # reload + ``` + (`.path` is concatenated ahead of the system PATH for every job; `.env` injects + process env. Both persist across reboots because the service re-reads them.) +- **launchd plist (macOS), if not using `svc.sh`.** In the runner's + `~/Library/LaunchAgents/.plist`, set: + ```xml + EnvironmentVariables + + PATH/Users/<you>/kakeya-venv/bin:/usr/bin:/bin:/usr/sbin:/sbin + + ``` + then `launchctl unload/load` the plist. +- **systemd (Linux self-hosted).** In the runner unit: + `Environment="PATH=/opt/kakeya-venv/bin:%h/.local/bin:/usr/bin:/bin"`, then + `systemctl daemon-reload && systemctl restart `. + +Verify: `python3 -c "import mlx_lm, torch, transformers; print('ok')"` from a job. + +### Layer B — Make the workflow/executor resolve a *pinned* interpreter (repo side, robust) + +Never call a bare `python3` for the heavy job. Resolve an explicit interpreter so +the repo is robust even if Layer A drifts: + +1. Add a repo/runner variable, e.g. `KAKEYA_MAC_PYTHON`, pointing at the venv + python (`/Users//kakeya-venv/bin/python`). Default-discover if unset: + ```bash + PYBIN="${KAKEYA_MAC_PYTHON:-}" + for c in "$PYBIN" "$HOME/kakeya-venv/bin/python" "$(command -v python3.13)" "$(command -v python3)"; do + [ -n "$c" ] && [ -x "$c" ] && "$c" -c 'import mlx_lm' 2>/dev/null && { PYBIN="$c"; break; } + done + ``` +2. Use `$PYBIN` (or substitute a `${PYTHON}` token in your command templates) + instead of `python3` for the actual workload. If your executor spawns argv + lists (no shell), resolve the token to `$PYBIN` before `subprocess.run`. + +### Layer C — Fail fast with a clear message (repo side, observability) + +Before the expensive step, assert the deps and **print a fix hint** so the next +failure is self-explanatory instead of a deep `ModuleNotFoundError`: + +```bash +"$PYBIN" - <<'PY' || { echo "::error::runner python missing ML deps — see pin-selfhosted-runner-python-env-skill.md (Layer A)"; exit 90; } +import mlx_lm, torch, transformers # noqa +PY +``` + +--- + +## 4. Verify the fix + +1. Re-run the lightweight env-probe → correct `python`/`exe`, all deps present. +2. Re-run one **real** (heavy) job → no `ModuleNotFoundError`, completes. +3. **Reboot the host and re-run** (the actual regression you are fixing) → still + green. This step is the whole point; do not skip it. + +--- + +## 5. Generalizing to a *Cloud Agent* VM env setup (different machine!) + +Do **not** confuse the self-hosted runner with the Cloud Agent VM: +- The **Cloud Agent VM** is typically Linux; it runs the *client* that dispatches + jobs and the unit-test gate. **Mac-only deps (MLX) do not belong there.** Put + only what the client/tests need into the Cloud Agent env setup (base image + + startup script), and pin versions. +- The **self-hosted runner** is where the heavy/native/Mac deps live. Pin them + there (Layers A–C above), not in the Cloud VM env setup. + +For the Cloud Agent VM specifically: bake stable deps into the **base image**, do +slow-changing installs in the **startup script**, and pin versions so a new VM is +reproducible. (In Cursor, this is the "env setup agent" config.) + +--- + +## 6. Anti-patterns + +- ❌ `pip install` the missing dep into whatever `python3` happens to be active + (often a too-new system Python with no wheels for `torch`/`mlx_lm`). Pin to the + known-good venv instead. +- ❌ Hardcoding an absolute interpreter path in many places. Resolve once + (variable + discovery) and reuse. +- ❌ "It works now" without a reboot test — the regression is reboot-triggered. +- ❌ Relying on an interactive shell's `source venv/bin/activate`; CI jobs and + services don't run your `.zshrc`. + +--- + +## Appendix — ready-to-paste prompt for a setup agent + +> **Task: make our self-hosted CI runner's Python environment reboot-proof.** +> +> Symptom: jobs on our self-hosted runner fail fast with +> `ModuleNotFoundError: No module named 'mlx_lm'` after the host rebooted; a +> lightweight env-probe shows the runner's default `python3` switched to a newer +> interpreter that lacks our ML stack (`mlx_lm`/`torch`/`transformers`), while the +> known-good venv still exists but is no longer on `PATH`. +> +> Do all of the following, smallest-diff first, and verify each: +> 1. **Diagnose:** run a tiny probe that prints `sys.version`, `sys.executable`, +> and import status of `mlx_lm, torch, transformers` through the same path the +> real jobs use. Confirm the wrong interpreter / missing modules. +> 2. **Host (runner agent):** pin the venv's `bin` ahead of system `PATH` for the +> runner service so a bare `python3` resolves to the venv across reboots — via +> the runner's `.path`/`.env` files (GitHub Actions `svc.sh`), or the +> launchd/systemd unit's `PATH` env. Reload the service. +> 3. **Repo (workflow/executor):** stop calling bare `python3` for the heavy job. +> Resolve a pinned interpreter from a `*_PYTHON` repo/runner variable, with a +> discovery fallback that picks the first candidate where `import mlx_lm` +> succeeds; use it for the workload commands. +> 4. **Repo (fail-fast):** before the expensive step, assert +> `import mlx_lm, torch, transformers` and emit a clear `::error::` with a link +> to this skill if missing (exit non-zero). +> 5. **Verify, including a reboot:** env-probe green, one real heavy job green, +> then reboot the host and re-run the same job — must still be green. +> 6. **Pin versions** in the venv (freeze a lockfile) and document the venv path + +> rebuild steps so the environment is reproducible, not just patched. +> +> Keep the heavy/native deps on the self-hosted runner only; do NOT add Mac-only +> deps to the Cloud Agent (Linux) VM env setup. From 16440ff48fc76a4a64a9d36f230059ea9422b964 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 18 Jun 2026 06:51:50 +0000 Subject: [PATCH 2/2] feat(mac-bridge): pin workload interpreter (Layer B) + import self-check gate (Layer C) Reboots can repoint the runner's default python3 to one without mlx_lm, which broke every full-engine preset with a deep ModuleNotFoundError. Make the workload interpreter explicit and verified: - inference_engine/bridge/runner_python.py (NEW, pure + 100% unit-tested): workload_python_candidates (pin KAKEYA_MAC_PYTHON -> venvs -> PATH), resolve_workload_python (first interpreter that can import mlx_lm; else fallback), preset_requires_gate (mlx-/k3- engine presets, minus env-probe/ upgrade), substitute_python, gate_error_message. - scripts/mac_bridge/run_preset.py: resolve the pinned interpreter, rewrite bare python3 argv0 to it, export KAKEYA_MAC_PYTHON to the subprocess, and FAIL FAST (exit 90 + ::error::) when a gated preset has no mlx_lm-capable interpreter. - scripts/run_kakeya_mac.sh: honor KAKEYA_MAC_PYTHON; preflight asserts mlx+mlx_lm. CI enforcement: the resolution/gate logic lives in the unit-tested, 100%-coverage library (runner_python.py), so every PR exercises it on the Linux gate. See docs/skills/pin-selfhosted-runner-python-env-skill.md. Co-authored-by: FluffyAIcode --- inference_engine/bridge/runner_python.py | 123 ++++++++++++++++++ scripts/mac_bridge/run_preset.py | 44 ++++++- scripts/run_kakeya_mac.sh | 13 +- .../bridge/test_runner_python.py | 108 +++++++++++++++ 4 files changed, 283 insertions(+), 5 deletions(-) create mode 100644 inference_engine/bridge/runner_python.py create mode 100644 tests/inference_engine/bridge/test_runner_python.py diff --git a/inference_engine/bridge/runner_python.py b/inference_engine/bridge/runner_python.py new file mode 100644 index 0000000..1b0c27b --- /dev/null +++ b/inference_engine/bridge/runner_python.py @@ -0,0 +1,123 @@ +"""Pin the Mac-bridge workload interpreter (Layer B) + import self-check (Layer C). + +A self-hosted runner's default ``python3`` can silently change across reboots / +OS upgrades (observed 2026-06-18: it flipped to a Python 3.14 without ``mlx_lm``, +breaking every full-engine preset with a deep ``ModuleNotFoundError``). The +mac-bridge executor used to invoke a bare ``python3`` for the workload, so it +inherited whatever interpreter happened to be first on ``PATH``. + +This module makes the workload interpreter **explicit and verified**: + +* **Layer B — resolution.** Build an ordered candidate list (a pinned + ``KAKEYA_MAC_PYTHON``, common venv paths, then ``PATH`` pythons) and pick the + first one that can import the gate module (``mlx_lm``); fall back to the first + existing candidate otherwise. +* **Layer C — gate.** For presets whose workload needs ``mlx_lm`` (the ``mlx-`` / + ``k3-`` engine families, minus the env-probe / upgrade tools that exist to + diagnose/repair the env), fail fast with a clear message instead of a deep + import error when no capable interpreter exists. + +All functions here are pure / dependency-injected so they are unit-tested on the +Linux gate (the CLI ``scripts/mac_bridge/run_preset.py`` is a thin caller). See +``docs/skills/pin-selfhosted-runner-python-env-skill.md``. +""" + +from __future__ import annotations + +import os +import shutil +from dataclasses import dataclass +from typing import Callable, List, Mapping, Optional, Sequence + +# The single module whose absence broke the runner; importing it implies the +# full MLX-LM stack is wired for the interpreter. +GATE_MODULE = "mlx_lm" + +# ``mlx-``/``k3-`` presets that must NOT be import-gated: these exist precisely +# to probe or repair the environment, so they must run even when mlx_lm is gone. +_IMPORT_GATE_SKIP = frozenset({"mlx-env-probe", "mlx-upgrade"}) + +SKILL_DOC = "docs/skills/pin-selfhosted-runner-python-env-skill.md" + + +def workload_python_candidates( + environ: Mapping[str, str], + *, + which: Callable[[str], Optional[str]] = shutil.which, + expanduser: Callable[[str], str] = os.path.expanduser, +) -> List[str]: + """Ordered, de-duplicated interpreter candidates for the heavy workload. + + Priority: the explicit pin (``KAKEYA_MAC_PYTHON``), then conventional venv + locations, then ``PATH`` pythons (a pinned minor version before the bare + ``python3`` that a reboot may have repointed).""" + raw = [ + environ.get("KAKEYA_MAC_PYTHON"), + expanduser("~/kakeya-venv/bin/python"), + expanduser("~/.venv/bin/python"), + which("python3.13"), + which("python3"), + ] + out: List[str] = [] + for c in raw: + if c and c not in out: + out.append(c) + return out + + +@dataclass(frozen=True) +class ResolvedPython: + """The interpreter chosen for the workload.""" + + path: str + gate_module_ok: bool # whether ``path`` can import GATE_MODULE + from_pin: bool # whether it came from ``KAKEYA_MAC_PYTHON`` + + +def resolve_workload_python( + candidates: Sequence[str], + can_import: Callable[[str], bool], + *, + pinned: Optional[str] = None, +) -> Optional[ResolvedPython]: + """Pick the first candidate that can import :data:`GATE_MODULE`; otherwise + the first candidate (a fallback whose ``gate_module_ok`` is ``False``). + Returns ``None`` only when there are no candidates at all.""" + first: Optional[str] = None + for c in candidates: + if first is None: + first = c + if can_import(c): + return ResolvedPython(c, True, c == pinned) + if first is None: + return None + return ResolvedPython(first, False, first == pinned) + + +def preset_requires_gate(preset_name: str) -> bool: + """True iff a preset's workload needs :data:`GATE_MODULE` (so a missing + import must fail fast). The ``mlx-`` / ``k3-`` engine presets do; the + env-probe and upgrade tools (which diagnose/repair the env) are exempt.""" + if preset_name in _IMPORT_GATE_SKIP: + return False + return preset_name.startswith(("mlx-", "k3-")) + + +def substitute_python(argv: Sequence[str], pybin: str) -> List[str]: + """Rewrite a leading bare ``python3`` to the resolved interpreter ``pybin``. + Non-``python3`` argv (e.g. ``bash run_kakeya_mac.sh``, which reads + ``KAKEYA_MAC_PYTHON`` itself) is returned unchanged.""" + a = list(argv) + if a and a[0] == "python3": + a[0] = pybin + return a + + +def gate_error_message(preset_name: str, pybin: str) -> str: + """The fail-fast message when a gated preset has no mlx_lm-capable python.""" + return ( + f"runner python '{pybin}' cannot import {GATE_MODULE!r}, which preset " + f"'{preset_name}' requires. The runner's default python likely changed " + f"(e.g. after a reboot). Pin the venv via KAKEYA_MAC_PYTHON or the runner " + f"agent PATH and reinstall the ML stack — see {SKILL_DOC}." + ) diff --git a/scripts/mac_bridge/run_preset.py b/scripts/mac_bridge/run_preset.py index a95122a..4a63c00 100644 --- a/scripts/mac_bridge/run_preset.py +++ b/scripts/mac_bridge/run_preset.py @@ -32,10 +32,29 @@ build_commands, parse_manifest_text, ) +from inference_engine.bridge.runner_python import ( + GATE_MODULE, + gate_error_message, + preset_requires_gate, + resolve_workload_python, + substitute_python, + workload_python_candidates, +) LOG_DIR = Path(".mac-bridge/logs") +def _can_import_gate_module(pybin: str) -> bool: + """True iff interpreter ``pybin`` can import the gate module (mlx_lm).""" + try: + return subprocess.run( + [pybin, "-c", f"import {GATE_MODULE}"], + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ).returncode == 0 + except OSError: + return False + + def main() -> int: ap = argparse.ArgumentParser(description=__doc__) ap.add_argument("--manifest", default=".mac-bridge/request.json") @@ -59,6 +78,23 @@ def main() -> int: print(json.dumps(argv)) return 0 + # Layer B — resolve a PINNED workload interpreter instead of trusting the + # bare ``python3`` on PATH (which a reboot can repoint to a python without + # mlx_lm). Layer C — gate: mlx-/k3- engine presets fail fast with a clear + # message when no mlx_lm-capable interpreter exists. + pinned = os.environ.get("KAKEYA_MAC_PYTHON") + candidates = workload_python_candidates(os.environ) + resolved = resolve_workload_python( + candidates, _can_import_gate_module, pinned=pinned) + pybin = resolved.path if resolved else "python3" + gate_ok = bool(resolved and resolved.gate_module_ok) + print(f"[mac-bridge] workload python={pybin} {GATE_MODULE}_ok={gate_ok} " + f"pinned={pinned!r} candidates={candidates}", file=sys.stderr) + if preset_requires_gate(request.preset.name) and not gate_ok: + print(f"::error::{gate_error_message(request.preset.name, pybin)}", + file=sys.stderr) + return 90 + LOG_DIR.mkdir(parents=True, exist_ok=True) summary = { "preset": request.preset.name, @@ -66,13 +102,19 @@ def main() -> int: "nonce": request.nonce, "commands": [], } + # Make the resolved interpreter authoritative for BOTH bare-``python3`` + # commands (rewritten here) and the launcher (which reads KAKEYA_MAC_PYTHON). + sub_env = dict(os.environ) + sub_env["KAKEYA_MAC_PYTHON"] = pybin rc = 0 for idx, argv in enumerate(commands): + argv = substitute_python(argv, pybin) log_path = LOG_DIR / f"{request.preset.name}-{idx}.log" print(f"[mac-bridge] exec[{idx}]: {argv}", file=sys.stderr) t0 = time.perf_counter() with log_path.open("wb") as log: - proc = subprocess.run(argv, stdout=log, stderr=subprocess.STDOUT) + proc = subprocess.run(argv, stdout=log, stderr=subprocess.STDOUT, + env=sub_env) elapsed = time.perf_counter() - t0 summary["commands"].append({ "argv": argv, diff --git a/scripts/run_kakeya_mac.sh b/scripts/run_kakeya_mac.sh index 4bf3308..8197ad2 100755 --- a/scripts/run_kakeya_mac.sh +++ b/scripts/run_kakeya_mac.sh @@ -55,6 +55,11 @@ done log() { echo "[run-kakeya-mac] $*" >&2; } +# Pinned interpreter (Layer B): prefer KAKEYA_MAC_PYTHON (the venv python with +# mlx_lm/torch/transformers) over a bare python3 that a host reboot may have +# repointed. See docs/skills/pin-selfhosted-runner-python-env-skill.md. +PYBIN="${KAKEYA_MAC_PYTHON:-python3}" + # ---- argv for the full-engine harness chat ---- args=( --verifier-path "$VERIFIER" @@ -80,7 +85,7 @@ log "drafter : $DRAFTER" log "f_theta : $FTHETA" log "params : sink=$SINK window=$WINDOW block=$BLOCK max_new=$MAX_NEW" -cmd=( python3 scripts/research/k3_integrated_niah_eval_mac.py "${args[@]}" "${EXTRA[@]}" ) +cmd=( "$PYBIN" scripts/research/k3_integrated_niah_eval_mac.py "${args[@]}" "${EXTRA[@]}" ) if [[ "$DRY_RUN" == "1" ]]; then echo "PYTHONPATH=.:sdks/python ${cmd[*]}" @@ -88,9 +93,9 @@ if [[ "$DRY_RUN" == "1" ]]; then fi # ---- preflight (Apple Silicon + MLX + model) ---- -command -v python3 >/dev/null || { log "python3 not found"; exit 1; } -python3 -c "import mlx.core" 2>/dev/null \ - || { log "MLX not importable — this needs Apple Silicon + 'pip install mlx mlx-lm'"; exit 2; } +command -v "$PYBIN" >/dev/null 2>&1 || { log "interpreter not found: $PYBIN (set KAKEYA_MAC_PYTHON)"; exit 1; } +"$PYBIN" -c "import mlx.core, mlx_lm" 2>/dev/null \ + || { log "mlx/mlx_lm not importable by $PYBIN — Apple Silicon + a venv with 'mlx mlx-lm'; set KAKEYA_MAC_PYTHON. See docs/skills/pin-selfhosted-runner-python-env-skill.md"; exit 2; } [[ -d "$VERIFIER" ]] \ || { log "verifier model dir not found: $VERIFIER (set KAKEYA_MAC_VERIFIER_PATH)"; exit 3; } if [[ "$FAST" != "1" && ! -e "$FTHETA" ]]; then diff --git a/tests/inference_engine/bridge/test_runner_python.py b/tests/inference_engine/bridge/test_runner_python.py new file mode 100644 index 0000000..8a0a6bb --- /dev/null +++ b/tests/inference_engine/bridge/test_runner_python.py @@ -0,0 +1,108 @@ +"""Unit tests for the mac-bridge workload interpreter pinning (Layers B/C). + +Pure / dependency-injected logic from ``inference_engine.bridge.runner_python``; +the CLI ``scripts/mac_bridge/run_preset.py`` is a thin caller (coverage-exempt). +""" + +from __future__ import annotations + +from inference_engine.bridge.runner_python import ( + GATE_MODULE, + SKILL_DOC, + ResolvedPython, + gate_error_message, + preset_requires_gate, + resolve_workload_python, + substitute_python, + workload_python_candidates, +) + + +# --------------------------------------------------------------------------- # +# workload_python_candidates +# --------------------------------------------------------------------------- # +def test_candidates_prioritise_pin_then_venvs_then_path(): + env = {"KAKEYA_MAC_PYTHON": "/pin/bin/python"} + which = {"python3.13": "/usr/bin/python3.13", "python3": "/usr/bin/python3"}.get + cands = workload_python_candidates( + env, which=which, expanduser=lambda p: p.replace("~", "/home/me")) + assert cands == [ + "/pin/bin/python", + "/home/me/kakeya-venv/bin/python", + "/home/me/.venv/bin/python", + "/usr/bin/python3.13", + "/usr/bin/python3", + ] + + +def test_candidates_drop_empty_and_dedupe(): + # no pin, python3.13 missing, and python3 == an expanded venv path (dedupe). + env: dict = {} + which = {"python3.13": None, "python3": "/home/me/.venv/bin/python"}.get + cands = workload_python_candidates( + env, which=which, expanduser=lambda p: p.replace("~", "/home/me")) + assert cands == [ + "/home/me/kakeya-venv/bin/python", + "/home/me/.venv/bin/python", + ] + assert None not in cands + + +# --------------------------------------------------------------------------- # +# resolve_workload_python +# --------------------------------------------------------------------------- # +def test_resolve_picks_first_importable(): + cands = ["/a/py", "/b/py", "/c/py"] + r = resolve_workload_python(cands, lambda p: p == "/b/py", pinned="/a/py") + assert r == ResolvedPython(path="/b/py", gate_module_ok=True, from_pin=False) + + +def test_resolve_marks_from_pin_when_pinned_is_importable(): + r = resolve_workload_python(["/pin/py", "/x/py"], lambda p: True, + pinned="/pin/py") + assert r.path == "/pin/py" and r.gate_module_ok is True and r.from_pin is True + + +def test_resolve_falls_back_to_first_when_none_importable(): + r = resolve_workload_python(["/a/py", "/b/py"], lambda p: False, + pinned="/a/py") + assert r == ResolvedPython(path="/a/py", gate_module_ok=False, from_pin=True) + + +def test_resolve_returns_none_without_candidates(): + assert resolve_workload_python([], lambda p: True) is None + + +# --------------------------------------------------------------------------- # +# preset_requires_gate +# --------------------------------------------------------------------------- # +def test_gate_required_for_mlx_and_k3_engine_presets(): + assert preset_requires_gate("mlx-kakeya-launcher-full") is True + assert preset_requires_gate("k3-step2-fused") is True + + +def test_gate_skips_diagnostic_and_installer_and_non_engine(): + assert preset_requires_gate("mlx-env-probe") is False # diagnostic + assert preset_requires_gate("mlx-upgrade") is False # installer + assert preset_requires_gate("integration-tests") is False + assert preset_requires_gate("agent-capacity-stress") is False + + +# --------------------------------------------------------------------------- # +# substitute_python / gate_error_message +# --------------------------------------------------------------------------- # +def test_substitute_rewrites_only_leading_bare_python3(): + assert substitute_python(["python3", "a.py", "--x"], "/v/py") == [ + "/v/py", "a.py", "--x"] + # non-python3 argv0 (e.g. the launcher) is untouched. + assert substitute_python(["bash", "run.sh"], "/v/py") == ["bash", "run.sh"] + # empty argv is safe. + assert substitute_python([], "/v/py") == [] + + +def test_gate_error_message_names_module_preset_and_skill(): + msg = gate_error_message("mlx-kakeya-launcher-full", "/usr/bin/python3") + assert GATE_MODULE in msg + assert "mlx-kakeya-launcher-full" in msg + assert "/usr/bin/python3" in msg + assert SKILL_DOC in msg