ci(kubernetes): add kube gateway e2e tests and gated CI workflow#1251
Merged
Conversation
TaylorMutch
requested review from
a team,
derekwaynecarr,
maxamillion and
mrunalp
as code owners
TaylorMutch
force-pushed
the
tmutch/kube-e2e-ci-take2
branch
6 times, most recently
from
d55f224 to
3982a35
Compare
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
Adds a label-gated GitHub Actions workflow that exercises the Helm chart end-to-end against the Rust e2e suite via `mise run e2e:helm`. Pipeline: - pr_metadata gates on the `test:e2e-helm` label via the pr-gate action. - build-gateway / build-supervisor build and push Docker images using the reusable docker-build.yml workflow. - helm-e2e (bare runner): apt-installs z3 build deps so cargo can compile the openshell-policy crate's z3-sys backend, creates a kind cluster via helm/kind-action, materializes the kind kubeconfig at the path mise's [env] block expects, side-loads the freshly built gateway/supervisor images, applies deploy/kube/manifests/agent-sandbox.yaml so the sandboxes.agents.x-k8s.io CRD and reconciling StatefulSet are in place, and finally runs `mise run e2e:helm`. Also expands the `e2e:helm` task to run the full Rust e2e suite (matching `e2e:podman`) instead of only the smoke test, with OPENSHELL_E2E_KUBE_TEST as an opt-in single-test override for local debugging. Extends the e2e-label-help workflow so applying `test:e2e-helm` posts the next-step hint pointing at this workflow. Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
TaylorMutch
force-pushed
the
tmutch/kube-e2e-ci-take2
branch
from
3982a35 to
7c6abc5
Compare
drew
reviewed
May 11, 2026
Six tests previously skipped on the kubernetes driver — three in host_gateway_alias.rs plus the forward_proxy_l7 + graphql_l7 cases — relied on `host.openshell.internal` reaching either a host process or a sibling docker container. The Helm chart already supports this via `server.hostGatewayIP`, but the e2e wrapper never set it. - with-kube-gateway.sh: auto-detect the host-routable IP (CoreDNS `host.k3d.internal` first to handle Docker Desktop, docker network gateway as a fallback for kind on Linux CI) and pass it to helm install. Also import locally-built images into existing k3d clusters and wait for namespace deletion to complete before exit. - e2e/rust/Cargo.toml + e2e-helm.sh: add an `e2e-host-gateway` feature that gates the three test files; docker and podman runs imply it, helm runs opt in by default (overridable via OPENSHELL_E2E_HELM_FEATURES for remote clusters where the test host is unreachable from pods). - Drop the `skip_if_kube` helper and its callers — feature gating now decides whether the tests compile in. Verified against a local k3d cluster: all six previously-skipped tests pass, smoke regression intact.
The runner exercises the gateway on Kubernetes; helm is just the deployment mechanism. Names now describe the target environment. - mise task: `e2e:helm` -> `e2e:kubernetes` - script: e2e/rust/e2e-helm.sh -> e2e/rust/e2e-kubernetes.sh - env var: OPENSHELL_E2E_HELM_FEATURES -> OPENSHELL_E2E_KUBERNETES_FEATURES - workflow: branch-helm-e2e.yml -> branch-kubernetes-e2e.yml (display name "Branch Kubernetes E2E", job "kubernetes-e2e") - PR gate label: test:e2e-helm -> test:e2e-kubernetes (e2e-label-help hint workflow updated to match) PR #1251 needs the new label applied; the old label can be removed.
TaylorMutch
added
test:e2e-kubernetes
TaylorMutch
changed the title
kind dual-stacks its network; the wrapper's awk picked the first non-empty gateway, which was the IPv6 entry (fc00:...). Sandbox pods can't reach the test host's IPv4 listener through that, so the L7 and host-gateway-alias tests failed in CI even though they passed locally against k3d (where host.k3d.internal in CoreDNS short-circuits the docker-network fallback). Restrict the awk filter to IPv4 octets.
Collaborator
Author
|
/ok to test 84b114f |
drew
approved these changes
May 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Kubernetes e2e harness (
mise run e2e:kubernetes) and aBranch Kubernetes E2Eworkflow gated ontest:e2e-kubernetes, so Helm chart and gateway packaging changes can be exercised end-to-end on demand against a real kind cluster.Related Issue
N/A — infrastructure follow-up to the earlier kube gateway e2e work.
Changes
Kube e2e harness
e2e/with-kube-gateway.shwrapper:OPENSHELL_E2E_KUBE_CONTEXTis set, installs the chart into an ephemeral namespace on the existing context (CI path).tasks/scripts/helm-k3s-local.shand tears it down on exit (dev path).gateway/supervisorimages (also for existing k3d clusters), helm-installs withci/values-tls-disabled.yaml, port-forwardssvc/openshell, registers a plaintext gateway, and runs the supplied command withOPENSHELL_E2E_DRIVER=kubernetes.e2e/rust/e2e-kubernetes.shthat buildsopenshell-cliand runs the Rust e2e tests through the wrapper.e2e:kubernetesmise task wired up intasks/test.toml..github/workflows/branch-kubernetes-e2e.yml:pull-request/*push andworkflow_dispatch../.github/actions/pr-gateontest:e2e-kubernetes.gatewayandsupervisorDocker images via the reusabledocker-build.ymlworkflow.helm/kind-action, materializes the kubeconfig at the mise-expected path, side-loads images tagged with${{ github.sha }}, and runsmise run --no-deps --skip-deps e2e:kubernetes..github/workflows/e2e-label-help.ymlto post the next-step hint whentest:e2e-kubernetesis applied.Testing
mise run pre-commitpassestest:e2e-kuberneteslabel is applied so the new Branch Kubernetes E2E workflow runs on this PRChecklist