Skip to content

fix: report clear ownership conflict for unmanaged same-name Deployment/DaemonSet in GatewayNamespace mode#9470

Open
Aias00 wants to merge 4 commits into
envoyproxy:mainfrom
Aias00:fix-issue-9132
Open

fix: report clear ownership conflict for unmanaged same-name Deployment/DaemonSet in GatewayNamespace mode#9470
Aias00 wants to merge 4 commits into
envoyproxy:mainfrom
Aias00:fix-issue-9132

Conversation

@Aias00

@Aias00 Aias00 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

In GatewayNamespace mode, when a pre-existing, unmanaged Deployment or DaemonSet shares the Gateway's name (e.g. an unrelated nginx Deployment, as in #9132), the reconcile surfaced a misleading error:

an illegal change in a custom label of EnvoyProxy is detected when updating default/nginx ...

This is really a same-name resource conflict, not a custom-label migration. It happened because createOrUpdateDeployment/createOrUpdateDaemonSet ran the selector-migration workaround before the ownership check: an unmanaged same-name resource with a different selector hit the selector-migration error and never reached the ownership guard that #9215 added for SA/ConfigMap/Service/PDB/HPA.

This PR moves checkOwnership to run right after the existing Get succeeds and before the selector-migration block, in both the Deployment and DaemonSet paths. Now an unmanaged same-name collision returns a clear error (Deployment ns1/gateway-1 already exists and is not owned by this Gateway) and the pre-existing resource is left untouched.

Non-GatewayNamespace mode is unaffected — checkOwnership returns nil immediately there — so the legitimate custom-label migration error path (and its existing tests) is preserved.

Which issue(s) this PR fixes:
Fixes #9132


PR Checklist

  • Authorship & ownership: Coding agents / AI assistants are welcome, but I have reviewed every change, understand how and why it works, can explain and maintain it, and take full responsibility for this PR. I have not submitted generated output I do not understand.
  • DCO: All commits are signed off (git commit -s). See DCO: Sign your work.
  • API agreed first: N/A: no API changes.
  • Required checks pass: make generate gen-check, make lint, and the unit-test/coverage build pass. (Flaky e2e failures are not considered breakages, but gen-check, lint, and coverage MUST pass.) — unit tests pass locally (go test ./internal/infrastructure/kubernetes/...); gen-check/lint/coverage to be confirmed by CI.
  • Tests added/updated: Added 3 unit tests covering the full reconcile path (Deployment + DaemonSet unmanaged-collision, plus a positive EG-owned reconcile), and a Deployment subtest in the GatewayNamespaceOwnership e2e test. The two collision tests were verified to FAIL on the unfixed code (they reproduce the exact misleading error) and PASS with the fix; all existing selector-migration tests still pass.
  • Docs: N/A: no user-facing behavior change beyond a clearer error message.
  • Release notes: Added release-notes/current/bug_fixes/9132-gateway-namespace-unmanaged-resource-conflict.md.
  • Generated files committed: N/A: no API/helm/module changes.
  • Scope & compatibility: Reasonably scoped; no breaking changes — only the error message for the unmanaged-collision case changes, and unmanaged resources are no longer touched.
  • Codex review: Requested a Codex review and addressed all of its comments.
  • Copilot review: Requested a Copilot review and addressed all of its comments.

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings July 10, 2026 04:31
@Aias00 Aias00 requested a review from a team as a code owner July 10, 2026 04:31
@netlify

netlify Bot commented Jul 10, 2026

Copy link
Copy Markdown

Deploy Preview for cerulean-figolla-1f9435 ready!

Name Link
🔨 Latest commit 5c6c1bb
🔍 Latest deploy log https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/6a564b3860b4690008219646
😎 Deploy Preview https://deploy-preview-9470--cerulean-figolla-1f9435.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a confusing reconcile failure in GatewayNamespace mode where a pre-existing, unmanaged Deployment/DaemonSet with the same name as the Gateway could trigger the selector-migration (“illegal change in a custom label”) error before the ownership guard ran. The ownership check is now evaluated earlier so same-name collisions surface a clear ownership-conflict error and the pre-existing resource is left untouched.

Changes:

  • Run checkOwnership immediately after a successful Get (and before selector-migration logic) for Deployments and DaemonSets.
  • Add unit regression coverage for unmanaged same-name collisions (Deployment + DaemonSet) and a positive “EG-owned reconcile still works” case.
  • Add an e2e subtest for a pre-existing unmanaged Deployment name collision, plus a release note fragment.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
internal/infrastructure/kubernetes/infra_resource.go Moves the ownership guard earlier in Deployment/DaemonSet reconcile paths to return a clear ownership-conflict error before selector-migration logic.
internal/infrastructure/kubernetes/infra_resource_test.go Adds regression unit tests for unmanaged same-name Deployment/DaemonSet collisions and validates EG-owned reconcile still succeeds.
test/e2e/tests/gateway_namespace_ownership.go Adds e2e coverage ensuring a pre-existing unmanaged Deployment is not adopted or garbage-collected in GatewayNamespace mode.
release-notes/current/bug_fixes/9132-gateway-namespace-unmanaged-resource-conflict.md Documents the bug fix and the improved error surfacing for unmanaged same-name infra resources.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +180 to +182
Spec: appsv1.DeploymentSpec{
Selector: &metav1.LabelSelector{MatchLabels: map[string]string{"app": "pre-existing"}},
Template: corev1.PodTemplateSpec{

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — done in d2da3f1: the pre-existing unmanaged Deployment now sets replicas=0 so it stays inert (no scheduling/image-pull). The ownership-collision behavior under test needs no Pods to run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2be9ab1f36

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +278 to +280
if err := i.checkOwnership(ctx, daemonSet); err != nil {
return err
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use an unfiltered read for DaemonSet ownership conflicts

When GatewayNamespace mode is running an EnvoyProxy as a DaemonSet and a same-name unmanaged DaemonSet lacks EG's managed labels, the production manager cache will not contain it because DaemonSets are registered with the managedSelector label filter in internal/provider/kubernetes/kubernetes.go. The earlier i.Client.Get then returns NotFound and this newly added ownership check is never reached, so applyIfOwned proceeds to server-side apply and users still get an API/immutability error or possible adoption instead of the promised ownership-conflict error. The new unit test uses an unfiltered fake client, so it does not cover the production cache behavior.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent catch — this was a real gap. The production cache filters DaemonSets (and HPA/PDB) by the envoy managed-label selector (internal/provider/kubernetes/kubernetes.go), so an unmanaged same-name DaemonSet is absent from the cache and the cached Get returned NotFound before the guard ran. The earlier unit tests used an unfiltered fake client and missed it.

Fixed in d2da3f1: the uncached API reader (mgr.GetAPIReader()) is now wired through KubernetesClientHolderProvider.GetAPIReaderInfra.apiReader, and checkOwnership uses it for the existence read (falling back to the cached client when no API reader is configured, e.g. in unit tests). This makes the ownership-conflict detection reliable for all resource kinds, matching the intent #9215 claimed.

Added TestCheckOwnership_UsesAPIReaderWhenCacheMisses: the cached Get returns NotFound (simulating the filtered cache) while the API reader sees the unmanaged resource; it asserts the ownership error is surfaced. Verified the test FAILs on the pre-API-reader code. Note this also closes the same latent gap for PDB/HPA from #9215.

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 61.53846% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.48%. Comparing base (10cc9d2) to head (5c6c1bb).

Files with missing lines Patch % Lines
internal/envoygateway/config/config.go 28.57% 4 Missing and 1 partial ⚠️
internal/provider/kubernetes/kubernetes.go 0.00% 4 Missing ⚠️
internal/provider/runner/runner.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9470      +/-   ##
==========================================
+ Coverage   75.47%   75.48%   +0.01%     
==========================================
  Files         252      252              
  Lines       41662    41687      +25     
==========================================
+ Hits        31444    31468      +24     
- Misses       8088     8091       +3     
+ Partials     2130     2128       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Aias00 added a commit to Aias00/gateway that referenced this pull request Jul 10, 2026
…yNamespace mode

Address review feedback on envoyproxy#9470 (Codex P2 + Copilot).

The production controller cache filters DaemonSets (and HPA/PDB) by the envoy
managed-label selector, so an unmanaged same-name DaemonSet is absent from the
cache: checkOwnership's cached Get returned NotFound and the ownership guard
never ran, so applyIfOwned proceeded to server-side apply and users still got
an API/immutability error or adoption instead of the ownership-conflict error.
The earlier unit tests used an unfiltered fake client and so missed this.

Wire the uncached API reader (mgr.GetAPIReader()) through
KubernetesClientHolder -> Provider.GetAPIReader -> Infra.apiReader, and use it
for the existence read in checkOwnership (falling back to the cached client
when no API reader is configured, e.g. in unit tests). This makes the
ownership-conflict detection reliable for all resource kinds, including the
label-filtered ones, matching the intent claimed by envoyproxy#9215.

Also set replicas=0 on the e2e pre-existing unmanaged Deployment to keep it
inert (no scheduling/image-pull), per Copilot review — the collision behavior
under test needs no Pods to run.

Tests:
- TestCheckOwnership_UsesAPIReaderWhenCacheMisses: cached Get returns NotFound
  (simulating the filtered cache) while the API reader sees the unmanaged
  resource; asserts the ownership error is surfaced. Verified to FAIL on the
  pre-API-reader code.
- TestCheckOwnership_FallsBackToCachedClientWhenNoAPIReader: preserves the
  cached-client path used by the existing tests.
- Existing unit + e2e tests still pass.

Signed-off-by: liuhy <liuhongyu@apache.org>
Co-Authored-By: Claude <noreply@anthropic.com>
@Aias00

Aias00 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Review feedback addressed (d2da3f1)

Thanks @copilot and @codex for the review. Both points addressed:

Codex P2 — use an unfiltered read for DaemonSet ownership conflicts (real bug)
The production controller cache filters DaemonSets/HPA/PDB by the envoy managed-label selector, so an unmanaged same-name DaemonSet was missing from the cache → checkOwnership's cached Get returned NotFound → the guard never ran → applyIfOwned still adopted/server-side-applied. The unit tests used an unfiltered fake client so they missed it.

Fix: wire the uncached API reader (mgr.GetAPIReader()) through KubernetesClientHolderProvider.GetAPIReaderInfra.apiReader, and use it for the existence read in checkOwnership (falling back to the cached client when unset, e.g. in unit tests). This makes conflict detection reliable for all resource kinds — including the label-filtered ones — and also closes the same latent gap for PDB/HPA inherited from #9215. New test TestCheckOwnership_UsesAPIReaderWhenCacheMisses covers the filtered-cache scenario (verified to FAIL on the pre-fix code).

Copilot — keep the e2e Deployment inert
The pre-existing unmanaged Deployment now sets replicas=0 so it schedules/pulls nothing; the collision behavior under test needs no Pods.

Checks

  • go build ./..., go vet, gofmt: clean
  • make kube-generate: no generated-file changes (no API/CRD changes)
  • All affected unit tests pass: ./internal/infrastructure/..., ./internal/envoygateway/config/..., ./internal/provider/runner/...
  • e2e builds with -tags=e2e

Note on the 3 failing e2e jobs from the first run

They failed on TestE2E/BackendTLSSettings with etcdserver: request timed out / context deadline exceeded during teardown — an infra/etcd timeout on the ipv4/default matrix cells, which don't run the GatewayNamespaceOwnership test. The gateway-namespace-mode e2e job (the only one that runs my new Deployment subtest) was green, and the subtest passed (PASS GatewayNamespaceOwnership/Deployment). CI is re-running on this new commit.

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

@zirain

zirain commented Jul 10, 2026

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: d2da3f16e6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Aias00 and others added 3 commits July 14, 2026 18:00
…nt/DaemonSet in GatewayNamespace mode

In GatewayNamespace mode, when a pre-existing, unmanaged Deployment or DaemonSet
shares the Gateway's name (e.g. an unrelated nginx Deployment), the reconcile
surfaced a misleading "illegal change in a custom label of EnvoyProxy" error
instead of a clear ownership-conflict error.

This happened because createOrUpdateDeployment/createOrUpdateDaemonSet ran the
selector-migration workaround before the ownership check: an unmanaged same-name
resource with a different selector hit the selector-migration error and never
reached the ownership guard added by envoyproxy#9215.

Move checkOwnership to run right after the existing Get succeeds and before the
selector-migration block, so unmanaged collisions return "already exists and is
not owned by this Gateway" and the pre-existing resource is left untouched.
Non-GatewayNamespace mode is unaffected (checkOwnership returns nil immediately),
so the legitimate custom-label migration error path is preserved.

Fixes envoyproxy#9132

Signed-off-by: liuhy <liuhongyu@apache.org>
Co-Authored-By: Claude <noreply@anthropic.com>
…yNamespace mode

Address review feedback on envoyproxy#9470 (Codex P2 + Copilot).

The production controller cache filters DaemonSets (and HPA/PDB) by the envoy
managed-label selector, so an unmanaged same-name DaemonSet is absent from the
cache: checkOwnership's cached Get returned NotFound and the ownership guard
never ran, so applyIfOwned proceeded to server-side apply and users still got
an API/immutability error or adoption instead of the ownership-conflict error.
The earlier unit tests used an unfiltered fake client and so missed this.

Wire the uncached API reader (mgr.GetAPIReader()) through
KubernetesClientHolder -> Provider.GetAPIReader -> Infra.apiReader, and use it
for the existence read in checkOwnership (falling back to the cached client
when no API reader is configured, e.g. in unit tests). This makes the
ownership-conflict detection reliable for all resource kinds, including the
label-filtered ones, matching the intent claimed by envoyproxy#9215.

Also set replicas=0 on the e2e pre-existing unmanaged Deployment to keep it
inert (no scheduling/image-pull), per Copilot review — the collision behavior
under test needs no Pods to run.

Tests:
- TestCheckOwnership_UsesAPIReaderWhenCacheMisses: cached Get returns NotFound
  (simulating the filtered cache) while the API reader sees the unmanaged
  resource; asserts the ownership error is surfaced. Verified to FAIL on the
  pre-API-reader code.
- TestCheckOwnership_FallsBackToCachedClientWhenNoAPIReader: preserves the
  cached-client path used by the existing tests.
- Existing unit + e2e tests still pass.

Signed-off-by: liuhy <liuhongyu@apache.org>
Co-Authored-By: Claude <noreply@anthropic.com>
CI lint failed on gateway_namespace_ownership.go:183 because
forbidigo (tools/linter/golangci-lint/.golangci.yml) forbids
ptr.To from k8s.io/utils/ptr in favor of the new() builtin.

ptr.To[int32](0) and new(int32) are equivalent (both yield a
*int32 to zero), so switch to new(int32) and drop the now-unused
import. The APIReader cache-miss fix is unchanged.

Signed-off-by: liuhy <liuhongyu@apache.org>
Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants