Skip to content

test: deflake RateLimitGlobalMergeTest/shared_no_client_selectors#9427

Open
mmmommm wants to merge 1 commit into
envoyproxy:mainfrom
mmmommm:fix-flaky-ratelimit-global-merge-test
Open

test: deflake RateLimitGlobalMergeTest/shared_no_client_selectors#9427
mmmommm wants to merge 1 commit into
envoyproxy:mainfrom
mmmommm:fix-flaky-ratelimit-global-merge-test

Conversation

@mmmommm

@mmmommm mmmommm commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

fix the flaky ratelimit e2e test.
ran the e2e test 50 times in the local kind cluster and all passed.

$ make create-cluster
$ kubectl config current-context
$ make kube-install-image
$ make kube-deploy
$ make install-ratelimit
$ make kube-install-examples-image
$ make e2e-prepare
$ make setup-mac-net-connect
$ fails=0
 for i in $(seq 1 50); do
 kubectl exec -n redis-system deploy/redis -- redis-cli FLUSHALL >/dev/null
 if go test -tags e2e -timeout 20m -run TestE2E ./e2e --gateway-class=envoy-gateway --cleanup-base-resources=false --run-test RateLimitGlobalMergeTest >"$TMPDIR/rl_run_$i.log" 2>&1; then
 echo "run $i: PASS"
 else
 echo "run $i: FAIL -> $TMPDIR/rl_run_$i.log"
 fails=$((fails+1))
 fi
 done
 echo "TOTAL FAILURES: $fails / 50"
run 1: PASS
run 2: PASS
run 3: PASS
run 4: PASS
run 5: PASS
run 6: PASS
run 7: PASS
run 8: PASS
run 9: PASS
run 10: PASS
E0707 10:29:46.185755   30011 websocket.go:296] Unknown stream id 1, discarding message
run 11: PASS
run 12: PASS
run 13: PASS
run 14: PASS
run 15: PASS
run 16: PASS
run 17: PASS
run 18: PASS
run 19: PASS
run 20: PASS
run 21: PASS
run 22: PASS
run 23: PASS
run 24: PASS
run 25: PASS
run 26: PASS
run 27: PASS
run 28: PASS
E0707 10:34:06.671796   50172 websocket.go:296] Unknown stream id 1, discarding message
run 29: PASS
run 30: PASS
run 31: PASS
run 32: PASS
run 33: PASS
run 34: PASS
run 35: PASS
run 36: PASS
run 37: PASS
run 38: PASS
run 39: PASS
run 40: PASS
run 41: PASS
run 42: PASS
run 43: PASS
run 44: PASS
run 45: PASS
run 46: PASS
run 47: PASS
run 48: PASS
run 49: PASS
run 50: PASS
TOTAL FAILURES: 0 / 50

Which issue(s) this PR fixes:

Fixes #8113


PR Checklist

  • Authorship & ownership: Coding agents / AI assistants are welcome, but I have reviewed every change, understand how and why it works, can explain and maintain it, and take full responsibility for this PR. I have not submitted generated output I do not understand.
  • DCO: All commits are signed off (git commit -s). See DCO: Sign your work.
  • API agreed first: If this PR contains API changes (changes under /api), the API was discussed and agreed before the implementation. The API change can be in a separate PR, or in the same PR, but the API must be agreed before implementation. N/A if this PR does not contain API changes.
  • Required checks pass: make generate gen-check, make lint, and the unit-test/coverage build pass. (Flaky e2e failures are not considered breakages, but gen-check, lint, and coverage MUST pass.)
  • Tests added/updated: New/changed code is covered by appropriate tests. N/A if this PR does not contain code changes.
  • Docs: User-facing changes update the docs, either in this PR or a follow-up PR. N/A if this PR does not contain user-facing changes.
  • Release notes: For any non-trivial change, added a release-note fragment under release-notes/current/<section>/<pr-number>-<slug>.md (see release-notes/current/README.md for sections and naming). N/A if this PR does not contain non-trivial changes.
  • Generated files committed: Ran make gen-check and committed the result if API/helm charts/modules changed.
  • Scope & compatibility: The PR is reasonably scoped (no unrelated changes) and preserves backward compatibility, or any breaking change is called out above and documented in release-notes/current/breaking_changes/.
  • Codex review: Requested a Codex review and addressed all of its comments.
  • Copilot review: Requested a Copilot review and addressed all of its comments.

@mmmommm mmmommm requested a review from a team as a code owner July 7, 2026 02:25
@netlify

netlify Bot commented Jul 7, 2026

Copy link
Copy Markdown

Deploy Preview for cerulean-figolla-1f9435 ready!

Name Link
🔨 Latest commit 8eb766e
🔍 Latest deploy log https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/6a5606aef1a2c60008eb8f4d
😎 Deploy Preview https://deploy-preview-9427--cerulean-figolla-1f9435.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@mmmommm mmmommm changed the title fix ratelimit flaky tests test: deflake RateLimitGlobalMergeTest/shared_no_client_selectors Jul 7, 2026
var sent int
require.Eventually(t, func() bool {
cReq, cRes, err := suite.RoundTripper.CaptureRoundTrip(expectOkReq)
sent++

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root cause was that the test hard-coded the exact request at which a 429 is expected (send N, then assert the next is 429).
The fix send requests until a 429 is actually observed instead of pinning the count. I also added per-request logging so a timeout stays diagnosable now that we no longer check an exact count.

@mmmommm

mmmommm commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 34a84022e8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.45%. Comparing base (c9e6a44) to head (8eb766e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9427      +/-   ##
==========================================
+ Coverage   75.41%   75.45%   +0.04%     
==========================================
  Files         252      252              
  Lines       41660    41660              
==========================================
+ Hits        31416    31435      +19     
+ Misses       8115     8098      -17     
+ Partials     2129     2127       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Kise Ryota <kiseryota.contact@gmail.com>
@zirain zirain force-pushed the fix-flaky-ratelimit-global-merge-test branch from 34a8402 to 8eb766e Compare July 14, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TestE2E/RateLimitGlobalMergeTest/shared_no_client_selectors is flaky

1 participant