Skip to content

refactor(e2e): remove reliance on Repository.Status in wait functions#2789

Open
zakisk wants to merge 1 commit into
tektoncd:mainfrom
zakisk:SRVKP-12156-remove-repo-status-update-in-e2e
Open

refactor(e2e): remove reliance on Repository.Status in wait functions#2789
zakisk wants to merge 1 commit into
tektoncd:mainfrom
zakisk:SRVKP-12156-remove-repo-status-update-in-e2e

Conversation

@zakisk

@zakisk zakisk commented Jun 18, 2026

Copy link
Copy Markdown
Member

📝 Description of the Change

The Repository CR's Status field is deprecated and will be removed. This refactors E2E test wait helpers to use PipelineRun objects directly instead of polling Repository.Status:

  • Remove UntilRepositoryUpdated and UntilRepositoryHasStatusReason
  • Remove FailOnRepoCondition from wait.Opts
  • Modify UntilPipelineRunCreated and UntilPipelineRunHasReason to return []PipelineRun so callers can inspect annotations and conditions
  • Add UntilPipelineRunsFinished for mixed-outcome tests (1 pass + 1 fail)
  • Rewrite Succeeded() helper to validate PipelineRun annotations instead of repo.Status fields
  • Migrate all ~40 call sites across 20 test files

🔗 Linked GitHub Issue

Fixes #

🧪 Testing Strategy

  • Unit tests
  • Integration tests
  • End-to-end tests
  • Manual testing
  • Not Applicable

🤖 AI Assistance

AI assistance can be used for various tasks, such as code generation,
documentation, or testing.

Please indicate whether you have used AI assistance
for this PR and provide details if applicable.

  • I have not used any AI assistance for this PR.
  • I have used AI assistance for this PR.

Important

Slop will be simply rejected, if you are using AI assistance you need to make sure you
understand the code generated and that it meets the project's standards. you
need at least know how to run the code and deploy it (if needed). See
startpaac to make it easy
to deploy and test your code changes.

If the majority of the code in this PR was generated by an AI, please add a Co-authored-by trailer to your commit message.
For example:

Co-authored-by: Claude noreply@anthropic.com

✅ Submitter Checklist

  • 📝 My commit messages are clear, informative, and follow the project's How to write a git commit message guide. The Gitlint linter ensures in CI it's properly validated
  • ✨ I have ensured my commit message prefix (e.g., fix:, feat:) matches the "Type of Change" I selected above.
  • ♽ I have run make test and make lint locally to check for and fix any
    issues. For an efficient workflow, I have considered installing
    pre-commit and running pre-commit install to
    automate these checks.
  • 📖 I have added or updated documentation for any user-facing changes.
  • 🧪 I have added sufficient unit tests for my code changes.
  • 🎁 I have added end-to-end tests where feasible. See README for more details.
  • 🔎 I have addressed any CI test flakiness or provided a clear reason to bypass it.
  • If adding a provider feature, I have filled in the following and updated the provider documentation:
    • GitHub App
    • GitHub Webhook
    • Gitea/Forgejo
    • GitLab
    • Bitbucket Cloud
    • Bitbucket Data Center

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the end-to-end tests to monitor and assert on Tekton PipelineRuns directly rather than waiting for Repository status updates. The feedback highlights a recurring issue across multiple test files where the code directly accesses the Conditions slice by index (e.g., Conditions[0]) on a PipelineRun status. Since the order of conditions is not guaranteed, it is highly recommended to use GetCondition(apis.ConditionSucceeded) to safely and reliably retrieve the 'Succeeded' condition.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread test/pkg/wait/wait.go
Comment thread test/github_gitops_commands_prefix_test.go Outdated
Comment thread test/github_pullrequest_test_comment_test.go Outdated
Comment thread test/github_pullrequest_test_comment_test.go Outdated
Comment thread test/gitlab_merge_request_test.go Outdated
Comment thread test/pkg/wait/check.go Outdated
@codecov

codecov Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.01%. Comparing base (27ac5d7) to head (9764163).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2789   +/-   ##
=======================================
  Coverage   61.01%   61.01%           
=======================================
  Files         211      211           
  Lines       17228    17228           
=======================================
  Hits        10511    10511           
  Misses       5917     5917           
  Partials      800      800           
Flag Coverage Δ
unit-tests 61.01% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch 3 times, most recently from c211bca to 51f1574 Compare June 21, 2026 11:29

@chmouel chmouel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Reviews]

The ordering fixes and restoration of the Gitea on-comment golden coverage are the correctness issues that need attention here. I have left the condition-access findings out because Gemini already opened threads for those lines.

One semantic note on TestGitlabMergeRequest: restoring SHA: mr.SHA directly would make the new helper wait for five PipelineRuns with that SHA, unlike the old helper which counted all statuses and then selected a matching SHA. The current unfiltered wait avoids that timeout, but if explicit SHA validation remains part of the test contract it should be preserved separately from the total-count wait.

Please add ordering unit coverage with deliberately reversed creation timestamps. The relevant local checks are go test ./test/pkg/wait and go test -run '^$' -tags=e2e ./test; go build ./test/... does not compile the changed _test.go files.

Comment thread test/pkg/wait/wait.go Outdated
Comment thread test/pkg/wait/wait.go
Comment thread test/gitea_gitops_commands_test.go Outdated
Comment thread test/gitlab_merge_request_test.go Outdated
Comment thread test/pkg/wait/wait_test.go Outdated
Comment thread test/pkg/wait/check.go Outdated
Comment thread test/pkg/wait/wait.go Outdated
@zakisk

zakisk commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

there are many failures and they will be on next pushes as well because our E2E tests were heavily reliant on repo.Status field, working to resolve them locally instead of running here because they're time consuming...

@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch 6 times, most recently from ea511da to b51ebaf Compare June 24, 2026 15:02
@chmouel

chmouel commented Jun 25, 2026

Copy link
Copy Markdown
Member

there are many failures and they will be on next pushes as well because our E2E tests were heavily reliant on repo.Status field, working to resolve them locally instead of running here because they're time consuming...

yeah not a trivial task you took! but this would be great and more robust when we get there

@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch 3 times, most recently from 778c5ae to 3eaa9a0 Compare June 25, 2026 11:20
@theakshaypant theakshaypant marked this pull request as draft June 26, 2026 04:58
@theakshaypant

Copy link
Copy Markdown
Member

Converting this to draft in the meanwhile.

@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch from 3eaa9a0 to 15d198e Compare June 29, 2026 10:12
@zakisk

zakisk commented Jun 29, 2026

Copy link
Copy Markdown
Member Author

TestGithubGHEMaxKeepRuns was failing but now it's passing so it's flake. I am gonna run E2E on this again and again to see frequency of flakiness...

@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch 2 times, most recently from 6b64f1a to b72ae2a Compare June 29, 2026 12:45
@zakisk

zakisk commented Jun 29, 2026

Copy link
Copy Markdown
Member Author

TestGithubGHEMaxKeepRuns was failing but now it's passing so it's flake. I am gonna run E2E on this again and again to see frequency of flakiness...

actually I found the reason for flakiness...

@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch 6 times, most recently from 3dfcc39 to f0cdf32 Compare June 30, 2026 12:01
@zakisk

zakisk commented Jun 30, 2026

Copy link
Copy Markdown
Member Author

bitbucket cloud API issue two time... 😕

{"level":"info","ts":1782822463.6097746,"caller":"bitbucketcloud/bitbucket.go:223","msg":"bitbucket-cloud: initialized client with provided token for user=cboudjna"}
    crd.go:27: assertion failed: error is not nil: 401 Unauthorized

@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch from f0cdf32 to bc62594 Compare July 1, 2026 06:17
@zakisk zakisk marked this pull request as ready for review July 1, 2026 07:03
@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch 5 times, most recently from 3aa2622 to 5b4f71e Compare July 3, 2026 07:32

@theakshaypant theakshaypant left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love that the CI is all green, I did rerun the concurrency tests.
Adding some comments from an AI-assisted review.

Comment thread test/pkg/wait/wait.go Outdated
Comment thread test/pkg/wait/wait.go Outdated
Comment thread test/github_pullrequest_rerequest_test.go Outdated
Comment thread test/pkg/wait/wait.go Outdated
Comment thread test/gitea_test.go Outdated
Comment thread test/pkg/wait/wait.go Outdated
@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch from 5b4f71e to 6a61ecd Compare July 3, 2026 15:11
The Repository CR's Status field is deprecated and will be removed.
This refactors E2E test wait helpers to use PipelineRun objects directly
instead of polling Repository.Status:

- Remove UntilRepositoryUpdated and UntilRepositoryHasStatusReason
- Remove FailOnRepoCondition from wait.Opts
- Modify UntilPipelineRunCreated and UntilPipelineRunHasReason to return
  []PipelineRun so callers can inspect annotations and conditions
- Add UntilPipelineRunsFinished for mixed-outcome tests (1 pass + 1 fail)
- Rewrite Succeeded() helper to validate PipelineRun annotations instead
  of repo.Status fields
- Migrate all ~40 call sites across 20 test files

Signed-off-by: Zaki Shaikh <zashaikh@redhat.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@zakisk zakisk force-pushed the SRVKP-12156-remove-repo-status-update-in-e2e branch from 6a61ecd to 9764163 Compare July 3, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants