TRT-2762: Component Readiness: adding variant filter inflates regression count with unrelated results#3714
Conversation
…Readiness Two bugs caused adding a variant filter to inflate regression counts with unrelated results: 1. updateVarsFromView() did not sync test_filters (lifecycles, capabilities) from the view into React state. When "Generate Report" dropped the view param and built URL params from state, the lifecycle filter was lost, causing all lifecycle tests to appear in the sample query and producing hundreds of spurious regressions. 2. replaceIncludeVariantsCheckedItems() and replaceCompareVariantsCheckedItems() mutated the existing state object directly instead of creating a new one, preventing React from detecting the state change and potentially causing stale renders. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Pipeline controller notification For optional jobs, comment This repository is configured in: automatic mode |
|
@openshift-trt-agent[bot]: This pull request references TRT-2762 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: openshift-trt-agent[bot] The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @openshift-trt-agent[bot]. Thanks for your PR. I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/ok-to-test |
|
/test agentic-staging |
|
Scheduling required tests: |
|
/test agentic-staging |
Sippy Staging EnvironmentURL: https://vat-salad-sox-psychiatry.trycloudflare.com This environment is built from this PR and will remain available for approximately 60 minutes (until ~14:10 UTC). |
|
Hm, I don't think this is working. For example this URL: should not even be showing me Seeded data should include some things not selected in the default view as well, so we can better test this fix. |
… seed data The PostgreSQL data provider was not filtering by testCapabilities, causing all capability types (e.g. cap1) to appear even when only specific capabilities (e.g. install) were requested. This adds capability intersection filtering to queryTestStatus(), matching the BigQuery provider's existing behavior. Also adds an azure Platform job and a networking-capability test to seed data so that variant and capability filtering can be tested against data not included in the default view. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@stbenjam Good catch. The root cause was that the PostgreSQL data provider was not filtering by Changes in this push:
Verification:
|
|
Scheduling required tests: |
|
@openshift-trt-agent[bot]: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Summary
Fixes TRT-2762: Adding a variant filter in Component Readiness inflated the regression count with unrelated results (from ~86 to 400+).
Two bugs were identified and fixed in
sippy-ng/src/component_readiness/CompReadyVars.js:Missing test_filters sync in
updateVarsFromView(): When a view was selected,updateVarsFromView()synced variant options and advanced options into React state but skippedtest_filters(lifecycles, capabilities). When the user clicked "Generate Report", the view URL param was dropped and explicit params were built from React state. Sincelifecycleswas never synced to state, the filter was lost. Thelifecyclesfilter is applied only to the sample query (not the base), so losing it caused all lifecycle tests to appear in the sample, producing hundreds of spurious regressions. This fix adds syncing oftest_filters.lifecyclesandtest_filters.capabilitiesfrom the view config into React state.State mutation bug in variant replace functions:
replaceIncludeVariantsCheckedItems()andreplaceCompareVariantsCheckedItems()mutated the existing state object directly (includeVariantsCheckedItems[variant] = checkedItems) then called the setter with the same object reference. React relies on reference equality to detect state changes, so this mutation was invisible to React and could cause stale renders. This fix creates a new object with the spread operator before calling the setter.Test plan
make lintpassesmake testpasses (Go + Jest)TestRegressionCacheLoader) is a pre-existing BigQuery credentials issue unrelated to this changeupdateVarsFromViewnow syncs all filter categories (variant_options, test_filters, advanced_options)Generated with Claude Code