Skip to content

TRT-2762: Component Readiness: adding variant filter inflates regression count with unrelated results#3714

Open
openshift-trt-agent[bot] wants to merge 2 commits into
openshift:mainfrom
openshift-trt:trt-2762
Open

TRT-2762: Component Readiness: adding variant filter inflates regression count with unrelated results#3714
openshift-trt-agent[bot] wants to merge 2 commits into
openshift:mainfrom
openshift-trt:trt-2762

Conversation

@openshift-trt-agent

Copy link
Copy Markdown

Summary

Fixes TRT-2762: Adding a variant filter in Component Readiness inflated the regression count with unrelated results (from ~86 to 400+).

Two bugs were identified and fixed in sippy-ng/src/component_readiness/CompReadyVars.js:

  1. Missing test_filters sync in updateVarsFromView(): When a view was selected, updateVarsFromView() synced variant options and advanced options into React state but skipped test_filters (lifecycles, capabilities). When the user clicked "Generate Report", the view URL param was dropped and explicit params were built from React state. Since lifecycles was never synced to state, the filter was lost. The lifecycles filter is applied only to the sample query (not the base), so losing it caused all lifecycle tests to appear in the sample, producing hundreds of spurious regressions. This fix adds syncing of test_filters.lifecycles and test_filters.capabilities from the view config into React state.

  2. State mutation bug in variant replace functions: replaceIncludeVariantsCheckedItems() and replaceCompareVariantsCheckedItems() mutated the existing state object directly (includeVariantsCheckedItems[variant] = checkedItems) then called the setter with the same object reference. React relies on reference equality to detect state changes, so this mutation was invisible to React and could cause stale renders. This fix creates a new object with the spread operator before calling the setter.

Test plan

  • make lint passes
  • make test passes (Go + Jest)
  • E2e tests: 107/108 passed; the single failure (TestRegressionCacheLoader) is a pre-existing BigQuery credentials issue unrelated to this change
  • Verified in browser with seed data: loading a view and clicking "Generate Report" preserves the regression count (5 regressed components before and after)
  • Code review: confirmed that updateVarsFromView now syncs all filter categories (variant_options, test_filters, advanced_options)
  • Code review: confirmed both variant replace functions now create new state objects

Generated with Claude Code

…Readiness

Two bugs caused adding a variant filter to inflate regression counts
with unrelated results:

1. updateVarsFromView() did not sync test_filters (lifecycles,
   capabilities) from the view into React state. When "Generate Report"
   dropped the view param and built URL params from state, the lifecycle
   filter was lost, causing all lifecycle tests to appear in the sample
   query and producing hundreds of spurious regressions.

2. replaceIncludeVariantsCheckedItems() and
   replaceCompareVariantsCheckedItems() mutated the existing state
   object directly instead of creating a new one, preventing React from
   detecting the state change and potentially causing stale renders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 29, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 29, 2026

Copy link
Copy Markdown

@openshift-trt-agent[bot]: This pull request references TRT-2762 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

Fixes TRT-2762: Adding a variant filter in Component Readiness inflated the regression count with unrelated results (from ~86 to 400+).

Two bugs were identified and fixed in sippy-ng/src/component_readiness/CompReadyVars.js:

  1. Missing test_filters sync in updateVarsFromView(): When a view was selected, updateVarsFromView() synced variant options and advanced options into React state but skipped test_filters (lifecycles, capabilities). When the user clicked "Generate Report", the view URL param was dropped and explicit params were built from React state. Since lifecycles was never synced to state, the filter was lost. The lifecycles filter is applied only to the sample query (not the base), so losing it caused all lifecycle tests to appear in the sample, producing hundreds of spurious regressions. This fix adds syncing of test_filters.lifecycles and test_filters.capabilities from the view config into React state.

  2. State mutation bug in variant replace functions: replaceIncludeVariantsCheckedItems() and replaceCompareVariantsCheckedItems() mutated the existing state object directly (includeVariantsCheckedItems[variant] = checkedItems) then called the setter with the same object reference. React relies on reference equality to detect state changes, so this mutation was invisible to React and could cause stale renders. This fix creates a new object with the spread operator before calling the setter.

Test plan

  • make lint passes
  • make test passes (Go + Jest)
  • E2e tests: 107/108 passed; the single failure (TestRegressionCacheLoader) is a pre-existing BigQuery credentials issue unrelated to this change
  • Verified in browser with seed data: loading a view and clicking "Generate Report" preserves the regression count (5 regressed components before and after)
  • Code review: confirmed that updateVarsFromView now syncs all filter categories (variant_options, test_filters, advanced_options)
  • Code review: confirmed both variant replace functions now create new state objects

Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Jun 29, 2026
@openshift-ci openshift-ci Bot requested review from petr-muller and smg247 June 29, 2026 11:31
@openshift-ci

openshift-ci Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: openshift-trt-agent[bot]
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 29, 2026
@openshift-ci

openshift-ci Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Hi @openshift-trt-agent[bot]. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@stbenjam

Copy link
Copy Markdown
Member

/ok-to-test

@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 29, 2026
@stbenjam

Copy link
Copy Markdown
Member

/test agentic-staging

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

@stbenjam

Copy link
Copy Markdown
Member

/test agentic-staging

@openshift-trt-agent

Copy link
Copy Markdown
Author

Sippy Staging Environment

URL: https://vat-salad-sox-psychiatry.trycloudflare.com

This environment is built from this PR and will remain available for approximately 60 minutes (until ~14:10 UTC).

@stbenjam

Copy link
Copy Markdown
Member

… seed data

The PostgreSQL data provider was not filtering by testCapabilities,
causing all capability types (e.g. cap1) to appear even when only
specific capabilities (e.g. install) were requested. This adds
capability intersection filtering to queryTestStatus(), matching the
BigQuery provider's existing behavior.

Also adds an azure Platform job and a networking-capability test to seed
data so that variant and capability filtering can be tested against data
not included in the default view.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@openshift-trt-agent

Copy link
Copy Markdown
Author

@stbenjam Good catch. The root cause was that the PostgreSQL data provider was not filtering by testCapabilities at all -- the BigQuery provider had the correct filtering, but the PG provider (used with seed data) was missing it entirely.

Changes in this push:

  1. PostgreSQL capability filtering (provider.go): Added capability intersection filtering to queryTestStatus(). When testCapabilities is specified in the request, only tests whose capabilities overlap with the requested set are included. This matches the BigQuery provider's existing behavior.

  2. Seed data improvements (seed_data.go): Added a Platform:azure job and a networking-capability test that are NOT included in the default seed view (Platform:aws,gcp only). The ExtremeRegression test also runs on azure now. This means:

    • Default view correctly excludes azure results (variant filtering testable)
    • Adding testCapabilities=install correctly excludes cap1/networking results (capability filtering testable)

Verification:

  • Default view: 5 regressed components, 13 total
  • With testCapabilities=install: 0 regressed components, 1 component (comp-Install only, no cap1)
  • With Platform:azure added to include_variants: azure results appear correctly
  • E2e: 107/108 passed (1 pre-existing BigQuery credentials failure)

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

@openshift-ci

openshift-ci Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

@openshift-trt-agent[bot]: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/agentic-staging 4bc03a1 link false /test agentic-staging
ci/prow/e2e 54c1db7 link true /test e2e

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants