Skip to content

OPRUN-4572: Expand OTE docs with more comprehensive details#697

Open
camilamacedo86 wants to merge 1 commit intoopenshift:mainfrom
camilamacedo86:improve-docs
Open

OPRUN-4572: Expand OTE docs with more comprehensive details#697
camilamacedo86 wants to merge 1 commit intoopenshift:mainfrom
camilamacedo86:improve-docs

Conversation

@camilamacedo86
Copy link
Copy Markdown
Contributor

@camilamacedo86 camilamacedo86 commented Apr 15, 2026

Summary by CodeRabbit

  • Documentation
    • Expanded CI-oriented docs explaining end-to-end OLMv1 test execution and automatic discovery across release, presubmit, and periodic jobs.
    • Added design architecture for standard and extended suites with label-driven routing and a breakdown of extended sub-suites.
    • Documented presubmit CI job behavior, example job configuration, ephemeral release handling, environment coverage guidance, a sanity job example, and useful links.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Apr 15, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@camilamacedo86: This pull request explicitly references no jira issue.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 15, 2026

Walkthrough

Expanded openshift/tests-extension/README.md with CI-oriented documentation describing olmv1 test execution, binary/image mapping, automatic suite discovery, environment coverage guidance, design architecture for standard and extended suites, presubmit CI job examples, and a tests-extension sanity job example.

Changes

Cohort / File(s) Summary
Documentation Update
openshift/tests-extension/README.md
Added CI integration workflow ("How it works"), environment coverage guidance (architectures, network modes, topologies), design architecture with standard suites (olmv1/parallel, olmv1/serial, olmv1/slow, olmv1/all) and extended/QE suites (test/qe/...) including label routing rules, replaced E2E Test Configuration with Presubmit CI Jobs (example ci-operator YAML, ephemeral release note), added tests-extension sanity job example, and fixed trailing newline.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 10
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Stable And Deterministic Test Names ✅ Passed Documentation-only change with no Ginkgo test code containing dynamic test names.
Test Structure And Quality ✅ Passed PR exclusively modifies documentation file (README.md); no Ginkgo test code is added or modified.
Microshift Test Compatibility ✅ Passed This PR contains only documentation updates to openshift/tests-extension/README.md with no new Ginkgo e2e test code additions.
Single Node Openshift (Sno) Test Compatibility ✅ Passed This pull request is a documentation-only change that updates the openshift/tests-extension/README.md file without adding any new Ginkgo e2e test code, so the SNO compatibility check does not apply and passes by default.
Topology-Aware Scheduling Compatibility ✅ Passed This PR is a documentation-only change to openshift/tests-extension/README.md with no scheduling constraints introduced.
Ote Binary Stdout Contract ✅ Passed PR contains only documentation changes to openshift/tests-extension/README.md with no Go source code modifications. The existing cmd/main.go contains no uncontrolled stdout calls and complies with the OTE Binary Stdout Contract.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR contains only documentation updates to README.md with code examples for instructional purposes; no new Ginkgo e2e tests are added to the test suite.
Title check ✅ Passed The title directly and clearly summarizes the main change: expanding OTE (OpenShift Test Environment) documentation with comprehensive details.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Apr 15, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: camilamacedo86
Once this PR has been reviewed and has the lgtm label, please assign tmshort for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
openshift/tests-extension/README.md (2)

377-413: ⚠️ Potential issue | 🟡 Minor

Remove duplicate content.

This entire section duplicates content already covered in the "Presubmit CI Jobs" section (lines 304-376). The YAML example at lines 380-400 is nearly identical to the example at lines 316-335, and the explanation about include_built_images: true (lines 407-413) duplicates lines 355-357.

🧹 Proposed fix: remove redundant lines
-Here is a CI job example:
-
-```yaml
-- as: e2e-aws-techpreview-olmv1-ext
-  steps:
-    cluster_profile: aws
-    env:
-      FEATURE_SET: TechPreviewNoUpgrade
-
-      # Only enable 'watch-namespaces' monitor to avoid job failures from other default monitors 
-      # in openshift-tests (like apiserver checks, alert summaries, etc). In this job, the selected 
-      # OLMv1 test passed, but the job failed because a default monitor failed. 
-      #
-      # 'watch-namespaces' is very lightweight and rarely fails, so it's a safe choice.
-      # There is no way to fully disable all monitors, but we can use this option to reduce noise.
-      #
-      # See: ./openshift-tests run --help (option: --monitor)
-      TEST_ARGS: --monitor=watch-namespaces
-
-      TEST_SUITE: olmv1/all
-    test:
-    - ref: openshift-e2e-test
-    workflow: openshift-e2e-aws
-```
-
-This uses the `openshift-tests` binary to run OLMv1 tests against a test OpenShift release.
-
-It works for pull request testing because of this:
-
-```yaml
-releases:
-  latest:
-    integration:
-      include_built_images: true
-```
-
-More info: https://docs.ci.openshift.org/docs/architecture/ci-operator/#testing-with-an-ephemeral-openshift-release
-
 ## Makefile Commands
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@openshift/tests-extension/README.md` around lines 377 - 413, The README
contains a duplicated CI job example and explanation that repeats content from
the "Presubmit CI Jobs" section; remove the redundant block that starts with the
e2e-aws-techpreview-olmv1-ext YAML sample and the follow-up paragraph about
releases/include_built_images to avoid duplication, leaving only the original
example in the "Presubmit CI Jobs" section and preserving surrounding headings
(the duplicate YAML sample and the sentence beginning “This uses the
`openshift-tests` binary…” through the `include_built_images: true` snippet
should be deleted).

107-118: ⚠️ Potential issue | 🟡 Minor

Remove or rename duplicate section heading.

The heading "How to Run the Tests Locally" appears twice (lines 107 and 118). This creates confusion and breaks document navigation. Consider either:

  1. Merging the command table (lines 109-116) into the detailed section starting at line 118, or
  2. Renaming the first heading to "Quick Reference" or "Command Reference"
📝 Suggested fix: rename first heading
-## How to Run the Tests Locally
+## Quick Command Reference

 | Command                                         | Description                                                              |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@openshift/tests-extension/README.md` around lines 107 - 118, The README
contains a duplicate "How to Run the Tests Locally" heading; remove ambiguity by
renaming the first occurrence (the heading immediately above the command table)
to "Quick Reference" or "Command Reference" and keep the detailed "How to Run
the Tests Locally" section as the primary heading, or alternatively merge the
command table under the detailed section; update only the first heading text
(the first "How to Run the Tests Locally") so the command table (the block with
`make build`, `./bin/olmv1-tests-ext info`, `list`, `run-suite`, `run-test`) is
either moved under or referenced by the detailed section to avoid duplicate
headings.
🧹 Nitpick comments (1)
openshift/tests-extension/README.md (1)

75-85: Add language specification to fenced code block.

The fenced code block lacks a language identifier. Add a language specification (e.g., text) to improve rendering and comply with Markdown best practices.

📝 Proposed fix
-```
+```text
 olmv1/extended                        # All Extended tests
 ├── extended/releasegate              # Extended + ReleaseGate (also in standard suites)
 └── extended/candidate                # Extended without ReleaseGate

As per coding guidelines, the markdownlint tool flagged this: "Fenced code blocks should have a language specified (MD040)".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@openshift/tests-extension/README.md` around lines 75 - 85, The fenced code
block showing the test tree in README.md is missing a language identifier;
update the opening fence (the triple backticks before the "olmv1/extended" tree)
to include a language spec such as "text" (e.g., change "```" to "```text") so
markdownlint MD040 is satisfied and the block renders correctly; ensure only the
opening fence is changed and the block content (the tree) is left intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@openshift/tests-extension/README.md`:
- Around line 377-413: The README contains a duplicated CI job example and
explanation that repeats content from the "Presubmit CI Jobs" section; remove
the redundant block that starts with the e2e-aws-techpreview-olmv1-ext YAML
sample and the follow-up paragraph about releases/include_built_images to avoid
duplication, leaving only the original example in the "Presubmit CI Jobs"
section and preserving surrounding headings (the duplicate YAML sample and the
sentence beginning “This uses the `openshift-tests` binary…” through the
`include_built_images: true` snippet should be deleted).
- Around line 107-118: The README contains a duplicate "How to Run the Tests
Locally" heading; remove ambiguity by renaming the first occurrence (the heading
immediately above the command table) to "Quick Reference" or "Command Reference"
and keep the detailed "How to Run the Tests Locally" section as the primary
heading, or alternatively merge the command table under the detailed section;
update only the first heading text (the first "How to Run the Tests Locally") so
the command table (the block with `make build`, `./bin/olmv1-tests-ext info`,
`list`, `run-suite`, `run-test`) is either moved under or referenced by the
detailed section to avoid duplicate headings.

---

Nitpick comments:
In `@openshift/tests-extension/README.md`:
- Around line 75-85: The fenced code block showing the test tree in README.md is
missing a language identifier; update the opening fence (the triple backticks
before the "olmv1/extended" tree) to include a language spec such as "text"
(e.g., change "```" to "```text") so markdownlint MD040 is satisfied and the
block renders correctly; ensure only the opening fence is changed and the block
content (the tree) is left intact.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b4593940-621d-40f2-9b35-0191d3809d26

📥 Commits

Reviewing files that changed from the base of the PR and between a3e5b4f and 84aca0e.

📒 Files selected for processing (1)
  • openshift/tests-extension/README.md

@tmshort
Copy link
Copy Markdown
Contributor

tmshort commented Apr 15, 2026

@camilamacedo86 CodeRabbit has some suggestions for doc updates you might want to look into.

@camilamacedo86
Copy link
Copy Markdown
Contributor Author

/retest-required

@camilamacedo86 camilamacedo86 changed the title NO-ISSUE: Expand OTE docs with more comprehensive details OPRUN-4572: Expand OTE docs with more comprehensive details Apr 16, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Apr 16, 2026

@camilamacedo86: This pull request references OPRUN-4572 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary by CodeRabbit

  • Documentation
  • Expanded CI-oriented docs explaining end-to-end OLMv1 test execution and automatic discovery across release, presubmit, and periodic jobs.
  • Added design architecture for standard and extended suites with label-driven routing and a breakdown of extended sub-suites.
  • Documented presubmit CI job behavior, example job configuration, ephemeral release handling, environment coverage guidance, a sanity job example, and useful links.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@camilamacedo86
Copy link
Copy Markdown
Contributor Author

/test e2e-aws-upgrade-ovn-single-node

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Apr 16, 2026

@camilamacedo86: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants