feat: Adding e2e hook tests#990
Open
pen-pal wants to merge 12 commits into
Open
Conversation
pen-pal
requested review from
MaxymVlasov,
antonbabenko and
yermulnik
as code owners
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds end-to-end test infrastructure: a bash test runner, README, GitHub Actions workflow, and two Terraform-focused e2e cases with inputs, expected outputs, and requirements files. ChangesEnd-to-End Test Infrastructure
Sequence Diagram(s)sequenceDiagram
participant Workflow
participant RunScript as tests/e2e/run_e2e_tests.sh
participant PreCommit as pre-commit
participant Git as git diff
Workflow->>RunScript: run tests in container
RunScript->>PreCommit: pre-commit run --all-files
PreCommit-->>RunScript: exit code and file changes
RunScript->>Git: compare produced tree to expected/
Git-->>RunScript: diff result
RunScript-->>Workflow: PASS / FAIL summary
Estimated code review effort: 3 (Moderate) | ~25 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/e2e-tests.yaml:
- Around line 31-33: The workflow uses env.IMAGE set to
ghcr.io/antonbabenko/pre-commit-terraform:latest which allows image drift;
change the IMAGE value to a digest-pinned reference
(ghcr.io/antonbabenko/pre-commit-terraform@sha256:<digest>) so the e2e runner
always pulls an immutable artifact, updating any references to IMAGE used in the
workflow (the env block and any docker pull/docker run invocations); obtain the
exact sha256 digest for the desired image tag via a docker registry query or
docker pull + docker inspect (or GHCR API) and replace :latest with
`@sha256`:<digest>.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 0f0b0b52-8698-431c-922f-5536c499b88a
📒 Files selected for processing (18)
.github/workflows/e2e-tests.yamltests/e2e/README.mdtests/e2e/cases/terraform_fmt/reformats-misaligned-hcl/.pre-commit-config.yamltests/e2e/cases/terraform_fmt/reformats-misaligned-hcl/expected/main.tftests/e2e/cases/terraform_fmt/reformats-misaligned-hcl/expected_returncodetests/e2e/cases/terraform_fmt/reformats-misaligned-hcl/input/main.tftests/e2e/cases/terraform_fmt/reformats-misaligned-hcl/requirestests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/.pre-commit-config.yamltests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/expected/main.tftests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/expected/versions.tftests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/expected/wrappers/README.mdtests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/expected/wrappers/main.tftests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/expected/wrappers/outputs.tftests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/expected/wrappers/variables.tftests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/expected/wrappers/versions.tftests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/input/main.tftests/e2e/cases/terraform_wrapper_module_for_each/generates-wrapper-for-root-module/input/versions.tftests/e2e/run_e2e_tests.sh
yermulnik
reviewed
May 28, 2026
yermulnik
left a comment
yermulnik
left a comment
Collaborator
There was a problem hiding this comment.
Thank you for the contribution.
I'm out of context around tests and therefore I've added solely tech comments around Bash code.
pen-pal
added a commit
to pen-pal/pre-commit-terraform
that referenced
this pull request
May 29, 2026
Per review on antonbabenko#990: - replace the `RETURN` trap with explicit cleanup at the function's single exit (the `RETURN` trap is Bash-only and confused reviewers); - read the expected return code with `$(< file)` instead of `$(cat file)`; - quote the `true`/`false` flag values to avoid confusion with the builtins; - use `&>` instead of `> ... 2>&1`; - drop the redundant trailing slash on the copy destination; - print the summary with `printf` instead of a loop. No behavior change; the suite still passes natively and inside the image. Signed-off-by: penpal <unameme@proton.me>
pen-pal
added a commit
to pen-pal/pre-commit-terraform
that referenced
this pull request
May 29, 2026
Per review on antonbabenko#990, pin `ghcr.io/antonbabenko/pre-commit-terraform` by `sha256` digest instead of the floating `:latest` tag, so the bundled toolchain can't drift between runs. A Renovate annotation keeps the digest updated. Signed-off-by: penpal <unameme@proton.me>
yermulnik
reviewed
May 29, 2026
yermulnik
left a comment
yermulnik
left a comment
Collaborator
There was a problem hiding this comment.
@MaxymVlasov Shell code looks good to me, though I'm not really good with the rest of the tests logic — could you please have a look when you've got a moment? Thank you.
Introduce `tests/e2e/run_e2e_tests.sh`, a behavioral test runner that exercises each hook the way a user would -- through `pre-commit` against fixture files -- and compares the result against a committed golden tree. It auto-discovers `cases/<hook>/<case>/`, builds a temp git repo from `input/`, renders the case's `.pre-commit-config.yaml` (`__PCT_REPO__` -> checkout root), runs the hook via `pre-commit run --all-files`, then asserts the exit code matches `expected_returncode` (default 0) and the working tree is byte-identical to `expected/`. The tree compare uses `git diff --no-index` so it works with the BusyBox `diff` shipped in the project image. A `requires` file skips a case when a needed CLI tool is absent. Part of antonbabenko#823. Signed-off-by: penpal <unameme@proton.me>
Add `generates-wrapper-for-root-module`: a root module (with no
`provider_meta`) is wrapped, and the generated
`wrappers/{main,outputs,variables,versions}.tf` plus `README.md` are
checked against the golden tree. Exits 0 (only new files are created).
Part of antonbabenko#823.
Signed-off-by: penpal <unameme@proton.me>
Add `reformats-misaligned-hcl`: misaligned HCL is rewritten to canonical style. The hook modifies a tracked file, so `pre-commit` exits 1 (`expected_returncode=1`). The case requires `terraform`. Part of antonbabenko#823. Signed-off-by: penpal <unameme@proton.me>
Add `.github/workflows/e2e-tests.yaml`. Checkout runs on the host (the JS action needs Node), then the suite runs inside the published multi-tool image (`ghcr.io/antonbabenko/pre-commit-terraform:latest`), which supplies terraform/hcledit/pre-commit while the hook and test code come from the checkout -- so the image tag need not be in sync. Part of antonbabenko#823. Signed-off-by: penpal <unameme@proton.me>
Add `tests/e2e/README.md` describing the case layout, how to run the suite natively and inside the project image, and how to add a new case. Part of antonbabenko#823. Signed-off-by: penpal <unameme@proton.me>
Per review on antonbabenko#990: - replace the `RETURN` trap with explicit cleanup at the function's single exit (the `RETURN` trap is Bash-only and confused reviewers); - read the expected return code with `$(< file)` instead of `$(cat file)`; - quote the `true`/`false` flag values to avoid confusion with the builtins; - use `&>` instead of `> ... 2>&1`; - drop the redundant trailing slash on the copy destination; - print the summary with `printf` instead of a loop. No behavior change; the suite still passes natively and inside the image. Signed-off-by: penpal <unameme@proton.me>
Per review on antonbabenko#990, pin `ghcr.io/antonbabenko/pre-commit-terraform` by `sha256` digest instead of the floating `:latest` tag, so the bundled toolchain can't drift between runs. A Renovate annotation keeps the digest updated. Signed-off-by: penpal <unameme@proton.me>
Signed-off-by: penpal <unameme@proton.me>
pen-pal
force-pushed
the
feat/823-e2e-hook-tests
branch
from
235410e to
a6052ac
Compare
MaxymVlasov
reviewed
Jun 1, 2026
MaxymVlasov
left a comment
MaxymVlasov
left a comment
Collaborator
There was a problem hiding this comment.
Thank you for introducing this simple but powerful e2e test strategy!
I have a few requests for you before we proceed.
| env: | ||
| # renovate: datasource=docker depName=ghcr.io/antonbabenko/pre-commit-terraform | ||
| # yamllint disable-line rule:line-length | ||
| IMAGE: ghcr.io/antonbabenko/pre-commit-terraform:latest@sha256:4ef4b8323b27fc263535ad88c9d2f20488fcb3b520258e5e7f0553ed5f6692b5 |
Collaborator
There was a problem hiding this comment.
If we pin it to hash, then there's no sense to use the latest at all, which contradicts the main idea - test againt latest tool set & against the latest stable hook version.
In other words, we need 2 tests: against latest w/o pinning and against the latest available hook version, which can be bumped by renovate, but I'd like to have it as part of the release process.
Comment on lines
+71
to
+72
| docker build -t pct:e2e --build-arg INSTALL_ALL=true . | ||
| docker run --rm -v "$PWD:/lint" -w /lint --entrypoint bash pct:e2e \ |
Collaborator
There was a problem hiding this comment.
And what's the reason to build a separate image?
Just use a exisitng one https://github.com/antonbabenko/pre-commit-terraform#1-install-dependencies
Collaborator
There was a problem hiding this comment.
Let's add a section about requesting the addition of e2e hooks to https://github.com/antonbabenko/pre-commit-terraform/blob/master/.github/CONTRIBUTING.md#add-new-hook
|
This PR has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this PR will be closed in 10 days |
github-actions
Bot
added
the
stale
Co-authored-by: Maksym Vlasov <MaxymVlasov@users.noreply.github.com>
Co-authored-by: Maksym Vlasov <MaxymVlasov@users.noreply.github.com>
Co-authored-by: Maksym Vlasov <MaxymVlasov@users.noreply.github.com>
Contributor
Author
|
will apply the fixes based on comments |
github-actions
Bot
removed
the
stale
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Increment toward #823: a behavioral end-to-end test harness for the hooks. It runs each hook the way a user would - through
pre-commitagainst fixture files — and compares the result against a committed "golden" output tree.Addresses #823 (lays the foundation and seeds it with two hooks). It does not close the issue, since most hooks still need cases.
Harness -
tests/e2e/run_e2e_tests.shtests/e2e/cases/<hook_id>/<case_name>/. Adding a hook is just adding a dir — no runner changes.input/, renders the case's.pre-commit-config.yaml(__PCT_REPO__→ checkout root), runs the hook viapre-commit run --all-files.expected_returncode(default0) and the working tree is byte-identical toexpected/.requiresfile skips a case when a needed CLI tool is absent.Seed cases
terraform_fmtreformats-misaligned-hclpre-commitexits1(hook modifies a tracked file)terraform_wrapper_module_for_eachgenerates-wrapper-for-root-modulewrappers/{main,outputs,variables,versions}.tf+README.mdgenerated for aCI -
.github/workflows/e2e-tests.yamlRuns the suite on every PR inside the published multi-tool image (
ghcr.io/antonbabenko/pre-commit-terraform:latest). The image supplies the tooling (terraform,hcledit, …); the hook and test code come from the checkout, so the image tag need not be in sync.How can we test changes
Locally (needs
pre-commit,git, and the hooks' tools —terraform,hcledit):bash tests/e2e/run_e2e_tests.sh # => 2 passed, 0 failed, 0 skippedInside the project image (matches CI):