Skip to content

Add explicit codecov.yml policy to unblock admin-merge workflow #3476

@MaxGhenis

Description

@MaxGhenis

Summary

codecov/patch enforces a 77.10% patch-coverage threshold derived from the base branch's current coverage. There is no codecov.yml in the repo, so the threshold is implicit and tracks master drift. Recent bug-fix PRs (e.g. #3454) failed this check despite adding regression tests for every change, because fixes that touch non-code paths (exception handlers, whitelist guards, return-None short-circuits) don't proportionally lift coverage.

What goes wrong

  • The check blocks merge via mergeStateStatus: BLOCKED even though mergeable: MERGEABLE — admins have to merge with --admin to proceed.
  • There is no visible threshold-policy document: contributors can't see what target they're shooting at without opening Codecov's UI.
  • The de-facto workflow (admin-merge on coverage failure) undermines the signal — if admin-merge is always fine, the check should not be required; if the check matters, the threshold should be published and tunable.

Suggested fix

Add a codecov.yml at repo root with an explicit policy:

coverage:
  status:
    project:
      default:
        target: auto
        threshold: 1%
    patch:
      default:
        target: 70%    # or whatever realistic floor matches the codebase
        informational: false

Options:

  • Target auto with a threshold of 1–2% — patch can drop coverage by at most 1–2% relative to base (forgiving for bug fixes that touch few lines).
  • Fixed 70% floor — easier to reason about; allows fixes that don't add tests to non-essential defensive paths.
  • Informational = true — keeps the signal visible without blocking merges; admin-merge workaround stops being necessary.

Document the choice in CONTRIBUTING.md so contributors know the target.

Severity

Medium — procedural friction that degrades to "admin-merge everything" in practice, which hides the coverage signal entirely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions