d-flex flex-column width-full">

fix: treat a 5xx robots.txt response as complete disallow

4c5cb1c

Select commit

Loading

Failed to load commit list.

Uh oh!

There was an error while loading. Please reload this page.

Sign in for the full log view

Apify pull request toolkit on: pull_request 1

apify-pr-toolkit

CI (PR) on: pull_request 6

PR title check

Checks / Actions lint check

Checks / Spell check

Checks / Lint check / Lint check (3.10)

Checks / Lint check / Lint check (3.11)

Checks / Lint check / Lint check (3.12)

Checks / Lint check / Lint check (3.13)

Checks / Lint check / Lint check (3.14)

Checks / Type check / Type check (3.10)

Checks / Type check / Type check (3.11)

Checks / Type check / Type check (3.12)

Checks / Type check / Type check (3.13)

Checks / Type check / Type check (3.14)

Checks / Markdown lint check

Checks / Website lint check

Checks / Image lint check

Checks / Unit tests / Unit tests (ubuntu-latest, 3.10)

Checks / Unit tests / Unit tests (ubuntu-latest, 3.11)

Checks / Unit tests / Unit tests (ubuntu-latest, 3.12)

Checks / Unit tests / Unit tests (ubuntu-latest, 3.13)

Checks / Unit tests / Unit tests (ubuntu-latest, 3.14)

Checks / Unit tests / Unit tests (windows-latest, 3.10)

Checks / Unit tests / Unit tests (windows-latest, 3.11)

Checks / Unit tests / Unit tests (windows-latest, 3.12)

Checks / Unit tests / Unit tests (windows-latest, 3.13)

Checks / Unit tests / Unit tests (windows-latest, 3.14)

Checks / Unit tests / Unit tests (macos-latest, 3.10)

Checks / Unit tests / Unit tests (macos-latest, 3.11)

Checks / Unit tests / Unit tests (macos-latest, 3.12)

Checks / Unit tests / Unit tests (macos-latest, 3.13)

Checks / Unit tests / Unit tests (macos-latest, 3.14)

Checks / Package check

Checks / Doc check / Docs check

Annotations

1 warning

run pull-request-toolkit action

Skipping toolkit action for PR from external fork: anxkhn/crawlee-python

apify-pr-toolkit

succeeded Jul 13, 2026 in 5s

Loading

Re-running jobs...