diff --git a/cli/audit.mdx b/cli/audit.mdx index 9b37a34..56e8a44 100644 --- a/cli/audit.mdx +++ b/cli/audit.mdx @@ -34,7 +34,8 @@ squirrel audit [options] | `--publish` | `-p` | Publish report to reports.squirrelscan.com (costs 2 credits) | `false` | | `--visibility` | | Visibility: `public`, `unlisted`, `private` | `public` | | `--yes` | `-y` | Skip confirmation prompts (e.g. cloud credit spend) | `false` | -| `--render` | | Render pages with a cloud browser for this run ([uses credits](/guides/browser-rendering); requires login) | `false` | +| `--render` | | Force cloud browser rendering for this run ([uses credits](/guides/browser-rendering); requires login) | auto | +| `--http` | | Force plain HTTP for this run - never render, never spend render credits | auto | | `--offline` | | Run fully offline: skip cloud features, publishing, and telemetry | `false` | ### Coverage Modes @@ -114,6 +115,24 @@ squirrel audit https://example.com --publish --visibility unlisted Publishing requires authentication. Run `squirrel auth login` first. +### Browser Rendering + +When you're **logged in, audits render every page in a cloud browser by default** (2 credits/page) - so JavaScript-heavy sites are audited against their fully rendered HTML. The first time this would spend credits the CLI asks once and remembers your answer. + +Force plain HTTP for a run (no rendering, no render credits): + +```bash +squirrel audit https://example.com --http +``` + +Force rendering for a run even if your config disables it: + +```bash +squirrel audit https://example.com --render +``` + +`--render` and `--http` are mutually exclusive. To set the behavior permanently, use `rendering` in `[cloud]` - see the [browser rendering guide](/guides/browser-rendering). + ### Fully Offline Audit Run an audit with zero network calls beyond the site being audited — no cloud diff --git a/cli/report.mdx b/cli/report.mdx index c66fa7e..150f7cc 100644 --- a/cli/report.mdx +++ b/cli/report.mdx @@ -303,7 +303,7 @@ Filter reports by these categories: - `video` - Video optimization - `analytics` - Analytics - `eeat` - E-E-A-T signals -- `adblock` - Adblock detection +- `blocking` - Ad-blocker & privacy-filter impact (`adblock` still accepted) ## Exit Codes diff --git a/cloud/credits.mdx b/cloud/credits.mdx index 8bb4bba..6c2e9ac 100644 --- a/cloud/credits.mdx +++ b/cloud/credits.mdx @@ -45,7 +45,7 @@ Top up: https://squirrelscan.com/account/credits | AI page analysis | 1 per page | [`ai/llm-parsability`](/rules/ai/llm-parsability), [`ai/page-type-match`](/rules/ai/page-type-match) | | Site type detection | 5 per audit | [`ai/site-type`](/rules/ai/site-type) | | Authority signals | 1 per page | [`eeat/authority-signals`](/rules/eeat/authority-signals) | -| Adblock/privacy check | 5 per audit | All three [adblock rules](/rules/adblock) share one check | +| Blocking check (ad + privacy) | 5 per audit | All three [Blocking rules](/rules/adblock) share one check | | Dead link checking | 1 per 100 URLs | [`links/dead-links`](/rules/links/dead-links) | | Browser rendering | 2 per page | [Cloud rendering](/cloud/rendering) | | Keyword gaps | 25 per audit | [`gaps/keywords`](/rules/gaps/keywords) (opt-in) | @@ -60,7 +60,7 @@ Three guardrails keep an audit from spending more than you expect: ```toml squirrel.toml [cloud] -max_credits_per_audit = 200 # hard cap per audit; 0 = unlimited +max_credits_per_audit = 1000 # hard cap per audit; 0 = unlimited confirm_threshold = 50 # estimates above this prompt before spending enabled = true # master switch for all cloud features ``` diff --git a/cloud/rendering.mdx b/cloud/rendering.mdx index dc61cff..8d6265c 100644 --- a/cloud/rendering.mdx +++ b/cloud/rendering.mdx @@ -4,15 +4,33 @@ description: "How cloud browser rendering works under the hood" icon: "globe" --- -By default the crawler fetches pages with plain HTTP - fast, free, and accurate for server-rendered sites. Sites that build their content with JavaScript (SPAs, client-side frameworks) can look nearly empty to a plain fetch, which skews every content rule. +A plain HTTP fetch is fast, free, and accurate for server-rendered sites - but sites that build their content with JavaScript (SPAs, client-side frameworks) can look nearly empty to it, which skews every content rule. Cloud browser rendering fixes that: each page is loaded in a **real headless browser** in the cloud, and the fully rendered DOM is returned to your local crawler. Everything downstream - parsing, rules, reports - works exactly as it does with plain HTTP, just on the post-JavaScript HTML. Looking for setup instructions? See the [browser rendering guide](/guides/browser-rendering). This page explains how the service behaves. +## When it runs + +Rendering follows your login state by default: + +- **Logged in** - audits render every page in the cloud automatically. The first time this would spend credits, the CLI asks once (`Render pages in the cloud? [Y/n]`) and remembers your choice in `squirrel self settings` - it never asks again. +- **Logged out, `--offline`, or you declined the prompt** - plain HTTP, no credits. + +Override the default any time: + +| Want | How | +|------|-----| +| Plain HTTP for one run | `squirrel audit --http` | +| Render for one run | `squirrel audit --render` | +| Always plain HTTP for a project | `rendering = "http"` in `[cloud]` | +| Always render for a project | `rendering = "browser"` in `[cloud]` | + +Flags win over config, and config wins over the auto default. Non-interactive runs (CI, agents, pipes) never prompt - they render only if you've already consented or set `rendering = "browser"`, otherwise they use plain HTTP. + ## Cost -**2 credits per page rendered.** A 100-page crawl with rendering costs 200 credits, so mind your [per-audit cap](/cloud/credits#spend-controls) - the default cap of 200 covers rendering for about 100 pages and nothing else. The crawl progress line shows the running spend: +**2 credits per page rendered.** A 100-page crawl with rendering costs 200 credits, so mind your [per-audit cap](/cloud/credits#spend-controls) - the default cap of 1000 covers rendering for about 500 pages. The crawl progress line shows the running spend: ``` Crawling (cloud-rendered) 34/100 · ~68 credits diff --git a/cloud/rules.mdx b/cloud/rules.mdx index 540303e..929ad9a 100644 --- a/cloud/rules.mdx +++ b/cloud/rules.mdx @@ -21,7 +21,7 @@ Cloud rules are regular audit rules that need data your machine doesn't have - A | [`gaps/keywords`](/rules/gaps/keywords) | Keywords competitors rank for that you don't | 25 credits per audit, **opt-in** | | [`gaps/content`](/rules/gaps/content) | Topic clusters with search demand you don't cover | 25 credits per audit, **opt-in** | -The three adblock rules share a single 5-credit blocklist check per audit - enabling all three costs the same as enabling one. +The three [Blocking](/rules/adblock) rules (IDs keep the `adblock/` prefix) share a single 5-credit blocklist check per audit - enabling all three costs the same as enabling one. They report as warnings in their own report section. The two gap-analysis rules are **disabled by default** because of their cost. Enable them explicitly: diff --git a/configuration/cloud.mdx b/configuration/cloud.mdx index 96d7218..c291fe9 100644 --- a/configuration/cloud.mdx +++ b/configuration/cloud.mdx @@ -12,10 +12,10 @@ All cloud settings are no-ops when you're not logged in - cloud rules simply rep ```toml squirrel.toml [cloud] enabled = true # master switch for all cloud features -max_credits_per_audit = 200 # hard spend cap per audit; 0 = unlimited +max_credits_per_audit = 1000 # hard spend cap per audit; 0 = unlimited confirm_threshold = 50 # estimates above this prompt for confirmation batch_size = 20 # pages per cloud service request -rendering = "http" # "http" (default) or "browser" +# rendering # omit = auto (render when logged in); set "http" or "browser" to force render_concurrency = 6 # concurrent browser-render jobs (1-8) ``` @@ -27,7 +27,7 @@ Master switch. Set to `false` to disable all cloud calls for this project - clou ### `max_credits_per_audit` -**Default:** `200` +**Default:** `1000` Hard cap on credits a single audit may spend. When the estimate exceeds the cap, cloud work is truncated deterministically and the rules that didn't run report `skipped` with reason `credit-cap-reached`. Set to `0` for unlimited. @@ -45,9 +45,14 @@ Pages per request to per-page cloud services. Lower it only if you hit request-s ### `rendering` -**Default:** `"http"` +**Default:** unset (auto) -Crawl fetch mode. `"browser"` renders every page with a cloud headless browser at **2 credits per page** - see the [browser rendering guide](/guides/browser-rendering). The `--render` flag on `squirrel audit` enables it for a single run without changing config. +Crawl fetch mode. Leave it unset and squirrelscan decides: **logged-in users render every page in a cloud browser; logged-out runs use plain HTTP**. The first time an auto-render would spend credits, the CLI asks once and remembers your answer (`squirrel self settings`). Set it explicitly to override the auto behavior: + +- `"http"` - always plain HTTP; never render, never prompt (no credits) +- `"browser"` - always render every page at **2 credits per page** + +The `--render` and `--http` flags on `squirrel audit` force rendering on or off for a single run and always win over config. See the [browser rendering guide](/guides/browser-rendering). ### `render_concurrency` diff --git a/configuration/index.mdx b/configuration/index.mdx index 0437eb1..e8cdf4d 100644 --- a/configuration/index.mdx +++ b/configuration/index.mdx @@ -15,7 +15,8 @@ squirrelscan works out of the box with sensible defaults. You only need configur - Respects robots.txt - Checks external links (cached 7 days) - Runs all rules (gap-analysis rules are opt-in) -- Cloud rules run when logged in, capped at 200 credits per audit +- Cloud rules run when logged in, capped at 1000 credits per audit +- Logged-in audits render pages in a cloud browser by default (one-time opt-in; `--http` to skip) - Console output format ## Configuration System diff --git a/configuration/rule-options.mdx b/configuration/rule-options.mdx index c40c29c..78b4bd1 100644 --- a/configuration/rule-options.mdx +++ b/configuration/rule-options.mdx @@ -623,7 +623,7 @@ If 90%+ of crawled pages are noindexed → `warn` (possible misconfiguration) --- -### Adblock Rules +### Blocking Rules #### `adblock/element-hiding` diff --git a/configuration/rules.mdx b/configuration/rules.mdx index a647558..35893df 100644 --- a/configuration/rules.mdx +++ b/configuration/rules.mdx @@ -511,13 +511,13 @@ disable = [] # Remove ai/* from disable list --- -### Adblock Detection +### Blocking **ID:** `adblock/*` -**Rules:** 2 -**Description:** Adblock filter impact +**Rules:** 3 +**Description:** Ad-blocker & privacy-filter impact (ad blocking + privacy blocking) -Element hiding, blocked tracking domains. +Element hiding, blocked tracking links, and privacy-filter matches. Reported as warnings in their own report section. **Enable:** ```toml @@ -526,8 +526,9 @@ enable = ["adblock/*"] ``` **Rules:** -- `adblock/element-hiding` - Elements hidden by adblockers -- `adblock/blocked-links` - Links to blocked domains +- `adblock/element-hiding` - Elements hidden by adblockers (ad blocking) +- `adblock/blocked-links` - Links to blocked domains (ad blocking) +- `adblock/privacy-blocked` - Trackers EasyPrivacy would block (privacy blocking) --- diff --git a/docs.json b/docs.json index e18b031..d58aff5 100644 --- a/docs.json +++ b/docs.json @@ -406,7 +406,7 @@ ] }, { - "group": "Adblock Detection", + "group": "Blocking", "pages": [ "rules/adblock/index", "rules/adblock/element-hiding", diff --git a/guides/browser-rendering.mdx b/guides/browser-rendering.mdx index 2b22600..060ae02 100644 --- a/guides/browser-rendering.mdx +++ b/guides/browser-rendering.mdx @@ -9,36 +9,48 @@ Browser rendering solves this: pages are loaded in a real headless browser in th Browser rendering is a [cloud feature](/cloud) - it requires login and costs **2 credits per page**. See [how it works under the hood](/cloud/rendering). -## Try it on one run +## It's automatic when you're logged in ```bash squirrel auth login # once -squirrel audit https://example.com --render +squirrel audit https://example.com ``` -`--render` uses the cloud browser for this run only. The crawl progress shows the running credit spend: +Once you're logged in, audits render every page in the cloud by default. The first time this would spend credits, the CLI asks once and remembers your answer - so you opt in knowingly and never get prompted again. The crawl progress shows the running credit spend: ``` Crawling (cloud-rendered) 34/100 · ~68 credits ``` -## Make it permanent for a project +Logged-out runs (and `--offline`) always use plain HTTP and cost nothing. -If a site always needs rendering, set it in `squirrel.toml`: +## Turn it off for one run + +If a site is server-rendered and you don't want to spend render credits: + +```bash +squirrel audit https://example.com --http +``` + +`--http` forces plain HTTP for that run. The opposite, `--render`, forces rendering even when your config disables it. The two are mutually exclusive. + +## Make the choice permanent for a project + +Pin the behavior in `squirrel.toml` so it never depends on login state or the prompt: ```toml squirrel.toml [cloud] -rendering = "browser" # default: "http" +rendering = "browser" # always render; use "http" to always skip ``` -Every audit of this project now renders pages, no flag needed. +Setting `rendering` explicitly overrides the auto default - no prompt, every audit of this project behaves the same. ## Watch your page count At 2 credits per page, rendering is the most spend-sensitive cloud feature. Two settings to keep in mind: - `--max-pages` / `[crawler] max_pages` - fewer pages, fewer credits. -- `[cloud] max_credits_per_audit` (default **200**) - the hard cap. A 100-page rendered crawl alone hits it; raise it or cap your pages. +- `[cloud] max_credits_per_audit` (default **1000**) - the hard cap. It covers rendering for about 500 pages; raise it or cap your pages for larger rendered crawls. ```toml squirrel.toml [cloud] diff --git a/reports.mdx b/reports.mdx index f138cd4..014a061 100644 --- a/reports.mdx +++ b/reports.mdx @@ -424,7 +424,7 @@ squirrel report --category core,links,images - `video` - Video optimization (schema, thumbnails) - `analytics` - Analytics tracking (GTM, consent) - `eeat` - E-E-A-T signals (author, expertise, trust) -- `adblock` - Adblock detection +- `blocking` - Ad-blocker & privacy-filter impact (`adblock` still accepted) ### Combined Filters diff --git a/rules/adblock/blocked-links.mdx b/rules/adblock/blocked-links.mdx index 962ca67..8c63d08 100644 --- a/rules/adblock/blocked-links.mdx +++ b/rules/adblock/blocked-links.mdx @@ -10,9 +10,9 @@ Checks for links and resources pointing to blocked tracking domains (matched ser | | | |---|---| | **Rule ID** | `adblock/blocked-links` | -| **Category** | [Adblock Detection](/rules/adblock) | +| **Category** | [Blocking](/rules/adblock) → Ad blocking | | **Scope** | Site-wide | -| **Severity** | info | +| **Severity** | warning | | **Weight** | 2/10 | ## Solution @@ -43,7 +43,7 @@ maxMatchesToReport = 20 disable = ["adblock/blocked-links"] ``` -### Disable all Adblock Detection rules +### Disable all Blocking rules ```toml squirrel.toml [rules] diff --git a/rules/adblock/element-hiding.mdx b/rules/adblock/element-hiding.mdx index e9e5a26..2b73d1c 100644 --- a/rules/adblock/element-hiding.mdx +++ b/rules/adblock/element-hiding.mdx @@ -10,7 +10,7 @@ Checks for elements that would be hidden by common adblockers (matched server-si | | | |---|---| | **Rule ID** | `adblock/element-hiding` | -| **Category** | [Adblock Detection](/rules/adblock) | +| **Category** | [Blocking](/rules/adblock) → Ad blocking | | **Scope** | Site-wide | | **Severity** | warning | | **Weight** | 3/10 | @@ -44,7 +44,7 @@ maxMatchesToReport = 20 disable = ["adblock/element-hiding"] ``` -### Disable all Adblock Detection rules +### Disable all Blocking rules ```toml squirrel.toml [rules] diff --git a/rules/adblock/index.mdx b/rules/adblock/index.mdx index c6c3c20..36a46cd 100644 --- a/rules/adblock/index.mdx +++ b/rules/adblock/index.mdx @@ -1,13 +1,15 @@ --- -title: "Adblock Detection" -description: "Detection of content blocked by ad blockers and privacy filters" +title: "Blocking" +description: "Content, links, and trackers that ad blockers and privacy filters block" --- **Cloud rules** - requires login (`squirrel auth login`). All three rules share a single **5-credit** blocklist check per audit, matched server-side against the full EasyList and EasyPrivacy filter lists. Skipped when not logged in. See [Cloud rules](/cloud/rules). -Detection of content blocked by ad blockers and privacy filters. +The **Blocking** category covers content, links, and trackers that ad blockers and privacy filters would block. It reports in its own report section with two sub-groups - **Ad blocking** and **Privacy blocking** - and every check is a **warning** (it's signal worth knowing, not a hard failure). Enabled by default for logged-in users. -## Rules +Rule IDs and config filters keep the `adblock/` prefix (e.g. `adblock/*`) - only the report category was renamed to Blocking. + +## Ad blocking @@ -16,12 +18,17 @@ Detection of content blocked by ad blockers and privacy filters. Checks for links and resources pointing to blocked tracking domains + + +## Privacy blocking + + Checks for trackers and analytics that EasyPrivacy would block -## Disable All Adblock Detection Rules +## Disable all Blocking rules ```toml squirrel.toml [rules] diff --git a/rules/adblock/privacy-blocked.mdx b/rules/adblock/privacy-blocked.mdx index d71193b..95f126b 100644 --- a/rules/adblock/privacy-blocked.mdx +++ b/rules/adblock/privacy-blocked.mdx @@ -10,9 +10,9 @@ Checks for trackers and analytics that privacy filter lists (EasyPrivacy) would | | | |---|---| | **Rule ID** | `adblock/privacy-blocked` | -| **Category** | [Adblock Detection](/rules/adblock) | +| **Category** | [Blocking](/rules/adblock) → Privacy blocking | | **Scope** | Site-wide | -| **Severity** | info | +| **Severity** | warning | | **Weight** | 2/10 | | **Default** | Enabled | @@ -33,7 +33,7 @@ Analytics, session replay, and tracking pixels on matched domains will silently disable = ["adblock/privacy-blocked"] ``` -### Disable all adblock rules +### Disable all Blocking rules ```toml squirrel.toml [rules] diff --git a/rules/index.mdx b/rules/index.mdx index 36dd5b1..8fc4eb8 100644 --- a/rules/index.mdx +++ b/rules/index.mdx @@ -67,8 +67,8 @@ Most rules run locally for free. A handful of [cloud rules](/cloud/rules) use AI Experience, expertise, authority, trust signals (15 rules) - - Detection of content blocked by ad blockers (3 rules) + + Content, links, and trackers blocked by ad blockers and privacy filters (3 rules) Keyword and content gaps backed by live search data (2 rules, opt-in)