diff --git a/ChangeLog.md b/ChangeLog.md index 86bb4fc..299a4af 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -17,6 +17,460 @@ _No unreleased changes._ --- +## 26.30 — 2026-06-05 + +VNC fingerprinting. Port 5900 was in the deep-probe list but had no handler, +so `Port.Service` stayed empty for VNC servers. Post-backlog feature work +(no `Planning.md` number). + +### Added + +- **VNC identification** (`vncBanner`) — reads the RFB ProtocolVersion + greeting the server sends on connect (e.g. `RFB 003.008`) → + `VNC: RFB 003.008`. Validates the `RFB ` prefix, so a non-VNC service on + 5900 is left unlabelled rather than mistagged. + +### Changed + +- **`fingerprint()` now dispatches port 5900** to the new handler. + +### Tests + +- `internal/scanner/banner_test.go` — RFB greeting identified; a non-RFB + greeting yields "". + +### Notes + +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.29 — 2026-06-05 + +macOS MAC/vendor enrichment. ARP enrichment was Linux-only (`/proc/net/arp`); +on macOS the `MACAddress`/`Vendor` fields were always empty. This adds a +macOS neighbour lookup **without invoking a shell** — it reads the routing +table via a routing socket (`golang.org/x/net/route`), preserving the +project's "no external processes are invoked" posture (OWASP A03). +Post-backlog feature work (no `Planning.md` number). + +### Added + +- **macOS neighbour resolution** (`arp_darwin.go`) — dumps the routing + information base and returns the link-layer address for a target IP. + Verified on a live host against `arp -n`. +- **`golang.org/x/net`** promoted from an indirect to a direct dependency + (already in the module graph; still pure Go, no cgo). + +### Changed + +- **`arp.go` refactored for portability**: the `/proc/net/arp` parser is now + `parseProcARP` (pure, testable on any OS); `lookupARP` consults it first, + then a platform `neighbourMAC` (`arp_darwin.go` for macOS, + `arp_fallback.go` returning "" for Linux/Windows/other — Linux is fully + served by the proc path). + +### Tests + +- `internal/scanner/arp_test.go` — parser tests rewritten around + `parseProcARP`; a cross-platform `lookupARP` happy-path test via a proc + fixture. +- `internal/scanner/arp_darwin_test.go` — validates `neighbourMAC` against + the live routing table (skips when no neighbour entries exist) and the + no-match/bad-input paths. + +### Notes + +- Windows neighbour enrichment remains a graceful no-op (no + `GetIpNetTable` implementation yet); it degrades to vendor-less inventory + rather than guessing. +- Builds verified for darwin, linux, and windows; `go test ./...`, + `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.28 — 2026-06-05 + +JSON scan-history API. The query API exposed hosts but not scans; +programmatic consumers (freshness/coverage dashboards) had to scrape the +HTML `/scans` page. Adds a symmetric endpoint. Post-backlog feature work +(no `Planning.md` number). + +### Added + +- **`GET /api/v1/scans`** — paginated JSON scan history with the same + `?limit=`/`?offset=` envelope (`{total,limit,offset,scans}`) as + `/api/v1/hosts`, plus an optional `?subnet=` exact-match filter. Newest + first. + +### Tests + +- `internal/admin/api_test.go` — full list, pagination (total vs page), + subnet filter, and invalid `limit` → 400. + +### Notes + +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.27 — 2026-06-05 + +Scanner config-honesty fixes. Two small consistency gaps. Post-backlog +polish (no `Planning.md` number). + +### Changed + +- **Reverse-DNS now honours the configured per-host timeout** instead of a + hardcoded 500ms. Aggressive or relaxed `scanner.timeout` / per-profile + `timeout` values now apply to PTR lookups too (falls back to 500ms only + when unset). + +### Added + +- **`inventory_udp_probe_failure_total`** metric, incremented on a + definitive closed (ICMP port-unreachable) UDP result — the counterpart to + the existing success counter. Ambiguous no-reply probes are still not + counted (they're genuinely unknown, not failures). + +### Notes + +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.26 — 2026-06-05 + +OUI table expansion for camera / NAS / IoT vendors, wired into the +classifier. The 26.23 `camera` rule could only fire on RTSP (not a default +probe port), so it rarely triggered; this adds the vendor OUIs that make +vendor-based detection work. Post-backlog feature work (no `Planning.md` +number). + +### Added + +- **14 new OUI prefixes**, each verified against the IEEE registry (via + maclookup.app): Hikvision (3), Dahua (2), Axis (2), QNAP (1), + Ubiquiti (3), Espressif (3). +- **Classifier vendor rules** using them: Hikvision/Dahua/Axis → `camera`, + QNAP → `nas` (joining Synology/WD), Espressif → `embedded` (joining + Raspberry Pi). + +### Tests + +- `internal/scanner/arp_test.go` — the new prefixes resolve to the right + vendor, plus a guard that `00:08:9b` stays **not** QNAP (it's ICP + Electronics — a candidate that verification rejected). +- `internal/scanner/classify_test.go` — camera/nas/embedded by vendor. + +### Notes + +- Verification mattered: a plausible "QNAP" prefix (`00:08:9b`) turned out + to be ICP Electronics and was excluded rather than shipped wrong. +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.25 — 2026-06-05 + +Pagination for the admin host and scan list pages. `/hosts` and `/scans` +rendered every row in one response — a deployment with tens of thousands of +hosts produced a multi-megabyte HTML page. Both pages now paginate with the +same `?limit=`/`?offset=` convention as the JSON API. Post-backlog +reliability work (no `Planning.md` number). + +### Added + +- **`?limit=` / `?offset=` on `/hosts` and `/scans`** (default 100, capped + at 1000), reusing the API's `parsePagination`. Prev/Next controls and a + "Showing X–Y of N" indicator render via a shared `pager` template + partial; the controls hide when everything fits on one page. +- **`pager` type + `newPager` / `pageSlice` helpers** in `internal/admin`. + +### Changed + +- **The host-inventory subtitle now reports the full total**, not the + current page size. + +### Tests + +- `internal/admin/handlers_extra_test.go` — page windowing (rows in/out of + range), Prev/Next link targets, total reporting, and an invalid `limit` + → 400. + +### Notes + +- This bounds the rendered page size. The underlying `store.List` still + loads the full slice before windowing; pushing `LIMIT`/`OFFSET` into the + store interface is a larger change left for later. +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.24 — 2026-06-05 + +UDP service fingerprinting. The scanner recorded open UDP ports but left +`Port.Service` empty (an in-code "out of scope" note); a DNS resolver and a +plain open UDP port were indistinguishable in the inventory. UDP ports +confirmed open are now fingerprinted for the two highest-value protocols. +Post-backlog feature work (no `Planning.md` number). + +### Added + +- **DNS fingerprint** — sends a standard A query for the root and confirms + the reply is a DNS response (QR bit set, transaction ID echoed) → + `DNS`. Identifies the service regardless of the answer (REFUSED still + proves DNS). +- **NTP fingerprint** — sends an NTPv3 client request and checks the reply + is server mode → `NTP`, with the stratum appended when valid + (`NTP (stratum 2)`). +- **`udpFingerprint` / `udpExchange`** helpers in a new `udp_banner.go`. + +### Changed + +- **`udpScan` now fingerprints open UDP ports** (ports 53/123 today; others + still record an empty Service). One extra datagram per matched open port. + +### Tests + +- `internal/scanner/udp_banner_test.go` — DNS identified / not-DNS, NTP with + stratum / non-server mode, unfingerprinted port, and the no-responder + timeout, via a UDP test responder. + +### Notes + +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.23 — 2026-06-05 + +Device-classifier expansion. Several common asset classes previously fell +through to the generic `appliance` tag (or no tag). The classifier gains +five new categories, built only on signals already available (NIC OUI + +open ports) — no dead vendor strings. Post-backlog feature work (no +`Planning.md` number). + +### Added + +- **`nas`** — Synology / Western Digital by NIC OUI, or NFS (2049) + + SMB (445). Fires before the Windows SMB rule so a NAS isn't mislabelled + `windows-host`. +- **`hypervisor`** — now also matches QEMU/KVM, VirtualBox, and Microsoft + Hyper-V by OUI, and Proxmox VE by its 8006 management port (previously + only VMware-by-ports). +- **`kubernetes-node`** — kube-apiserver (6443), etcd (2379), or kubelet + (10250). +- **`container-host`** — Docker daemon (2375/2376). +- **`camera`** — RTSP (554). + +### Tests + +- `internal/scanner/classify_test.go` — cases for each new category plus a + regression that SMB alone is still `windows-host` (not `nas`). + +### Notes + +- The Kubernetes/Docker/Proxmox/RTSP ports sit outside the default + deep-probe list, so those labels fire when an operator adds the ports via + a per-subnet `deep_probe_ports` profile; NAS-by-NFS+SMB and the OUI-based + rules fire under the default configuration. +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.22 — 2026-06-05 + +Test-coverage fill for previously untested units. No behaviour change. +Post-backlog quality work (no `Planning.md` number). + +### Tests + +- **`internal/logging`** — `0% → 100%`. Level parsing (debug/info/warn/ + error + unknown→info fallback), JSON vs text handler selection, and the + `agent` field injection, via a captured-stdout helper. +- **`internal/health/client.go`** — the watchdog peer client had no test + file. Added `Ping` (200 / non-200 / bearer-token / connection error) + and `FetchStatus` (decode OK / bad JSON) coverage. +- **`internal/admin`** — covered the previously untested handlers: + `handleWatchdog` (with and without a peer) and `handleScanTrigger` + (not-wired → 501, success → 204, already-pending → 503), plus the CSRF + rejection path on `POST /scan` (missing token → 403). + +### Notes + +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + Package coverage after this change: logging 100%, health 79.6%, admin + 74.1%, scanner 71.7%. + +--- + +## 26.21 — 2026-06-05 + +Service fingerprinting for PostgreSQL, Redis, and Memcached. These ports +(5432, 6379, 11211) were already in the deep-probe list and recorded as +open, but `fingerprint()` had no handler for them, so `Port.Service` was +always empty — operators couldn't tell a Redis from a random open port. +Post-backlog feature work (no `Planning.md` number). + +### Added + +- **PostgreSQL identification** (`postgresProbe`) — issues the SSLRequest + startup packet and reads the single-byte `S`/`N` reply. Reliable + identification without authenticating; labelled `PostgreSQL`. +- **Redis/Valkey fingerprinting** (`redisInfo`) — sends `INFO server` and + parses `redis_version:` → `Redis: `. An auth-gated server + (`-NOAUTH`) is still identified as `Redis (auth required)`. +- **Memcached fingerprinting** (`memcachedVersion`) — sends the text + `version` command → `Memcached: `. + +### Changed + +- **`fingerprint()` now dispatches ports 5432/6379/11211** to the new + handlers; all other ports are unchanged. + +### Tests + +- `internal/scanner/banner_test.go` — Redis version + NOAUTH paths, + Memcached version + non-memcached reply, Postgres `S`/`N` identification + + non-postgres reply. Adds a `startRequestResponse` test helper for + client-speaks-first protocols. + +### Notes + +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.20 — 2026-06-05 + +Scan-history retention. The `scans` table grew without bound: hosts had a +`host_ttl` pruner, but completed scan records accumulated forever (≈105K +rows/year at the 5-minute default), bloating the DB and the unbounded +`/scans` view. This adds an optional retention policy mirroring host +pruning. Post-backlog reliability work (no `Planning.md` number). + +### Added + +- **`scanner.scan_history_ttl` config** — when set, the end of each scan + cycle deletes scan rows whose `started_at` is older than `now - TTL`. + Zero (the default) keeps full history, so existing deployments are + unchanged. +- **`store.ScanStore.DeleteBefore(ctx, cutoff)`** + its SQLite + implementation (`DELETE FROM scans WHERE started_at < ?`, returning the + row count) — a single bounded DELETE, not a list-then-delete loop. +- **`inventory_scans_pruned_total`** Prometheus counter. + +### Changed + +- **The agent now runs a scan-history prune each cycle**, right after the + host prune (`Agent.pruneScans`). The `Agent` retains the `ScanStore` + passed to `New` for this; no constructor signature change. + +### Tests + +- `internal/sqlite/scan_test.go` — `DeleteBefore` removes only rows older + than the cutoff and reports the count; no-match returns 0. +- `internal/agent/agent_test.go` — old scans pruned when TTL is set; full + history kept when TTL is 0. +- `internal/config/config_test.go` — `scan_history_ttl` parses; default 0. + +### Notes + +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.19 — 2026-06-05 + +Alert-sink URL validation. `watchdog.peer_addr` was scheme-validated at +config load, but `alerts.webhook.url` was passed to the HTTP client with +no validation at all — a typo or hostile config value like +`file:///etc/passwd` or `gopher://…` reached the client verbatim. Syslog +addresses were validated only when the sink dialed. This extends the +existing peer-address guard to every outbound sink target (OWASP A10). +Post-backlog security hardening (no `Planning.md` number). + +### Added + +- **`validateSinkURL`** in `internal/config` — a shared scheme/host check + used for both alert sinks. + +### Changed + +- **`alerts.webhook.url` is now scheme-validated at config load** — + rejected unless `http` or `https` with a non-empty host. The agent + refuses to start on an invalid value instead of failing silently on the + first event. +- **`alerts.syslog.addr` is now scheme-validated at config load** too + (`udp`/`tcp` + host), giving a clear boot-time error before any network + dial is attempted. The eager-dial check in `NewSyslogSink` remains as + defence in depth. + +### Notes + +- Private/internal hosts are deliberately **not** blocked — internal + webhook receivers and localhost syslog are legitimate, common + deployments, consistent with `peer_addr` allowing loopback. The guard + targets scheme confusion, not network egress policy. +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green. + +--- + +## 26.18 — 2026-06-05 + +Admin console authentication gate. The admin console — which serves the +full host/port inventory, `/export.{json,csv}`, the `/api/v1/*` query API, +and the `POST /scan` trigger — previously had no authentication and no +off-loopback guard, unlike the health server. Binding `admin.addr` to a +non-loopback address (e.g. `0.0.0.0` for Docker, or via +`INVENTORY_ADMIN_ADDR`) exposed everything to anyone on the network. This +closes that asymmetry. Post-backlog security hardening (no `Planning.md` +number). + +### Added + +- **`admin.auth_token` config field** (env: `INVENTORY_ADMIN_TOKEN`) — a + shared secret that gates every admin route. When set, clients + authenticate with either `Authorization: Bearer ` (curl, the JSON + API, exports, scripts) or HTTP Basic auth using the token as the + password and any username (so browsers show a native login prompt). + Tokens are compared in constant time. Empty/unset is a no-op, so the + loopback default stays credential-free. +- **`admin.ServerOptions`** — carries `AuthToken` into `admin.NewServer`, + mirroring `health.ServerOptions`. + +### Changed + +- **`admin.NewServer` takes a trailing `ServerOptions` argument.** In-tree + callers (runtime + tests) updated. The admin package has no external + callers, so no compatibility shim was added. +- **Off-loopback admin binds now require a token.** Config validation + refuses to start when `admin.addr` is non-loopback and no + `admin.auth_token` is set — the same rule already enforced for + `health.addr`. +- **`chmod 600` enforcement extended** — a config file carrying + `admin.auth_token` must not be group/world-readable, matching the + existing check for `health.auth_token` and `watchdog.peer_token`. + +### Tests + +- `internal/admin/auth_test.go` — no-creds → 401 + `WWW-Authenticate`; + correct Bearer → 200; correct Basic password → 200; wrong Bearer/Basic + → 401; exports and `POST /scan` gated (auth precedes CSRF); empty token + → ungated (loopback regression guard). +- `internal/config/config_test.go` — off-loopback admin without token → + error; with token → ok; `INVENTORY_ADMIN_TOKEN` override satisfies the + rule; world-readable file carrying an admin token → refused. + +### Notes + +- `go test ./...`, `go vet ./...`, and `golangci-lint run ./...` all green + (0 issues). The shipped `configs/*.json` use loopback binds, so no token + is required for the default local/paired deployments. + +--- + ## 26.17 — 2026-05-27 Documentation catch-up. No behaviour change — closes the gap between diff --git a/README.md b/README.md index 60f0c6a..955c417 100644 --- a/README.md +++ b/README.md @@ -11,12 +11,12 @@ The system is designed to run as **two cooperating agent instances** — named * ## Features - **Active discovery** — concurrent TCP-probe scanning across configurable CIDR ranges to find live hosts. Optional deep TCP and UDP probe passes per profile. -- **Asset fingerprinting** — banner-grab on SSH, FTP, SMTP, POP3, IMAP, HTTP, HTTPS (with TLS cert peek), MySQL handshake, Telnet. Stored per-port in `Port.Service`. -- **Device-type classifier** — heuristic rules over (vendor, OS banner, open ports) tag hosts as printer / router / hypervisor / windows-host / windows-dc / database (mysql|postgres|…) / mail-server / linux-host / appliance / iot-broker / embedded. -- **MAC + vendor enrichment** — `/proc/net/arp` lookup on Linux + embedded OUI prefix table for ~80 common vendors. +- **Asset fingerprinting** — banner-grab on SSH, FTP, SMTP, POP3, IMAP, HTTP, HTTPS (with TLS cert peek), MySQL handshake, PostgreSQL (SSLRequest probe), Redis (`INFO`), Memcached (`version`), VNC (RFB greeting), Telnet, plus UDP DNS and NTP (stratum) probes. Stored per-port in `Port.Service`. +- **Device-type classifier** — heuristic rules over (vendor, OS banner, open ports) tag hosts as printer / router / hypervisor / windows-host / windows-server / windows-dc / nas / database (mysql|postgres|…) / mail-server / dns-server / kubernetes-node / container-host / camera / linux-host / appliance / iot-broker / embedded. +- **MAC + vendor enrichment** — neighbour-cache lookup on Linux (`/proc/net/arp`) and macOS (routing socket, no shell) + embedded OUI prefix table for ~90 common vendors, including major IP-camera (Hikvision/Dahua/Axis), NAS (Synology/QNAP/WD), networking (Ubiquiti), and IoT (Espressif) brands that also drive device classification. - **Per-subnet scan profiles** — aggressive hourly deep scans on critical infra, lazy daily liveness on guest networks, all in one config. - **Change detection + alerts** — diffs host inventory each cycle; fires `host.discovered` / `host.vanished` events to HTTP webhook and/or RFC 5424 syslog. -- **JSON query API** — `/api/v1/hosts` with filters (vendor, device type, hostname, subnet, port) and pagination; `/api/v1/hosts/{ip}` with nested ports. +- **JSON query API** — `/api/v1/hosts` with filters (vendor, device type, hostname, subnet, port) and pagination; `/api/v1/hosts/{ip}` with nested ports; `/api/v1/scans` paginated scan history (optional `subnet` filter). - **Continuous monitoring** — periodic re-scans detect new devices, removed devices, and configuration changes over time. - **Mutual watchdog** — two agent instances cross-check each other for liveness, scan freshness, and inventory consistency. Optional mTLS between peers. - **Web admin console** — dark-themed browser UI with dashboard, host inventory, per-host port detail, scan history, watchdog peer status; auto-starts alongside each agent. @@ -241,9 +241,9 @@ Each agent automatically starts a browser-based admin console alongside the scan | Page | URL | Description | |------|-----|-------------| | Dashboard | `/` | Summary cards and latest 10 scans and hosts; auto-refreshes every 30 s | -| Host inventory | `/hosts` | Full list of all discovered hosts with metadata | +| Host inventory | `/hosts` | List of discovered hosts with metadata; paginated (`?limit=`, `?offset=`, default 100) | | Host detail | `/hosts/{ip}` | Per-host metadata and open port table | -| Scan history | `/scans` | All subnet sweeps with duration and status | +| Scan history | `/scans` | Subnet sweeps with duration and status; paginated (`?limit=`, `?offset=`, default 100) | ### Terminal UI console @@ -337,15 +337,16 @@ Each agent reads a JSON config file and then applies environment variable overri | `scanner.subnets` | `[]` | Legacy flat CIDR list. Mutually exclusive with `scanner.profiles`. | | `scanner.profiles` | `[]` | Per-subnet override list (see below). | | `scanner.scan_interval` | `5m` | How often to re-scan; default for any profile that doesn't set its own. | -| `scanner.timeout` | `2s` | Per-host TCP probe timeout. | +| `scanner.timeout` | `2s` | Per-host TCP probe timeout; also bounds reverse-DNS (PTR) lookups. | | `scanner.workers` | `50` | GLOBAL concurrent probe cap across every subnet (not per-subnet). | | `scanner.max_hosts` | `65535` | Maximum usable addresses per subnet; larger subnets are rejected. | | `scanner.probe_ports` | `[22, 80, 443, 8080]` | TCP liveness ports — host alive if any answer. | | `scanner.deep_probe` | `false` | Second-pass scan of `deep_probe_ports` on every live host. | | `scanner.deep_probe_ports` | `top-services list` | TCP ports for the deep pass when `deep_probe` is on. | | `scanner.udp_ports` | `[]` | UDP ports to probe per live host. Empty disables UDP probing. | -| `scanner.enrich_arp` | `false` | Populate Host.MACAddress + Vendor from `/proc/net/arp` (Linux). | +| `scanner.enrich_arp` | `false` | Populate Host.MACAddress + Vendor from the OS neighbour cache (Linux `/proc/net/arp`, macOS routing socket). No-op on other platforms. | | `scanner.host_ttl` | `0` (disabled) | Hosts not seen within this duration are deleted at the end of each cycle. | +| `scanner.scan_history_ttl` | `0` (disabled) | Scan-history rows older than this duration are deleted at the end of each cycle, bounding the `scans` table and `/scans` view. | | **Scanner — per-subnet profile (each item in `scanner.profiles`)** | | | | `subnet` | required | CIDR for this profile. Must be unique. | | `scan_interval` | inherits global | Per-profile scan cadence. | @@ -366,6 +367,7 @@ Each agent reads a JSON config file and then applies environment variable overri | `health.client_ca_path` | — | When set, requires mTLS (clients must present a cert signed by this CA). | | **Admin console** | | | | `admin.addr` | `127.0.0.1:9090` | Listen address for the admin console + `/api/v1/*`. | +| `admin.auth_token` | — | Shared secret gating the whole console. Required when `admin.addr` is off-loopback. Clients send `Authorization: Bearer ` or HTTP Basic with the token as the password. | | **Watchdog** | | | | `watchdog.peer_addr` | — | Base URL of the partner agent's health server. | | `watchdog.peer_token` | — | Bearer token sent to the peer. Must match peer's `health.auth_token`. | @@ -379,9 +381,9 @@ Each agent reads a JSON config file and then applies environment variable overri | **Tracing** | | | | `tracing.endpoint` | — | OTLP/HTTP collector URL. Empty = no-op exporter (instrumentation active, spans discarded). | | **Alerts** | | | -| `alerts.webhook.url` | — | HTTP POST target for host.discovered / host.vanished events. | +| `alerts.webhook.url` | — | HTTP POST target for host.discovered / host.vanished events. Must be `http`/`https`; scheme-validated at startup. | | `alerts.webhook.auth_header` | — | Verbatim `Authorization` header (e.g. `Bearer abc123`). | -| `alerts.syslog.addr` | — | `udp://host:514` or `tcp://host:514`. RFC 5424. | +| `alerts.syslog.addr` | — | `udp://host:514` or `tcp://host:514`. RFC 5424. Scheme-validated at startup. | | `alerts.syslog.tag` | `network-inventory` | APP-NAME field. | | `alerts.syslog.facility` | `16` (local0) | RFC 5424 facility number 0..23. | @@ -421,6 +423,7 @@ fails fast if both are set. | `INVENTORY_ADMIN_ADDR` | `admin.addr` | | `INVENTORY_AUTH_TOKEN` | `health.auth_token` | | `INVENTORY_PEER_TOKEN` | `watchdog.peer_token` | +| `INVENTORY_ADMIN_TOKEN` | `admin.auth_token` | ## Health endpoints @@ -434,19 +437,20 @@ Both agents expose two HTTP endpoints used by the watchdog and for external moni | `/status` | GET | JSON-encoded status snapshot (see below) | | `/metrics` | GET | Prometheus text exposition format — counters for scans, probes, DB, watchdog, alerts; gauges for host count + peer-up state | -**Admin console** (default `127.0.0.1:9090`, unauthenticated — keep loopback unless on a trusted segment): +**Admin console** (default `127.0.0.1:9090`). Unauthenticated on the loopback default; set `admin.auth_token` (or `INVENTORY_ADMIN_TOKEN`) to gate every route below. A token is **required** when binding off-loopback — the agent refuses to start otherwise. Authenticate with `Authorization: Bearer ` or HTTP Basic auth using the token as the password (browsers get a native login prompt): | Endpoint | Method | Response | |----------|--------|----------| | `/` | GET | HTML dashboard | -| `/hosts` | GET | HTML host inventory | +| `/hosts` | GET | HTML host inventory (paginated: `?limit=`, `?offset=`) | | `/hosts/{ip}` | GET | HTML host detail (with ports) | -| `/scans` | GET | HTML scan history | +| `/scans` | GET | HTML scan history (paginated: `?limit=`, `?offset=`) | | `/watchdog` | GET | HTML watchdog peer-status panel | | `/export.json` | GET | Full inventory snapshot as JSON | | `/export.csv` | GET | Full inventory snapshot as CSV | | `/api/v1/hosts` | GET | Filterable JSON list — `?vendor=`, `?device_type=`, `?hostname=`, `?subnet=`, `?port=`, `?limit=`, `?offset=` | | `/api/v1/hosts/{ip}` | GET | Single-host JSON with nested ports | +| `/api/v1/scans` | GET | Paginated JSON scan history — `?subnet=`, `?limit=`, `?offset=` | | `/scan` | POST | Trigger an out-of-cycle scan (CSRF-gated) | ### `/status` response diff --git a/SECURITY.md b/SECURITY.md index afd0cf8..acac634 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -43,16 +43,16 @@ The following table documents the project's posture against the [OWASP Top 10 (2 | # | Category | Status | Notes | |---|----------|--------|-------| -| A01 | Broken Access Control | ⚠️ Partial | `/health` and `/status` are intentionally unauthenticated for simplicity. The default bind address is `127.0.0.1` (loopback only). Operators who expose these endpoints on a wider interface accept responsibility for network-level access control. | +| A01 | Broken Access Control | ⚠️ Partial | `/health` and `/status` are unauthenticated on the loopback default but require `health.auth_token` when bound off-loopback (enforced at startup). The admin console (full inventory, exports, JSON API, `POST /scan`) likewise requires `admin.auth_token` when bound off-loopback — the agent refuses to start otherwise. Both default to `127.0.0.1` (loopback only). | | A02 | Cryptographic Failures | ✅ Pass | Peer-to-peer watchdog traffic supports TLS (with optional mTLS) — set `watchdog.tls.ca_cert_path` and `health.tls_cert_path`/`tls_key_path` in the configs. TLS 1.2+ enforced. Database is stored unencrypted; operators should apply filesystem-level encryption where needed. | | A03 | Injection | ✅ Pass | All SQL queries use parameterized `?` placeholders. No shell commands are invoked; the scanner uses `net.Dialer` directly. | | A04 | Insecure Design | ✅ Pass | Health server binds to loopback by default. `peer_addr` is validated to `http`/`https` schemes only, preventing SSRF via alternate URI schemes. No user-controlled input reaches internal APIs without validation. | | A05 | Security Misconfiguration | ✅ Pass | Default `health.addr` is `127.0.0.1:8080` (loopback only). HTTP server has explicit read, write, and idle timeouts. Response bodies from peers are capped at 1 MiB. | | A06 | Vulnerable Components | ✅ Pass | All dependencies are pure Go (no C libraries). `go.sum` is committed and verified on every build. `govulncheck` is required before dependency PRs (see CONTRIBUTING.md). | -| A07 | Auth Failures | ⚠️ Partial | No authentication on health endpoints by design. Mitigated by loopback-only default and operator guidance in this document. | +| A07 | Auth Failures | ⚠️ Partial | Loopback-only defaults are unauthenticated by design. Off-loopback binds of both the health server and the admin console require a shared bearer/Basic token, enforced at startup; tokens are compared in constant time. | | A08 | Data Integrity | ✅ Pass | `go.sum` provides cryptographic verification of all module downloads. Config validation rejects malformed or unexpected values at startup. | | A09 | Logging & Monitoring | ✅ Pass | Structured `log/slog` output in text or JSON format. All three watchdog failure conditions (liveness, freshness, consistency) are logged at `WARN` or `ERROR` level with structured fields. | -| A10 | SSRF | ✅ Pass | `peer_addr` is validated to `http`/`https` only at config load time. Response bodies from external HTTP calls are limited to 1 MiB via `io.LimitReader`. Scanner targets come from operator-controlled config, not external input. | +| A10 | SSRF | ✅ Pass | All outbound targets are scheme-validated at config load: `watchdog.peer_addr` and `alerts.webhook.url` to `http`/`https`, `alerts.syslog.addr` to `udp`/`tcp`. This blocks scheme-confusion vectors (`file://`, `gopher://`, …) before the URL reaches a client. Response bodies from external HTTP calls are limited to 1 MiB via `io.LimitReader`. Scanner targets come from operator-controlled config, not external input. | ## OWASP AI Top 10 @@ -62,7 +62,9 @@ The OWASP AI Top 10 is **not applicable** to this project. NetworkInventoryAgent NetworkInventoryAgent is designed to run on a trusted internal network. Before deploying, consider the following: -**Health endpoints are unauthenticated.** The `/health` and `/status` endpoints expose agent name, scan counts, host counts, and timestamps to anyone who can reach the listening address. The default bind address is `127.0.0.1` (loopback only). Do not change this to `0.0.0.0` unless the network segment is trusted or access is controlled at the firewall. +**Health endpoints are unauthenticated.** The `/health` and `/status` endpoints expose agent name, scan counts, host counts, and timestamps to anyone who can reach the listening address. The default bind address is `127.0.0.1` (loopback only). Binding off-loopback requires `health.auth_token` (or `INVENTORY_AUTH_TOKEN`); the agent refuses to start without it. + +**The admin console is gated off-loopback.** The console at `admin.addr` (default `127.0.0.1:9090`) serves the full host/port inventory, `/export.json|csv`, the `/api/v1/*` query API, and the `POST /scan` trigger. On the loopback default it is unauthenticated for convenience; binding it off-loopback (e.g. `0.0.0.0` for Docker, or via `INVENTORY_ADMIN_ADDR`) requires `admin.auth_token` (or `INVENTORY_ADMIN_TOKEN`) and the agent refuses to start without it. Clients authenticate with `Authorization: Bearer ` or HTTP Basic auth using the token as the password. **Peer communication can use TLS.** Watchdog checks between Wintermute and Neuromancer default to plain HTTP for the loopback case. For off-loopback deployments, switch `watchdog.peer_addr` to `https://…`, set `watchdog.tls.ca_cert_path` to the CA that signs the peer's cert, and set `health.tls_cert_path` / `health.tls_key_path` on the peer. For full mutual auth, set `health.client_ca_path` on both sides and `watchdog.tls.client_cert_path` / `client_key_path` on the dialer side. Bearer tokens stack on top of TLS. diff --git a/cmd/internal/runtime/runtime.go b/cmd/internal/runtime/runtime.go index ce77024..8002fc2 100644 --- a/cmd/internal/runtime/runtime.go +++ b/cmd/internal/runtime/runtime.go @@ -148,6 +148,7 @@ func Run(opts Options) int { cfg.Admin.Addr, opts.Name, db.Hosts(), db.Ports(), db.Scans(), tracker.Get, a.Trigger, + admin.ServerOptions{AuthToken: cfg.Admin.AuthToken}, ) if err != nil { slog.Error("failed to create admin server", "err", err) diff --git a/go.mod b/go.mod index c6f46e2..ea10074 100644 --- a/go.mod +++ b/go.mod @@ -11,6 +11,8 @@ require ( go.opentelemetry.io/otel v1.43.0 go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 go.opentelemetry.io/otel/sdk v1.43.0 + go.opentelemetry.io/otel/trace v1.43.0 + golang.org/x/net v0.52.0 modernc.org/sqlite v1.50.0 ) @@ -48,9 +50,7 @@ require ( go.opentelemetry.io/auto/sdk v1.2.1 // indirect go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect go.opentelemetry.io/otel/metric v1.43.0 // indirect - go.opentelemetry.io/otel/trace v1.43.0 // indirect go.opentelemetry.io/proto/otlp v1.10.0 // indirect - golang.org/x/net v0.52.0 // indirect golang.org/x/sys v0.42.0 // indirect golang.org/x/text v0.35.0 // indirect google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect diff --git a/internal/admin/api.go b/internal/admin/api.go index 8eeb954..e256563 100644 --- a/internal/admin/api.go +++ b/internal/admin/api.go @@ -49,6 +49,15 @@ type hostDetailResponse struct { Ports []*models.Port `json:"ports"` } +// scansResponse is the JSON envelope for GET /api/v1/scans, mirroring +// hostsResponse. Scans are newest-first (as the store returns them). +type scansResponse struct { + Total int `json:"total"` + Limit int `json:"limit"` + Offset int `json:"offset"` + Scans []*models.Scan `json:"scans"` +} + // apiError sets standard headers and writes a JSON error body. The status // codes follow the standard REST conventions: 400 for bad input, 404 for // unknown resource, 500 for backend failure. @@ -128,6 +137,54 @@ func (s *Server) handleAPIHosts(w http.ResponseWriter, r *http.Request) { } } +// handleAPIScans returns scan history as paginated JSON. Optional `subnet` +// query param exact-matches Scan.Subnet. Mirrors handleAPIHosts. +func (s *Server) handleAPIScans(w http.ResponseWriter, r *http.Request) { + q := r.URL.Query() + limit, offset, err := parsePagination(q) + if err != nil { + apiError(w, http.StatusBadRequest, err.Error()) + return + } + + all, err := s.scans.List(r.Context()) + if err != nil { + slog.Error("api list scans", "err", err) + apiError(w, http.StatusInternalServerError, "list scans failed") + return + } + + matched := all + if subnet := q.Get("subnet"); subnet != "" { + matched = make([]*models.Scan, 0, len(all)) + for _, sc := range all { + if sc.Subnet == subnet { + matched = append(matched, sc) + } + } + } + + total := len(matched) + start := offset + if start > total { + start = total + } + end := start + limit + if end > total { + end = total + } + + w.Header().Set("Content-Type", "application/json") + if err := json.NewEncoder(w).Encode(scansResponse{ + Total: total, + Limit: limit, + Offset: offset, + Scans: matched[start:end], + }); err != nil { + slog.Error("api encode scans", "err", err) + } +} + func (s *Server) handleAPIHostDetail(w http.ResponseWriter, r *http.Request) { ip := r.PathValue("ip") host, err := s.hosts.GetByIP(r.Context(), ip) diff --git a/internal/admin/api_test.go b/internal/admin/api_test.go index 8b0149e..10cbd7d 100644 --- a/internal/admin/api_test.go +++ b/internal/admin/api_test.go @@ -249,3 +249,64 @@ func TestAPIHostDetail_NotFound(t *testing.T) { assert.Equal(t, http.StatusNotFound, resp.StatusCode) _ = resp.Body.Close() } + +func fixtureScans() []*models.Scan { + now := time.Now() + return []*models.Scan{ + {ID: 1, Subnet: "10.0.0.0/24", HostsFound: 3, StartedAt: now}, + {ID: 2, Subnet: "10.0.1.0/24", HostsFound: 1, StartedAt: now}, + {ID: 3, Subnet: "10.0.0.0/24", HostsFound: 4, StartedAt: now}, + } +} + +type scansBody struct { + Total int `json:"total"` + Limit int `json:"limit"` + Offset int `json:"offset"` + Scans []*models.Scan `json:"scans"` +} + +func TestAPIScans_ListsAllPaginated(t *testing.T) { + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: fixtureScans()}) + resp := get(t, srv, "/api/v1/scans") + defer func() { _ = resp.Body.Close() }() + require.Equal(t, http.StatusOK, resp.StatusCode) + + var body scansBody + decodeJSON(t, resp, &body) + assert.Equal(t, 3, body.Total) + assert.Equal(t, 100, body.Limit) + assert.Len(t, body.Scans, 3) +} + +func TestAPIScans_Pagination(t *testing.T) { + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: fixtureScans()}) + resp := get(t, srv, "/api/v1/scans?limit=2&offset=0") + defer func() { _ = resp.Body.Close() }() + + var body scansBody + decodeJSON(t, resp, &body) + assert.Equal(t, 3, body.Total, "total reflects the full set, not the page") + assert.Equal(t, 2, body.Limit) + assert.Len(t, body.Scans, 2) +} + +func TestAPIScans_SubnetFilter(t *testing.T) { + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: fixtureScans()}) + resp := get(t, srv, "/api/v1/scans?subnet=10.0.0.0/24") + defer func() { _ = resp.Body.Close() }() + + var body scansBody + decodeJSON(t, resp, &body) + assert.Equal(t, 2, body.Total, "only the two 10.0.0.0/24 scans match") + for _, sc := range body.Scans { + assert.Equal(t, "10.0.0.0/24", sc.Subnet) + } +} + +func TestAPIScans_InvalidLimit(t *testing.T) { + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: fixtureScans()}) + resp := get(t, srv, "/api/v1/scans?limit=0") + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusBadRequest, resp.StatusCode) +} diff --git a/internal/admin/auth_test.go b/internal/admin/auth_test.go new file mode 100644 index 0000000..438602d --- /dev/null +++ b/internal/admin/auth_test.go @@ -0,0 +1,115 @@ +package admin_test + +import ( + "context" + "net/http" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/Ronin48/NetworkInventoryAgent/internal/admin" +) + +const testAdminToken = "s3cr3t-admin-token" + +// newAuthedServer starts an admin server gated by testAdminToken. A non-nil +// trigger is wired so POST /scan exercises the auth gate (which must run before +// the CSRF check). +func newAuthedServer(t *testing.T) *admin.Server { + t.Helper() + srv, err := admin.NewServer(":0", "test-agent", + &mockHostStore{}, &mockPortStore{}, &mockScanStore{}, + healthyStatus, func() bool { return true }, + admin.ServerOptions{AuthToken: testAdminToken}, + ) + require.NoError(t, err) + require.NoError(t, srv.Start()) + t.Cleanup(func() { + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + _ = srv.Shutdown(ctx) + }) + return srv +} + +// doReq issues a request to the server with optional mutation (headers/auth). +func doReq(t *testing.T, srv *admin.Server, method, path string, mut func(*http.Request)) *http.Response { + t.Helper() + req, err := http.NewRequest(method, "http://"+srv.Addr()+path, nil) + require.NoError(t, err) + if mut != nil { + mut(req) + } + resp, err := http.DefaultClient.Do(req) + require.NoError(t, err) + return resp +} + +func TestAuth_NoCredentials_401WithChallenge(t *testing.T) { + srv := newAuthedServer(t) + resp := doReq(t, srv, http.MethodGet, "/", nil) + defer func() { _ = resp.Body.Close() }() + + assert.Equal(t, http.StatusUnauthorized, resp.StatusCode) + assert.Contains(t, resp.Header.Get("WWW-Authenticate"), "Basic") +} + +func TestAuth_CorrectBearer_200(t *testing.T) { + srv := newAuthedServer(t) + resp := doReq(t, srv, http.MethodGet, "/", func(r *http.Request) { + r.Header.Set("Authorization", "Bearer "+testAdminToken) + }) + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusOK, resp.StatusCode) +} + +func TestAuth_CorrectBasicPassword_200(t *testing.T) { + srv := newAuthedServer(t) + resp := doReq(t, srv, http.MethodGet, "/", func(r *http.Request) { + r.SetBasicAuth("anyuser", testAdminToken) + }) + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusOK, resp.StatusCode) +} + +func TestAuth_WrongBearer_401(t *testing.T) { + srv := newAuthedServer(t) + resp := doReq(t, srv, http.MethodGet, "/", func(r *http.Request) { + r.Header.Set("Authorization", "Bearer wrong") + }) + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusUnauthorized, resp.StatusCode) +} + +func TestAuth_WrongBasicPassword_401(t *testing.T) { + srv := newAuthedServer(t) + resp := doReq(t, srv, http.MethodGet, "/", func(r *http.Request) { + r.SetBasicAuth("anyuser", "wrong") + }) + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusUnauthorized, resp.StatusCode) +} + +// Export and the scan trigger must be behind the same gate. The unauthenticated +// POST /scan must be rejected by auth (401) before CSRF runs (403). +func TestAuth_GatesExportAndScanTrigger(t *testing.T) { + srv := newAuthedServer(t) + + exp := doReq(t, srv, http.MethodGet, "/export.json", nil) + _ = exp.Body.Close() + assert.Equal(t, http.StatusUnauthorized, exp.StatusCode, "export must require auth") + + scan := doReq(t, srv, http.MethodPost, "/scan", nil) + _ = scan.Body.Close() + assert.Equal(t, http.StatusUnauthorized, scan.StatusCode, "auth must precede CSRF on POST /scan") +} + +// Regression guard: the loopback default (no token) stays credential-free. +func TestAuth_EmptyToken_NoGate(t *testing.T) { + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{}) + resp := doReq(t, srv, http.MethodGet, "/", nil) + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusOK, resp.StatusCode) +} diff --git a/internal/admin/handlers.go b/internal/admin/handlers.go index cdf9cdf..061362b 100644 --- a/internal/admin/handlers.go +++ b/internal/admin/handlers.go @@ -52,9 +52,15 @@ func (s *Server) handleHosts(w http.ResponseWriter, r *http.Request) { http.Error(w, "failed to load hosts", http.StatusInternalServerError) return } + limit, offset, err := parsePagination(r.URL.Query()) + if err != nil { + http.Error(w, err.Error(), http.StatusBadRequest) + return + } s.render(w, "hosts", hostsData{ pageData: s.basePage("Hosts"), - Hosts: hosts, + Hosts: pageSlice(hosts, offset, limit), + Pager: newPager("/hosts", len(hosts), limit, offset), }) } @@ -88,9 +94,15 @@ func (s *Server) handleScans(w http.ResponseWriter, r *http.Request) { http.Error(w, "failed to load scans", http.StatusInternalServerError) return } + limit, offset, err := parsePagination(r.URL.Query()) + if err != nil { + http.Error(w, err.Error(), http.StatusBadRequest) + return + } s.render(w, "scans", scansData{ pageData: s.basePage("Scans"), - Scans: scans, + Scans: pageSlice(scans, offset, limit), + Pager: newPager("/scans", len(scans), limit, offset), }) } diff --git a/internal/admin/handlers_extra_test.go b/internal/admin/handlers_extra_test.go new file mode 100644 index 0000000..36afe1a --- /dev/null +++ b/internal/admin/handlers_extra_test.go @@ -0,0 +1,164 @@ +package admin_test + +import ( + "context" + "fmt" + "net/http" + "regexp" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/Ronin48/NetworkInventoryAgent/internal/admin" + "github.com/Ronin48/NetworkInventoryAgent/internal/health" + "github.com/Ronin48/NetworkInventoryAgent/models" +) + +// newServerWithTrigger starts an admin server wired with the given trigger. +func newServerWithTrigger(t *testing.T, trigger admin.Trigger, status func() health.Status) *admin.Server { + t.Helper() + srv, err := admin.NewServer(":0", "test-agent", + &mockHostStore{}, &mockPortStore{}, &mockScanStore{}, + status, trigger, admin.ServerOptions{}, + ) + require.NoError(t, err) + require.NoError(t, srv.Start()) + t.Cleanup(func() { + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + _ = srv.Shutdown(ctx) + }) + return srv +} + +var csrfRe = regexp.MustCompile(`name="csrf" value="([0-9a-f]+)"`) + +// scrapeCSRF GETs the dashboard and extracts the embedded CSRF token. +func scrapeCSRF(t *testing.T, srv *admin.Server) string { + t.Helper() + resp := get(t, srv, "/") + defer func() { _ = resp.Body.Close() }() + m := csrfRe.FindStringSubmatch(readBody(t, resp)) + require.Len(t, m, 2, "dashboard should embed a CSRF token") + return m[1] +} + +func postScan(t *testing.T, srv *admin.Server, csrf string) *http.Response { + t.Helper() + req, err := http.NewRequest(http.MethodPost, "http://"+srv.Addr()+"/scan", nil) + require.NoError(t, err) + if csrf != "" { + req.Header.Set("X-CSRF-Token", csrf) + } + resp, err := http.DefaultClient.Do(req) + require.NoError(t, err) + return resp +} + +func TestHandleWatchdog_NoPeer(t *testing.T) { + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{}) + resp := get(t, srv, "/watchdog") + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusOK, resp.StatusCode) + assert.Contains(t, resp.Header.Get("Content-Type"), "text/html") +} + +func TestHandleWatchdog_WithPeer(t *testing.T) { + status := func() health.Status { + return health.Status{ + Name: "test-agent", + Healthy: true, + Peer: &health.PeerStatus{ + Addr: "http://neuromancer:8081", + Reachable: true, + LastCheckedAt: time.Now(), + PeerHostCount: 12, + }, + } + } + srv := newServerWithTrigger(t, nil, status) + resp := get(t, srv, "/watchdog") + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusOK, resp.StatusCode) + assert.Contains(t, readBody(t, resp), "neuromancer") +} + +func TestHandleScanTrigger_NotWired(t *testing.T) { + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{}) // nil trigger + resp := postScan(t, srv, scrapeCSRF(t, srv)) + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusNotImplemented, resp.StatusCode) +} + +func TestHandleScanTrigger_Success(t *testing.T) { + srv := newServerWithTrigger(t, func() bool { return true }, healthyStatus) + resp := postScan(t, srv, scrapeCSRF(t, srv)) + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusNoContent, resp.StatusCode) +} + +func TestHandleScanTrigger_AlreadyPending(t *testing.T) { + srv := newServerWithTrigger(t, func() bool { return false }, healthyStatus) + resp := postScan(t, srv, scrapeCSRF(t, srv)) + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusServiceUnavailable, resp.StatusCode) +} + +func TestHandleScanTrigger_MissingCSRF(t *testing.T) { + srv := newServerWithTrigger(t, func() bool { return true }, healthyStatus) + resp := postScan(t, srv, "") // no token + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusForbidden, resp.StatusCode, "POST without CSRF token must be rejected") +} + +func TestHandleHosts_Pagination(t *testing.T) { + var hosts []*models.Host + for i := 1; i <= 5; i++ { + hosts = append(hosts, &models.Host{ + ID: int64(i), IPAddress: fmt.Sprintf("10.0.0.%d", i), LastSeen: time.Now(), + }) + } + srv := newTestServer(t, &mockHostStore{hosts: hosts}, &mockPortStore{}, &mockScanStore{}) + + // First page: two rows, a Next link, the true total in the subtitle. + resp := get(t, srv, "/hosts?limit=2&offset=0") + defer func() { _ = resp.Body.Close() }() + body := readBody(t, resp) + assert.Equal(t, http.StatusOK, resp.StatusCode) + assert.Contains(t, body, "of 5", "subtitle/pager should report the full total") + assert.Contains(t, body, "10.0.0.1") + assert.Contains(t, body, "10.0.0.2") + assert.NotContains(t, body, "10.0.0.3", "row beyond the page must not render") + assert.Contains(t, body, "/hosts?limit=2&offset=2", "Next link to the second page") + + // Last page: final row, no Next target. + resp2 := get(t, srv, "/hosts?limit=2&offset=4") + defer func() { _ = resp2.Body.Close() }() + body2 := readBody(t, resp2) + assert.Contains(t, body2, "10.0.0.5") + assert.Contains(t, body2, "/hosts?limit=2&offset=2", "Prev link back to page two") + assert.NotContains(t, body2, "offset=6", "no Next link past the end") +} + +func TestHandleHosts_InvalidLimit(t *testing.T) { + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{}) + resp := get(t, srv, "/hosts?limit=0") + defer func() { _ = resp.Body.Close() }() + assert.Equal(t, http.StatusBadRequest, resp.StatusCode) +} + +func TestHandleScans_Pagination(t *testing.T) { + var scans []*models.Scan + for i := 1; i <= 3; i++ { + scans = append(scans, &models.Scan{ID: int64(i), Subnet: fmt.Sprintf("10.0.%d.0/24", i), StartedAt: time.Now()}) + } + srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: scans}) + resp := get(t, srv, "/scans?limit=2&offset=0") + defer func() { _ = resp.Body.Close() }() + body := readBody(t, resp) + assert.Equal(t, http.StatusOK, resp.StatusCode) + assert.Contains(t, body, "of 3") + assert.Contains(t, body, "/scans?limit=2&offset=2", "Next link present") +} diff --git a/internal/admin/middleware.go b/internal/admin/middleware.go index be2400c..ec37235 100644 --- a/internal/admin/middleware.go +++ b/internal/admin/middleware.go @@ -4,13 +4,16 @@ import ( "crypto/subtle" "log/slog" "net/http" + "strings" "time" ) -// middleware wraps the mux with three cross-cutting concerns: +// middleware wraps the mux with four cross-cutting concerns: // - per-request access logging (one slog record per response) // - baseline security headers (defence-in-depth for the loopback console; // non-trivial once operators bind it to 0.0.0.0) +// - shared-secret authentication (no-op when no token is configured, so the +// loopback default stays credential-free) // - CSRF protection on state-changing methods (POST/PUT/PATCH/DELETE) // // CSP keeps 'unsafe-inline' for styles because the templates embed a single @@ -31,6 +34,12 @@ func (s *Server) middleware(next http.Handler) http.Handler { "form-action 'self'; "+ "frame-ancestors 'none'") + if !s.checkAuth(r) { + w.Header().Set("WWW-Authenticate", `Basic realm="inventory-admin"`) + http.Error(w, "unauthorized", http.StatusUnauthorized) + return + } + if !s.checkCSRF(r) { http.Error(w, "csrf token mismatch", http.StatusForbidden) return @@ -70,6 +79,29 @@ func (s *Server) checkCSRF(r *http.Request) bool { return subtle.ConstantTimeCompare([]byte(got), []byte(s.csrfToken)) == 1 } +// checkAuth enforces the shared-secret gate on every request. It is a no-op +// when the server was constructed without an auth token, so the loopback +// default needs no credentials. +// +// The console is a browser UI, so two credential carriers are accepted: a +// `Authorization: Bearer ` header (curl, the JSON API, exports) and +// HTTP Basic auth with the token as the password and any username (so a +// browser shows its native login dialog). Both are compared in constant time +// to deny timing-based bisection of the token. +func (s *Server) checkAuth(r *http.Request) bool { + if s.authToken == "" { + return true + } + want := []byte(s.authToken) + if got, ok := strings.CutPrefix(r.Header.Get("Authorization"), "Bearer "); ok { + return subtle.ConstantTimeCompare([]byte(got), want) == 1 + } + if _, pass, ok := r.BasicAuth(); ok { + return subtle.ConstantTimeCompare([]byte(pass), want) == 1 + } + return false +} + // statusRecorder lets the access-log middleware capture the response status // that handlers chose. Defaults to 200 because http.ResponseWriter only // records WriteHeader when an explicit status is set. diff --git a/internal/admin/render.go b/internal/admin/render.go index 8fe153b..2ca62b9 100644 --- a/internal/admin/render.go +++ b/internal/admin/render.go @@ -59,6 +59,7 @@ type dashboardData struct { type hostsData struct { pageData Hosts []*models.Host + Pager pager } type hostDetailData struct { @@ -70,6 +71,60 @@ type hostDetailData struct { type scansData struct { pageData Scans []*models.Scan + Pager pager +} + +// pager carries pagination state for list templates. PagePath is the route +// the prev/next links target (e.g. "/hosts"). From/To are 1-based indices of +// the rows shown (both 0 when the page is empty). +type pager struct { + PagePath string + Total int + Limit int + Offset int + From int + To int + HasPrev bool + HasNext bool + PrevOffset int + NextOffset int +} + +// newPager computes display + link state for a window [offset, offset+limit) +// over a list of `total` items. +func newPager(path string, total, limit, offset int) pager { + p := pager{PagePath: path, Total: total, Limit: limit, Offset: offset} + if offset < total { + p.From = offset + 1 + end := offset + limit + if end > total { + end = total + } + p.To = end + } + if offset > 0 { + p.HasPrev = true + if p.PrevOffset = offset - limit; p.PrevOffset < 0 { + p.PrevOffset = 0 + } + } + if offset+limit < total { + p.HasNext = true + p.NextOffset = offset + limit + } + return p +} + +// pageSlice returns the [offset, offset+limit) window of s, clamped to bounds. +func pageSlice[T any](s []T, offset, limit int) []T { + if offset >= len(s) { + return nil + } + end := offset + limit + if end > len(s) { + end = len(s) + } + return s[offset:end] } type watchdogData struct { diff --git a/internal/admin/server.go b/internal/admin/server.go index 1743900..5a67d2c 100644 --- a/internal/admin/server.go +++ b/internal/admin/server.go @@ -30,6 +30,16 @@ var templateFS embed.FS // already pending. Pass a nil Trigger to omit the endpoint. type Trigger func() bool +// ServerOptions carries the optional configuration for NewServer. The zero +// value produces the loopback-friendly default: no auth, so a browser or curl +// reaches the console without credentials. +type ServerOptions struct { + // AuthToken, when non-empty, gates every route. Clients present it as + // `Authorization: Bearer ` or via HTTP Basic auth using the token + // as the password (any username). Mismatches return 401 in constant time. + AuthToken string +} + // Server is the admin web console HTTP server. type Server struct { agentName string @@ -39,6 +49,7 @@ type Server struct { status func() health.Status trigger Trigger csrfToken string + authToken string srv *http.Server tmpl *template.Template } @@ -54,6 +65,7 @@ func NewServer( scans store.ScanStore, status func() health.Status, trigger Trigger, + opts ServerOptions, ) (*Server, error) { tmpl, err := template.New("").Funcs(funcMap).ParseFS(templateFS, "templates/*.html") if err != nil { @@ -73,6 +85,7 @@ func NewServer( status: status, trigger: trigger, csrfToken: csrf, + authToken: opts.AuthToken, tmpl: tmpl, } @@ -90,6 +103,7 @@ func NewServer( // future v2 changes can land alongside without breaking consumers. mux.HandleFunc("GET /api/v1/hosts", s.handleAPIHosts) mux.HandleFunc("GET /api/v1/hosts/{ip}", s.handleAPIHostDetail) + mux.HandleFunc("GET /api/v1/scans", s.handleAPIScans) s.srv = &http.Server{ Addr: addr, diff --git a/internal/admin/server_test.go b/internal/admin/server_test.go index 03d6298..fa8ad77 100644 --- a/internal/admin/server_test.go +++ b/internal/admin/server_test.go @@ -129,6 +129,25 @@ func (m *mockScanStore) List(_ context.Context) ([]*models.Scan, error) { return m.scans, m.err } +func (m *mockScanStore) DeleteBefore(_ context.Context, cutoff time.Time) (int64, error) { + m.mu.Lock() + defer m.mu.Unlock() + if m.err != nil { + return 0, m.err + } + kept := m.scans[:0] + var deleted int64 + for _, s := range m.scans { + if s.StartedAt.Before(cutoff) { + deleted++ + continue + } + kept = append(kept, s) + } + m.scans = kept + return deleted, nil +} + // --- helpers --- func healthyStatus() health.Status { @@ -141,7 +160,7 @@ func healthyStatus() health.Status { func newTestServer(t *testing.T, hosts *mockHostStore, ports *mockPortStore, scans *mockScanStore) *admin.Server { t.Helper() - srv, err := admin.NewServer(":0", "test-agent", hosts, ports, scans, healthyStatus, nil) + srv, err := admin.NewServer(":0", "test-agent", hosts, ports, scans, healthyStatus, nil, admin.ServerOptions{}) require.NoError(t, err) require.NoError(t, srv.Start()) t.Cleanup(func() { @@ -162,7 +181,7 @@ func get(t *testing.T, srv *admin.Server, path string) *http.Response { // --- tests --- func TestNewServer_ParsesTemplates(t *testing.T) { - _, err := admin.NewServer(":0", "agent", &mockHostStore{}, &mockPortStore{}, &mockScanStore{}, healthyStatus, nil) + _, err := admin.NewServer(":0", "agent", &mockHostStore{}, &mockPortStore{}, &mockScanStore{}, healthyStatus, nil, admin.ServerOptions{}) require.NoError(t, err, "template parsing should succeed on a clean build") } @@ -373,7 +392,7 @@ func TestAllPages_ContentType(t *testing.T) { } func TestServer_Shutdown(t *testing.T) { - srv, err := admin.NewServer(":0", "agent", &mockHostStore{}, &mockPortStore{}, &mockScanStore{}, healthyStatus, nil) + srv, err := admin.NewServer(":0", "agent", &mockHostStore{}, &mockPortStore{}, &mockScanStore{}, healthyStatus, nil, admin.ServerOptions{}) require.NoError(t, err) require.NoError(t, srv.Start()) diff --git a/internal/admin/templates/base.html b/internal/admin/templates/base.html index c7b885d..eee0b51 100644 --- a/internal/admin/templates/base.html +++ b/internal/admin/templates/base.html @@ -61,10 +61,26 @@ .btn:hover{background:#2ea043} .btn-link{display:inline-block;background:#21262d;color:#c9d1d9;border:1px solid #30363d;border-radius:6px;padding:6px 12px;font-size:13px;font-weight:600;text-decoration:none} .btn-link:hover{background:#30363d;text-decoration:none} +.btn-link.disabled{opacity:.4;pointer-events:none} +.pager{display:flex;align-items:center;justify-content:space-between;margin-top:16px} +.pager-info{color:#8b949e;font-size:12px} +.pager-links{display:flex;gap:8px} {{end}} +{{define "pager"}} +{{if gt .Total .Limit}} +
+ Showing {{.From}}–{{.To}} of {{.Total}} + + {{if .HasPrev}}‹ Prev{{else}}‹ Prev{{end}} + {{if .HasNext}}Next ›{{else}}Next ›{{end}} + +
+{{end}} +{{end}} + {{define "nav"}}