feat(compliance): integrate compliance-engine-assessor scanning#8910
feat(compliance): integrate compliance-engine-assessor scanning#8910robertwoj-microsoft wants to merge 5 commits into
Conversation
robertwoj-microsoft
changed the title
robertwoj-microsoft
changed the title
There was a problem hiding this comment.
Pull request overview
Adds initial “shadow” integration of the benchmark-agnostic Compliance Engine Assessor into the VHD scanning flow. This wires the download/staging of the assessor + MOFs on the build agent, runs the assessor on the scan VM alongside the existing CIS-CAT flow, and publishes native JSON outputs as pipeline artifacts (non-blocking).
Changes:
- Add a non-blocking Compliance Engine scan phase to
vhdbuilder/packer/vhd-scanning.sh(runs before CIS-CAT and emits JSON + logs). - Introduce an ADO template to download assessor + benchmark MOFs from an Azure Artifacts NuGet feed based on a repo-pinned manifest.
- Publish Compliance Engine JSON/log outputs as build artifacts for Ubuntu 22.04/24.04 builds.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
vhdbuilder/packer/vhd-scanning.sh |
Runs the Compliance Engine assessor on the scan VM (shadow mode) and downloads JSON/log outputs for publication. |
vhdbuilder/packer/compliance-versions.json |
Adds PR-gated pins (currently placeholders) for assessor + benchmark packages consumed by the download template. |
vhdbuilder/packer/compliance-engine-report.sh |
New run-command script executed on the scan VM to download assessor+MOF, run audit, and upload JSON/log. |
.pipelines/templates/.compliance-engine-scan-template.yaml |
New shared ADO steps template to download assessor + MOF packages (manifest-driven) onto the build agent. |
.pipelines/templates/.builder-release-template.yaml |
Hooks the new download template into the build and publishes compliance-engine JSON/log artifacts. |
| echo "$ret" | ||
| msg=$(echo -E "$ret" | jq -r '.value[].message' 2>/dev/null || true) | ||
| echo "$msg" |
| #!/bin/bash | ||
| set -eux | ||
|
|
| if [ -z "$AZURE_MSI_RESOURCE_STRING" ]; then | ||
| echo "AZURE_MSI_RESOURCE_STRING must be set for az login" | ||
| exit 1 | ||
| fi |
| - task: CopyFiles@2 | ||
| condition: and(eq(variables.OS_SKU, 'Ubuntu'), in(variables.OS_VERSION, '22.04', '24.04')) | ||
| continueOnError: true | ||
| displayName: Copy Compliance Engine Reports | ||
| inputs: |
| "assessor": "compliance-engine-assessor@TODO-pin-published-assessor-version", | ||
| "benchmarks": { | ||
| "ubuntu-22.04-l1": "cis_ubuntu_linux_22.04_lts_benchmark_v2.0.0__level_1__server@TODO-pin-published-version", | ||
| "ubuntu-22.04-l2": "cis_ubuntu_linux_22.04_lts_benchmark_v2.0.0__level_2__server@TODO-pin-published-version", | ||
| "ubuntu-24.04-l1": "cis_ubuntu_linux_24.04_lts_benchmark_v1.0.0__level_1__server@TODO-pin-published-version", |
AgentBaker Linux gate detectiveRun: https://msazure.visualstudio.com/CloudNativeCompute/_build/results?buildId=171985153 Detective summary: Ubuntu 22.04 HTTPSProxy PrivateDNS Likely cause / signature: E2E infra/environment outbound connectivity flake; wiki signature Confidence: High for flaky infra/environment on the HTTPSProxy PrivateDNS failures; medium that ACL ARM64 is an adjacent latency symptom rather than the same root. Recommended owner/action: Node Lifecycle E2E infra owner should inspect scenario logs for the outbound exit 50 retries and cluster/API latency; no PR author action unless reruns produce deterministic compliance-scanning-specific failures. Strongest alternative: PR #8910 compliance scanning integration changed image/provisioning behavior; less likely because the log labels exit 50 as a known transient outbound flake and this exact HTTPSProxy PrivateDNS signature already exists in the tracker. Evidence: build summary, E2E log 571/578, test run 527254252 with 5 failed results, scenario logs Wiki signature: �2e-httpsproxy-privatedns-err-outbound-conn-fail |
| # TEMPORARY (testing): the ENABLE_COMPLIANCE_ENGINE_SCAN gate is removed so | ||
| # the scan runs on every Ubuntu 22.04/24.04 build without setting the | ||
| # variable. RESTORE before merge: | ||
| # and(succeeded(), eq(variables['ENABLE_COMPLIANCE_ENGINE_SCAN'], 'true'), eq(variables['OS_SKU'], 'Ubuntu'), in(variables['OS_VERSION'], '22.04', '24.04')) | ||
| condition: and(succeeded(), eq(variables['OS_SKU'], 'Ubuntu'), in(variables['OS_VERSION'], '22.04', '24.04')) |
| @@ -0,0 +1,10 @@ | |||
| { | |||
| "_comment": "PR-gated pins for the Compliance Engine packages consumed from the ComplianceEnginePackages NuGet feed by the VHD builder's (shadow, non-blocking) compliance scan. 'assessor' is a single '<package>@<version>' pin; 'benchmarks' is a map of <key> -> '<package>@<version>' so multiple benchmarks (Ubuntu 22.04/24.04 x L1/L2, or CIS + STIG) can be scanned in one run. The <key> both selects which MOFs apply to the OS being built (vhd-scanning.sh audits mof/ubuntu-<version>-*.mof) and names the result JSON (compliance-engine-<key>.json). Bumping a pin REQUIRES a pull request (PR-gating): publishing a new package to the feed never changes what the pipeline uses until a PR updates the pin here. Versions are '<YYYYMMDD>.<buildid>[-suffix]', so the pin date is intrinsic to the version string; the scan derives a non-blocking staleness warning from the leading YYYYMMDD. Package names must exactly match the NuGet package ids published by the Compliance Augmentation Engine. If L1 and L2 ship as a single combined MOF, collapse the two entries into one. This schema is shared with acl-pipelines' pipelines/compliance-versions.json.", | |||
| "assessor": "compliance-engine-assessor@TODO-pin-published-assessor-version", | |||
| download_root="${{ parameters.downloadRoot }}" | ||
| mof_dir="${download_root}/mof" | ||
| tmp_dir="${download_root}/.download" | ||
| rm -rf "${download_root}" | ||
| mkdir -p "${mof_dir}" "${tmp_dir}" |
| if [ -z "$AZURE_MSI_RESOURCE_STRING" ]; then | ||
| echo "AZURE_MSI_RESOURCE_STRING must be set for az login" | ||
| exit 1 | ||
| fi |
AgentBaker Linux gate detectiveRun: https://msazure.visualstudio.com/CloudNativeCompute/_build/results?buildId=172153488 Detective summary: Two deterministic build/config failures were present: Compliance Engine attempted to download literal package version Likely cause / signature: $complianceSig plus $skuSig. Confidence: High for immediate causes. Recommended owner/action: VHD build/compliance owner or PR author if PR #8910 changed compliance package pinning, VHD matrix variables, or Packer template wiring; replace the TODO assessor version and restore SKU_NAME propagation. Strongest alternative: Feed outage or transient Azure resource lookup failure; less likely because the requested compliance package version is a literal TODO value and SKU_NAME fails a local preflight validation. Evidence: Timeline shows Compliance download Bash exit 22, Build VHD exit 2; logs show curl 404 for the TODO package and Wiki signatures: $complianceSig, $skuSig |
| - template: ./.compliance-engine-scan-template.yaml | ||
| parameters: | ||
| # TEMPORARY (testing): the ENABLE_COMPLIANCE_ENGINE_SCAN gate is removed so | ||
| # the scan runs on every Ubuntu 22.04/24.04 build without setting the | ||
| # variable. RESTORE before merge: | ||
| # and(succeeded(), eq(variables['ENABLE_COMPLIANCE_ENGINE_SCAN'], 'true'), eq(variables['OS_SKU'], 'Ubuntu'), in(variables['OS_VERSION'], '22.04', '24.04')) | ||
| condition: and(succeeded(), eq(variables['OS_SKU'], 'Ubuntu'), in(variables['OS_VERSION'], '22.04', '24.04')) | ||
| feedName: ComplianceEnginePackages |
| local ce_ts assessor_blob | ||
| ce_ts=$(date +%s%3N) | ||
| assessor_blob="compliance-engine-assessor-${BUILD_ID}-${ce_ts}" | ||
| echo "Uploading Compliance Engine assessor to blob ${assessor_blob}" | ||
| az storage blob upload --container-name "${SIG_CONTAINER_NAME}" --file "${CE_ASSESSOR_LOCAL}" --name "${assessor_blob}" --account-name "${STORAGE_ACCOUNT_NAME}" --auth-mode login --overwrite | ||
|
|
|
|
||
| echo "Compliance Engine: auditing ${key} (MOF ${m})" | ||
| az storage blob upload --container-name "${SIG_CONTAINER_NAME}" --file "${m}" --name "${mof_blob}" --account-name "${STORAGE_ACCOUNT_NAME}" --auth-mode login --overwrite | ||
|
|
| # Retrieve the JSON result (and the assessor log) for publication. | ||
| az storage blob download --container-name "${SIG_CONTAINER_NAME}" --name "${result_blob}" --file "${result_local}" --account-name "${STORAGE_ACCOUNT_NAME}" --auth-mode login || \ | ||
| echo "WARNING: no Compliance Engine result blob for ${key}" | ||
| az storage blob download --container-name "${SIG_CONTAINER_NAME}" --name "${log_blob}" --file "compliance-engine-${key}.log" --account-name "${STORAGE_ACCOUNT_NAME}" --auth-mode login || true | ||
|
|
| if [ -s "$RESULT_PATH" ]; then | ||
| az storage blob upload --container-name "$SIG_CONTAINER_NAME" --file "$RESULT_PATH" --name "$RESULT_BLOB_NAME" --account-name "$STORAGE_ACCOUNT_NAME" --auth-mode login --overwrite | ||
| else | ||
| echo "WARNING: assessor produced no JSON result on stdout" | ||
| fi | ||
| if [ -s "$LOG_PATH" ]; then | ||
| az storage blob upload --container-name "$SIG_CONTAINER_NAME" --file "$LOG_PATH" --name "$LOG_BLOB_NAME" --account-name "$STORAGE_ACCOUNT_NAME" --auth-mode login --overwrite | ||
| fi |
| # TEMPORARY (testing): the ENABLE_COMPLIANCE_ENGINE_SCAN gate is removed so | ||
| # the scan runs on every Ubuntu 22.04/24.04 build without setting the | ||
| # variable. RESTORE before merge: | ||
| # and(succeeded(), eq(variables['ENABLE_COMPLIANCE_ENGINE_SCAN'], 'true'), eq(variables['OS_SKU'], 'Ubuntu'), in(variables['OS_VERSION'], '22.04', '24.04')) | ||
| condition: and(succeeded(), eq(variables['OS_SKU'], 'Ubuntu'), in(variables['OS_VERSION'], '22.04', '24.04')) |
| # Retrieve the JSON result (and the assessor log) for publication. | ||
| az storage blob download --container-name "${SIG_CONTAINER_NAME}" --name "${result_blob}" --file "${result_local}" --account-name "${STORAGE_ACCOUNT_NAME}" --auth-mode login || \ | ||
| echo "WARNING: no Compliance Engine result blob for ${key}" | ||
| az storage blob download --container-name "${SIG_CONTAINER_NAME}" --name "${log_blob}" --file "compliance-engine-${key}.log" --account-name "${STORAGE_ACCOUNT_NAME}" --auth-mode login || true | ||
|
|
||
| # Clean up per-MOF blobs. | ||
| az storage blob delete --account-name "${STORAGE_ACCOUNT_NAME}" --container-name "${SIG_CONTAINER_NAME}" --name "${mof_blob}" --auth-mode login || true | ||
| az storage blob delete --account-name "${STORAGE_ACCOUNT_NAME}" --container-name "${SIG_CONTAINER_NAME}" --name "${result_blob}" --auth-mode login || true | ||
| az storage blob delete --account-name "${STORAGE_ACCOUNT_NAME}" --container-name "${SIG_CONTAINER_NAME}" --name "${log_blob}" --auth-mode login || true | ||
|
|
||
| # Non-blocking validation: warn (never fail) on a missing/invalid result. | ||
| if [ -s "${result_local}" ] && jq -e '.' "${result_local}" >/dev/null 2>&1; then | ||
| echo "Compliance Engine ${key}: valid JSON result at ${result_local}" | ||
| else | ||
| printf '##vso[task.logissue type=warning]Compliance Engine %s: missing or invalid JSON result (non-blocking).\n' "$key" | ||
| fi |
| # TEMPORARY (testing): the ENABLE_COMPLIANCE_ENGINE_SCAN gate is removed so | ||
| # the scan runs on every Ubuntu 22.04/24.04 build without setting the | ||
| # variable. RESTORE before merge: | ||
| # and(succeeded(), eq(variables['ENABLE_COMPLIANCE_ENGINE_SCAN'], 'true'), eq(variables['OS_SKU'], 'Ubuntu'), in(variables['OS_VERSION'], '22.04', '24.04')) | ||
| condition: and(succeeded(), eq(variables['OS_SKU'], 'Ubuntu'), in(variables['OS_VERSION'], '22.04', '24.04')) |
| # Retrieve the JSON result (and the assessor log) for publication. | ||
| az storage blob download --container-name "${SIG_CONTAINER_NAME}" --name "${result_blob}" --file "${result_local}" --account-name "${STORAGE_ACCOUNT_NAME}" --auth-mode login || \ | ||
| echo "WARNING: no Compliance Engine result blob for ${key}" | ||
| az storage blob download --container-name "${SIG_CONTAINER_NAME}" --name "${log_blob}" --file "compliance-engine-${key}.log" --account-name "${STORAGE_ACCOUNT_NAME}" --auth-mode login || true | ||
|
|
Initial integration of the Compliance Engine Assessor which will replace the current scripted CIS Assessor scanning.
PR descript will be extended later after the initial wiring is set up.