feat: implement deterministic exit codes and structured machine outpu… by KhushviB · Pull Request #173 · Azure/CosmosDBShell

KhushviB · 2026-07-11T18:15:38Z

Description (#155 )

This PR implements the deterministic machine output contract (Wave 1, P0) required for the Agentic & Automation Roadmap. It ensures that the Cosmos DB Shell can be safely consumed by downstream CI/CD pipelines, automation tools, and agentic wrappers without output corruption or ambiguous failure states.

Key Changes

Global --output and --quiet Flags:
- Added support for --output json, ndjson, table, and csv.
- Added a --quiet (-q) flag that suppresses all standard informational output (banners, connection logs, etc.).
Deterministic Exit Codes:
- 0: Success
- 1: Generic error
- 2: Bad arguments / Parser errors
- 3: Auth / Connection failure
- 4: Not found
- 5: Throttled / RU budget exceeded
Machine Mode Protections:
- When --output json|ndjson or --quiet are detected, the shell now implicitly disables ANSI color code injections (ColorSystem = NoColors) to prevent STDOUT corruption.
- When --output json|ndjson is specified, it inherently behaves as --quiet to guarantee that no interactive text logs interleave with the JSON objects.
Structured Error Handling on stderr:
- Any errors thrown during initialization (connection failures) or shell execution are gracefully mapped to their appropriate ExitCode and emitted as a structured JSON object ({"status": "error", "error": "..."}) to STDERR when in machine modes.

Validation / Acceptance Criteria Met

-c and -f runs produce parseable NDJSON implicitly.
No ANSI characters are rendered in machine modes.
Documented and stable exit codes correctly propagate to %ERRORLEVEL% / $LASTEXITCODE.
Tested parser error interception (returns Exit Code 2).
Tested early startup connection failure (returns Exit Code 1 / 3, formats to JSON).

…t (A1) - Add --output json/ndjson/table and --quiet flags - Implement deterministic exit codes (0: success, 1: generic, 2: bad args, 3: auth, 4: not found, 5: throttled) - Emit structured JSON errors on stderr during machine modes - Automatically disable ANSI colors and suppress informational output in machine modes

Copilot

Pull request overview

Implements a first-cut “machine output contract” for the Cosmos DB Shell by introducing global --output/--quiet flags, adding NDJSON output support, and mapping error conditions to deterministic exit codes intended for CI/CD and automation consumption.

Changes:

Adds root CLI flags --output/-o and --quiet/-q, and forces NoColors + quiet behavior for machine modes.
Introduces OutputFormat.Ndjson plus NDJSON formatting support for JSON-like command results.
Adds exit-code plumbing via CommandState.ExitCode, ParserErrorCommandState.ExitCode = 2, and ErrorCommandState.ExitCode mapping for Cosmos SDK failures and argument errors.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
CosmosDBShell/Program.cs	Adds root options, machine-mode detection, defaulting to NDJSON in non-interactive runs, and maps parse/connect failures to deterministic exit codes.
CosmosDBShell/lang/en.ftl	Adds help strings for `--output` and `--quiet`.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs	Implements `Quiet` suppression for `ShellInterpreter.Write*` and emits structured JSON errors to stderr in machine modes.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ParserErrorCommandState.cs	Sets parser-error command states to exit code 2.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/OutputFormat.cs	Adds `Ndjson` output format enum value.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ErrorCommandState.cs	Adds exit-code mapping based on exception type / Cosmos status codes.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/CommandState.cs	Adds `ExitCode` and NDJSON formatting logic; extends `SetFormat` to accept `ndjson`.

Comments suppressed due to low confidence (1)

CosmosDBShell/Azure.Data.Cosmos.Shell.Core/CommandState.cs:70

For an invalid format string, SetFormat currently throws ShellException, which maps to exit code 1 via ErrorCommandState. Since an invalid --output value is a bad-arguments case, throwing an ArgumentException here will allow it to deterministically map to exit code 2.

        {
            throw new ShellException(MessageService.GetString("error-invalid_output_format", new Dictionary<string, object> { { "format", outputFormat } }));
        }

- Handle root System.CommandLine parser errors via JSON serialization to STDERR in machine modes - Propagate COSMOSDB_SHELL_FORMAT automatically to command formatters - Ignore case on JSON/NDJSON format checks - Bypass interactive RBAC prompts in quiet/machine modes - Prevent trailing blank lines in JSON serialization output - Add integration tests for deterministic machine exit codes - Update CLI argument tables in README

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

mkrueger · 2026-07-13T09:04:27Z

Can you fix/comment/close the copilot issues - otherwise we can't merge.

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

CosmosDBShell/Program.cs:165

When --output json|ndjson or --quiet is used, this early --otel validation error still writes plain text to STDOUT (and ShellInterpreter.WriteLine can't honor o.Quiet yet because ShellInterpreter.Instance.Options is assigned later). This breaks the machine-output contract by contaminating STDOUT; emit the structured error JSON to STDERR in machine mode.

                    Environment.ExitCode = 2;
                    ShellInterpreter.WriteLine(MessageService.GetArgsString(
                        "otel-error-invalid-endpoint", "endpoint", o.OtlpEndpoint));
                    return;
                }

…ne mode

mkrueger · 2026-07-14T10:31:59Z

I really appreciate all input but copilot is the endboss that needs to be defeated for PRs beeing mergeable :(.

…cation scoping - Normalize the accepted --output token to lowercase in parseArgument so o.Output is always a canonical lowercase value (json, ndjson, table, csv). Previously the raw user-supplied casing was stored, making any plain equality check case-sensitive. - Hoist connectionMode and currentLocation declarations above the isQuiet guard in PrintConnectionInfoAsync so they are in scope for the JSON result dictionary regardless of quiet mode. The previous placement caused CS0841 build errors when isQuiet was true. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Only redirect stdin in RunShellAsync when stdinScript is provided. This keeps no-stdin invocations from being misclassified as machine mode via Console.IsInputRedirected and preserves expected root parser behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

- Gate connect RBAC guidance behind interactive mode so quiet/machine runs do not emit interactive stdout text. - In ShellInterpreter.PrintState, treat OperationCanceledException like other machine-mode failures by emitting structured JSON to stderr instead of a warning on stdout. - Reuse a shared inMachineMode condition in the catch block. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

When --output json or --output ndjson is selected, route AnsiConsole output through TextWriter.Null using a dedicated AnsiConsoleSettings instance. This prevents accidental AnsiConsole writes from commands from polluting machine-readable stdout, while command-state output still uses Console.Out. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

- Apply startup machine-mode handling to quiet mode as well as json/ndjson. - Route startup MCP errors through structured stderr output in machine mode instead of unconditional AnsiConsole stdout writes. - Keep interactive behavior unchanged for non-machine runs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

- Treat startup machine mode consistently for quiet/json/ndjson and suppress startup informational stdout (clear-history/theme warnings). - Route startup machine-mode errors through structured stderr JSON and return exit code 2 for invalid --mcp port argument. - In PrintState, bypass AnsiConsole markup rendering in machine mode so JSON/ text outputs always go through Console.Out. - Expand process integration coverage for implicit -c ndjson output, machine- mode clear-history behavior, and invalid --mcp port behavior. - Clarify README machine-mode wording to avoid over-promising stdout guarantees. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

KhushviB · 2026-07-14T20:08:09Z

@mkrueger FInally defeated the endboss 🙌🏻

KhushviB requested review from a team and Copilot July 11, 2026 18:15

Copilot started reviewing on behalf of KhushviB July 11, 2026 18:16 View session

Copilot AI reviewed Jul 11, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 11, 2026 18:32

Copilot started reviewing on behalf of KhushviB July 11, 2026 18:33 View session

Copilot AI reviewed Jul 11, 2026

View reviewed changes

fix/copilot-fix-suggestion

5cca375

Copilot AI review requested due to automatic review settings July 13, 2026 14:09

Merge branch 'main' into feature/a1-deterministic-output

c8cd2c8

Copilot started reviewing on behalf of KhushviB July 13, 2026 14:13 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Potential fix for pull request finding

2c6a7c0

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 13, 2026 14:18

Copilot started reviewing on behalf of KhushviB July 13, 2026 14:22 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Apply suggestions from code review

96c7c30

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 13, 2026 14:34

Copilot started reviewing on behalf of KhushviB July 13, 2026 14:37 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Comment thread CosmosDBShell/Program.cs

Comment thread CosmosDBShell.Tests/Integration/ShellProcessTests.cs

Apply suggestions from code review

6afcabe

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 13, 2026 14:47

Copilot started reviewing on behalf of KhushviB July 13, 2026 14:51 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Comment thread CosmosDBShell/Program.cs

fix: emit mutually-exclusive options error to STDERR as JSON in machi…

9addb34

…ne mode

Copilot AI review requested due to automatic review settings July 13, 2026 15:32

Copilot started reviewing on behalf of KhushviB July 13, 2026 15:35 View session

Copilot AI review requested due to automatic review settings July 14, 2026 18:46

Copilot started reviewing on behalf of KhushviB July 14, 2026 18:47 View session