skip service-events otlp pipelines when sigv4auth validation fails#2139
Open
jj22ee wants to merge 2 commits into
Open
skip service-events otlp pipelines when sigv4auth validation fails#2139jj22ee wants to merge 2 commits into
jj22ee wants to merge 2 commits into
Conversation
Contributor
|
This PR was marked stale due to lack of activity. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of the issue
Prevent CWAgent startup failure when Application Signals is enabled but the sigv4auth extension cannot resolve AWS credentials via the AWS SDK Go v2 default chain — most commonly on-prem hosts.
The Application Signals OTLP metrics export and the logs pipelines route data through the OTel
sigv4authextension, which signs requests using the AWS SDK Go v2 default credential chain.sigv4auth.Validate()eagerly resolves credentials during the collector's config-validation phase. If the v2 chain cannot resolve credentials, validation fails and CWAgent fails to start.Description of changes
Pre-check whether sigv4auth can resolve credentials before registering the pipelines that depend on it, and degrade gracefully instead of failing startup.
translator/translate/otel/extension/sigv4auth/translator.go — CanResolveCredentials()new method builds a throwaway sigv4auth config with the same region/role the real extension would use and runsxconfmap.Validate()on it — exercising the exact code path that would otherwise fail at collector startup. No sigv4auth instance exists at translation time, so a throwaway instance is needed.translator/translate/otel/pipeline/applicationsignals/translators.go —
NewTranslators()— updated so that when credentials cannot be resolved:The check runs the same resolution the real extension would, in the same credential environment, moments before the collector's own validation. If it passes, the real sigv4auth instances will also pass, and if it fails, the real sigv4auth instances will also fail identically, so we avoid creating them and log a warning instead.
Behavior summary:
License
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Tests
Run CWAgent with ApplicationSignals configuration in Docker environment without fix and without credentials:
Run CWAgent with same configuration/environment with fix:
Also added unit and sampleConfig tests:
TestNewTranslatorsMetricsNoCredentials— 1 translator (default) when creds unresolvableTestNewTranslatorsLogsNoCredentials— 0 log translators when creds unresolvableTestTranslatorMetricsDefault— default metrics pipeline structure (OTLP → EMF)TestAppSignalsNoCredentialsConfig— sampleConfig YAML for the no-credentials configRequirements
Before commiting your code, please do the following steps.
make fmtandmake fmt-shmake lintIntegration Tests
To run integration tests against this PR, add the
ready for testinglabel.