[uss_qualifier/reports] Add test artifacts obfuscator#1477
Open
BenjaminPelletier wants to merge 4 commits into
Open
[uss_qualifier/reports] Add test artifacts obfuscator#1477BenjaminPelletier wants to merge 4 commits into
BenjaminPelletier wants to merge 4 commits into
Conversation
the-glu
reviewed
Jun 4, 2026
| Detailed information on command-line options and toggles can be retrieved using the `--help` flag: | ||
|
|
||
| ```bash | ||
| PYTHONPATH=. uv run --index https://pypi.org/simple python monitoring/uss_qualifier/reports/obfuscate.py --help |
Contributor
There was a problem hiding this comment.
nit: This could be simplified to PYTHONPATH=. uv run monitoring/uss_qualifier/reports/obfuscate.py --help
(python is not needed, --index is environment-specific)
| -v "/path/to/local/input_artifacts:/input" \ | ||
| -v "/path/to/local/output_dir:/output" \ | ||
| interuss/monitoring \ | ||
| python monitoring/uss_qualifier/reports/obfuscate.py /input /output/obfuscated_artifacts.zip |
Contributor
There was a problem hiding this comment.
The path is wrong no, monitoring/ is extra? Also it should be run via uv no?
docker run --rm \
-v "/path/to/local/input_artifacts:/input" \
-v "/path/to/local/output_dir:/output" \
interuss/monitoring \
python monitoring/uss_qualifier/reports/obfuscate.py
python: can't open file '/app/monitoring/monitoring/uss_qualifier/reports/obfuscate.py': [Errno 2]
docker run --rm \
-v "/path/to/local/input_artifacts:/input" \
-v "/path/to/local/output_dir:/output" \
interuss/monitoring \
uv run uss_qualifier/reports/obfuscate.py
usage: obfuscate.py [-h] [--no-participants] [--no-hostnames] [--no-tokens]
input output
obfuscate.py: error: the following arguments are required: input, output
Suggested change
| python monitoring/uss_qualifier/reports/obfuscate.py /input /output/obfuscated_artifacts.zip | |
| uv run uss_qualifier/reports/obfuscate.py /input /output/obfuscated_artifacts.zip |
| elif k == "manager" and isinstance(v, str): | ||
| participant_ids.add(v) | ||
| elif isinstance(v, str): | ||
| for url in find_urls(v): |
Contributor
There was a problem hiding this comment.
reuse scan_text bellow? it do the same loop
|
|
||
|
|
||
| def test_find_urls(): | ||
| text = "Check http://dss1.uss1.localutm/dss/v1, or go to https://github.com/interuss. Also see (http://localhost:8082/status)." |
Contributor
There was a problem hiding this comment.
Should we test thoses extra cases?
http://user@localhost
http://user:password@localhost
http://localhost?q=a2
| # 2. Obfuscate hostnames | ||
| if config.obfuscate_hostnames: | ||
| for h, mapped_h in hostname_map.items(): | ||
| s = re.sub(rf"\b{re.escape(h)}\b", mapped_h, s) |
Contributor
There was a problem hiding this comment.
That an edge case, but if we have url in uppercase for some reason, urlparse normalize it to lowercase and it won't be replaced there.
urlparse("http://TEST").hostname
'test'
Suggested change
| s = re.sub(rf"\b{re.escape(h)}\b", mapped_h, s) | |
| s = re.sub(rf"\b{re.escape(h)}\b", mapped_h, s, flags=re.IGNORECASE) |
|
|
||
| # 1. Obfuscate tokens | ||
| if config.obfuscate_tokens: | ||
| s = re.sub(r"(?i)\bBearer\s+\S+", "Bearer REDACTED", s) |
Contributor
There was a problem hiding this comment.
Suggested change
| s = re.sub(r"(?i)\bBearer\s+\S+", "Bearer REDACTED", s) | |
| s = re.sub(r"(?i)\bBearer\s+\S+", "Bearer REDACTED", s, flags=re.IGNORECASE) |
To handle cases like bEarER that may be valid
| # Generate maps | ||
| participant_map = {} | ||
| for idx, pid in enumerate( | ||
| sorted(sorted(participant_ids), key=len, reverse=True), start=1 |
| scan_json(v, participant_ids, hostnames) | ||
| elif isinstance(obj, list): | ||
| for item in obj: | ||
| scan_json(item, participant_ids, hostnames) |
Contributor
There was a problem hiding this comment.
should scan_json do something on strings ?
There is an isinstance(v, str) in dict branch, but not in list, meaning str in dict and list are not handled the same way
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some users may be hesitant to include full test artifacts when reporting issues or requesting help because those artifacts may contain semi-sensitive information. This PR adds a tool that attempts to ease this problem somewhat by obfuscating sensitive information while still retaining nearly-full diagnostic value for the artifacts.
Written mostly with Gemini.