HDDS-15600. Fix ListObjects response for encoding-type, empty delimiter, and control-character prefix

[Gargi-jais11]

What changes were proposed in this pull request?

Ozone S3 Gateway fails s3-tests ListObjects cases that AWS S3 passes:

• test_bucket_list_encoding_basic — With EncodingType=url, keys and CommonPrefixes containing spaces are encoded with + (Java URLEncoder form encoding) instead of AWS-style %20.
  Example: prefix quux ab/ is returned as quux+ab/ but should be quux%20ab/.
• test_bucket_list_delimiter_empty — When Delimiter='' is sent, listing behavior is correct (all keys returned, no CommonPrefixes), but the response incorrectly includes a Delimiter field. AWS omits Delimiter from the XML when the client passes an empty delimiter.
• test_bucket_list_prefix_unreadable — ListObjects with Prefix='\x0a' (newline) should echo the prefix in the response and return empty Contents/CommonPrefixes. Ozone may not preserve or echo the control-character prefix correctly.

https://ozone.s3.peterxcli.dev/#latest-run-section

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15600

How was this patch tested?

Before fix:

1. encoding type:

bash-5.1$ aws s3api list-objects   --bucket buck1   --encoding-type url   --delimiter /   --endpoint-url http://s3g:9878/
{
    "Contents": [
        {
            "Key": "asdf%2Bb",
            "LastModified": "2026-06-19T09:51:39.569Z",
            "ETag": "\"f75b8179e4bbe7e2b4a074dcef62de95\"",
            "Size": 8,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "testuser",
                "ID": "bb2bd7ca4a327f84e6cd3979f8fa3828a50a08893c1b68f9d6715352c8d07b93"
            }
        }
    ],
    "CommonPrefixes": [
        {
            "Prefix": "foo/"
        },
        {
            "Prefix": "foo%2B1/"
        },
        {
            "Prefix": "quux+ab/".                       <----------------- wrong output
        }
    ],
    "RequestCharged": null,
    "Prefix": ""
}

2. empty delimiter:

bash-5.1$ aws --debug s3api list-objects \
  --bucket buck1 \
  --delimiter "" \
  --endpoint-url http://s3g:9878 2>&1 \
  | grep -oE '<Prefix>[^<]*</Prefix>|<Delimiter>[^<]*</Delimiter>|<KeyCount>[^<]*</KeyCount>'
<Prefix></Prefix>
<KeyCount>0</KeyCount>
<Delimiter></Delimiter>.           <---------- should not be present

3 echoed prefix always url-encoded:

bash-5.1$ aws s3api list-objects \
  --bucket buck1 \
  --prefix $'\n' \
  --endpoint-url http://s3g:9878/
{
    "RequestCharged": null,
    "Prefix": "%0A"               <------------------ wrong behaviour
}

After fix:

1. encoding-type:

bash-5.1$ aws s3api list-objects   --bucket buck1   --encoding-type url   --delimiter /   --endpoint-url http://s3g:9878/
{
    "Contents": [
        {
            "Key": "asdf%2Bb",
            "LastModified": "2026-06-19T09:08:58.933Z",
            "ETag": "\"f75b8179e4bbe7e2b4a074dcef62de95\"",
            "Size": 8,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "testuser",
                "ID": "bb2bd7ca4a327f84e6cd3979f8fa3828a50a08893c1b68f9d6715352c8d07b93"
            }
        }
    ],
    "CommonPrefixes": [
        {
            "Prefix": "foo/"
        },
        {
            "Prefix": "foo%2B1/"
        },
        {
            "Prefix": "quux%20ab/"                <------------------------correct output
        }
    ],
    "RequestCharged": null,
    "Prefix": null
}

2. empty delimeter :

bash-5.1$ aws --debug s3api list-objects \
  --bucket buck1 \
  --delimiter "" \
  --endpoint-url http://s3g:9878 2>&1 \
  | grep -oE '<Prefix>[^<]*</Prefix>|<Delimiter>[^<]*</Delimiter>|<KeyCount>[^<]*</KeyCount>'
  
<KeyCount>0</KeyCount>.                 <-------------------- no empty delimiter present

3. echoed prefix should not be url encoded:

bash-5.1$ aws s3api list-objects \
  --bucket buck1 \
  --prefix $'\n' \
  --endpoint-url http://s3g:9878/
{
    "RequestCharged": null,
    "Prefix": "\n"
}

[adoroszlai]

Thanks @Gargi-jais11 for the patch.

[adoroszlai]

Can we use PercentCodec from commons-codec?

https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/net/PercentCodec.html

[Gargi-jais11]

Okay let me check and get back to u.