Skip to content

HDDS-15581. ozone debug replicas chunk-info reports wrong block size for EC keys#10592

Open
len548 wants to merge 3 commits into
apache:masterfrom
len548:HDDS-15581
Open

HDDS-15581. ozone debug replicas chunk-info reports wrong block size for EC keys#10592
len548 wants to merge 3 commits into
apache:masterfrom
len548:HDDS-15581

Conversation

@len548

@len548 len548 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

ozone debug replicas chunk-info reported the same blockData.size for every EC replica instead of the per-replica size from each datanode.

Example steps to reproduce

  1. Create an EC key with rs-3-2-1024k of size 1,148,576 bytes (between 1 MiB and 2 MiB).
  2. Run:
    ozone debug replicas chunk-info <volume>/<bucket>/<key>
  3. Inspect blockData.size for each entry in keyLocations.

Expected (EC 3+2, 1,148,576 bytes):

Replica Expected size
Data 1 1,048,576
Data 2 100,000
Data 3 0
Parity 4, 5 1,048,576 each

Actual: all replicas show `1,048,576.

`

Root cause: After HDDS-13445, ChunkKeyHandler iterated over pipeline datanodes but called ContainerProtocolCalls.getBlock(), which uses tryEachDatanode and always queries the closest node unless errors reading the datanode. The loop variable of datanode was ignored by that. The current testing doesn't catch this bug because it doesn't check the size for each replicas.
Fix: Add ContainerProtocolCalls.getBlockFromDatanode() and use it in ChunkKeyHandler so each replica is queried on its own datanode with the correct EC replica index.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15581

How was this patch tested?

Replace .block file-count checks in ozone-debug-tests-ec3-2.robot with robot tests that assert blockData.size per replica for EC(3,2) covering different stripe layout scenarios. Add the same tests for EC(6,3) suite and wire it into ec-test.sh after scaling to 9 datanodes.

Also refactor keywords Count Datanodes In Service and Has Enough Datanodes from awss3ecstorage.robot to ec library.

CI: https://github.com/len548/ozone/actions/runs/28040511916/job/83005380343

@sarvekshayr sarvekshayr self-requested a review June 24, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant