Skip to content

Incorrect checksum calc for odd-sized buffers on big-endian #1044

@ghost

Description

On big-endian processors, the CalculateSum() func in core/utils/checksum.h will return an incorrect value when given a buffer with an odd number of bytes. This is because the last byte of the buffer is simply being promoted from 8 to 16 bits:

  // Add remaining 8-bit to the one's complement sum
  if (odd) {
    sum64 += *reinterpret_cast<const uint8_t *>(buf16);
  }

This works on LE, but on BE, the last byte should be left-shifted by 8 bits. (Recall, per the RFC, that the last odd byte should be treated as if it's the high-order byte of a 16-bit value in which the low-order byte is zero.)

For example, consider a five-byte buffer containing:

aabb ccdd ee

On BE, the expected adds should be:

     aabb
   + ccdd
   + ee00
    ~~~~~
    26598 = 659a (folded to 16 bits)

but will actually be:

     aabb
   + ccdd
   + 00ee
    ~~~~~
    17886 = 7887 (folded)

On LE, the adds should be (and will be):

     bbaa
   + ddcc
   + 00ee
    ~~~~~
    19a64 = 9a65 (folded)

Note that the LE result when byte-swapped equals the correct result from the BE calc, as expected.

This should give the correct result on both BE and LE:

   sum64 += be16_t(uint16_t(*reinterpret_cast<const uint8_t *>(buf16)) << 8).raw_value();

I can provide a test program to simulate this fix on an LE processor.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions