Some improvements#98
Merged
Xor-el merged 6 commits intoJul 13, 2026
Merged
Conversation
…lbacks
Naming pass: a kernel's name now states the ISA its body actually
requires, not the dispatch slot it serves - pure-GPR bodies are named
Gpr, VEX-128-only bodies Avx.
- KeccakF1600Sse2(+Absorb)_x86_64 -> KeccakF1600Gpr(+Absorb)_x86_64
- CRCFoldForwardSse2_{i386,x86_64} -> CRCFoldForwardGpr_*
- SHA1/SHA256CompressAvx2_i386 -> *CompressAvx_i386
- Blake2SCompressAvx2_{i386,x86_64} -> Blake2SCompressAvx_*
- Blake3CompressAvx2_x86_64 -> Blake3CompressAvx_x86_64
- ScryptSalsa8Avx2_{i386,x86_64} -> ScryptSalsa8Avx_*
- backend procs/wraps follow; banner comments list the dispatch tiers,
kernel headers no longer mention them
- Reflected-CRC keeps its Sse2 name (SSE2 genuinely required for the
128-bit loads and the CRC state shuttle) - headers now say so
New aarch64 fallbacks for the Crypto-Extensions-only hashes: SHA-1/
SHA-256/SHA-512 compress and Keccak-f[1600] permute + multi-block absorb
gain baseline-AArch64 GPR kernels dispatched at the NEON slot when
FEAT_SHA1/SHA256/SHA512/SHA3 are absent - previously scalar Pascal.
Bodies are the OpenSSL CRYPTOGAMS plain paths, .long-encoded and
verified byte-exact against the GNU-as reference objects.
- SHA-256/SHA-512 use sentinel-terminated K tables (K256_Gpr/K512_Gpr
with a zero terminator - the schedule loop ends on a zero K word);
SHA-1 needs no table (movz/movk round constants)
- Keccak's original round loop ends on a 256-aligned iotas table, which
a Pascal const cannot guarantee; the termination is patched to an
end-pointer compare so both kernels reuse the plain RC table
Every 32/64-bit lane rotate in the aarch64 NEON kernels was 3-op (ushr+shl+orr; Blake3's rot12/rot7 were even 4-op through a copy). The 2-op insert forms (ushr+sli, shl+sri, add+sri for ror63) land the result in a temp, so the generators now thread a register map: the rotated row renames into the temp instead of copying back, and rows normalize to their home registers only before writebacks and loop back-edges. Blake3 hash4's G is restructured 26 -> 21 ops with zero move instructions (b ping-pongs v_b -> t13 -> v_b; d lands in the register whose message word is dead). Instruction counts: Blake2S 632->573, Blake2B 1377->1233, Blake3 compress 714->645, Blake3 hash4 1653->1373 (-17%, the multi-chunk throughput path), Argon2 -12 ops per round slice (x16 executed per FillBlock). tbl-rot8 deliberately not used: official BLAKE3 NEON uses shl+sri and there is no mask setup or extra register cost.
The XXH3 test vectors all stayed under 240 bytes, so the SIMD accumulate/scramble/initSecret paths were only ever checked for self-consistency - a wrong kernel agreed with itself and passed. Four known-answer tests (values cross-checked against the official xxHash implementation) now cover accumulate512 (299 B), scrambleAcc (2 KiB) and the seeded long path (299 B with MaxUInt64 and with an asymmetric key) on every target. They surfaced three shipped bugs: - aarch64 NEON Acc512/Scramble: five vector encoders carried a stray bit in the source-register field, so every emitted uzp1/uzp2/umlal/ umlal2/mul executed with source register 2|m. Acc512 mixed the raw secret into accumulator lanes 2-3/6-7; Scramble multiplied by uninitialized registers. The mnemonic comments were correct - the encoded words were not. - i386 InitSecret (SSE2 + AVX2): the 64-bit seed materialization pushed lo before hi, leaving the high dword at [esp], so movq loaded a dword-swapped qword (-seed became 2^32 instead of 1). - InitSecret's seed parameter itself: a by-value UInt64 third parameter never rides in ecx under the i386 register convention, so the shared Proc3 prologue handed the kernels an undefined register - one compiler worked only by caller-codegen accident. The seed now travels by pointer (ASeedPtr: PUInt64), the same pattern as the Blake2 counter/flags pointers, and every kernel dereferences it on entry ([edi] on i386, mov r8,[r8] on x86_64, ldr x2,[x2] on NEON).
…xplicit XXH3_InitSecret_Scalar wrote the derived secret with native PUInt64 stores while every consumer reads the secret as little-endian wire bytes - on big-endian targets the custom secret came out byte-swapped, so seeded hashing of inputs over 240 bytes was wrong (caught by the seeded long-input known-answer tests on the big-endian CI; the unseeded ones passed since that path copies the default secret byte-for-byte). The custom-secret stores now go through WriteUInt64LittleEndian. XXH3_Accumulate512_Scalar / XXH3_ScrambleAcc_Scalar were already endian-correct (the accumulator is native value data, read and written natively everywhere including mergeAccs), but their raw PUInt64 pointer derefs looked identical to the buggy pattern. They now use TBinaryPrimitives.LoadUInt64/StoreUInt64 (native-order, alignment-safe) so wire accesses (Read/WriteUInt64LittleEndian) and value accesses (Load/StoreUInt64) are distinguishable at a glance.
Each SIMD kernel wrapper was declared twice - once per architecture
section - so signature changes had to be made in two places and could
drift. Following the CryptoLib4Pascal backend style, every kernel now
has ONE declaration with per-arch {$IFDEF} selecting the prologue and
body includes (x86_64 first), and pure-Pascal wraps that were
identical across architectures are shared.
Genuinely per-arch code keeps grouped arch sections: SHA3 (different
kernel sets per arch), the SHA-1/SHA-256 ShaNi pairs (i386 is a
4-param proc with the doubled-K table, x86_64 a 5-param proc with the
byte-swap mask) and the i386 Avx / x86_64 Avx2 wrap pairs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.