Summary
Two sync-vs-async asymmetries on the L2 hit path:
- No L1 backfill on sync L2 hit. The async wrapper re-warms L1 on every L2 hit; the sync wrapper does not. After L1 eviction/restart, sync decorators re-pay the full L2 read + deserialize on every call until the next miss-store.
- Stats stringify the whole object.
size_bytes = len(str(cached_result[1]).encode('utf-8')) renders the entire deserialized value (e.g. str(df) on a whole DataFrame) just for a stats number — twice, once unconditionally.
Evidence
- Backfill: only
_l1_cache.put in the sync wrapper is the miss-store at decorators/wrapper.py:797; async backfills at :1015,:1096,:1122. Sync L2-hit block :700-757 has none.
- Stringify:
decorators/wrapper.py:726 (unconditional) and :739; sync L1 already uses len(l1_bytes) correctly at :669.
Impact
Throughput cliff for sync large-object reads + a payload-proportional throwaway allocation/CPU on every sync L2 hit.
Fix
Backfill L1 on sync L2 hits (mirror async); reuse the known serialized length for stats instead of str(value).
Summary
Two sync-vs-async asymmetries on the L2 hit path:
size_bytes = len(str(cached_result[1]).encode('utf-8'))renders the entire deserialized value (e.g.str(df)on a whole DataFrame) just for a stats number — twice, once unconditionally.Evidence
_l1_cache.putin the sync wrapper is the miss-store atdecorators/wrapper.py:797; async backfills at:1015,:1096,:1122. Sync L2-hit block:700-757has none.decorators/wrapper.py:726(unconditional) and:739; sync L1 already useslen(l1_bytes)correctly at:669.Impact
Throughput cliff for sync large-object reads + a payload-proportional throwaway allocation/CPU on every sync L2 hit.
Fix
Backfill L1 on sync L2 hits (mirror async); reuse the known serialized length for stats instead of
str(value).