Skip to content

Sync L2 hit never backfills L1; sync L2-hit stats stringify the whole value #164

@27Bslash6

Description

@27Bslash6

Summary

Two sync-vs-async asymmetries on the L2 hit path:

  1. No L1 backfill on sync L2 hit. The async wrapper re-warms L1 on every L2 hit; the sync wrapper does not. After L1 eviction/restart, sync decorators re-pay the full L2 read + deserialize on every call until the next miss-store.
  2. Stats stringify the whole object. size_bytes = len(str(cached_result[1]).encode('utf-8')) renders the entire deserialized value (e.g. str(df) on a whole DataFrame) just for a stats number — twice, once unconditionally.

Evidence

  • Backfill: only _l1_cache.put in the sync wrapper is the miss-store at decorators/wrapper.py:797; async backfills at :1015,:1096,:1122. Sync L2-hit block :700-757 has none.
  • Stringify: decorators/wrapper.py:726 (unconditional) and :739; sync L1 already uses len(l1_bytes) correctly at :669.

Impact

Throughput cliff for sync large-object reads + a payload-proportional throwaway allocation/CPU on every sync L2 hit.

Fix

Backfill L1 on sync L2 hits (mirror async); reuse the known serialized length for stats instead of str(value).

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance improvementspythonPython library

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions