Skip to content

Caching foundation for a dependencyGraph which maps dependency relations.#4127

Open
ashprice wants to merge 7 commits into
GothenburgBitFactory:developfrom
ashprice:native-depmap
Open

Caching foundation for a dependencyGraph which maps dependency relations.#4127
ashprice wants to merge 7 commits into
GothenburgBitFactory:developfrom
ashprice:native-depmap

Conversation

@ashprice

@ashprice ashprice commented Jun 15, 2026

Copy link
Copy Markdown

This is an initial PR which lays the groundwork for one that will shortly follow. It introduces lazy caching of pending/completed tasks, a UUID index for pending tasks, the construction of a dependency map, and adds in-place mutation of pending tasks. This roughly halves the instruction count for many reports (though the speed gain from this PR is minor to the one that I will make if it is accepted, about 1.3x for many reports. I gave up waiting for the performance tests in ./performance/ to complete, and instead constructed a smaller, 300 task test database.)

Changes

  • pending_tasks() and completed_tasks() return const std::vector<Task>& instead of copying. all_tasks() is deliberately not cached, as it can be very large and is only read once per invocation.
  • an index of pending task UUIDs, _pending_index is lazily built with pending_tasks(). get() and modify() use this instead of scanning the vectors.
  • A DependencyGraph hashmap is introduced and built from the pending cache. (A temporary map is created for all_tasks(), in the event that dependency_scan() must refer to things in all_tasks.) This is not yet consumed by Task in this PR - that will be the PR to follow.
  • modify() makes updates in-place within the vector by leveraging the _pending_tasks index. This cache is invalidated in any case where there may be a change of state between the Rust replica and the cached index.
  • some minor changes relating to memory management for pending_tasks and completed_tasks.

Endnotes

This PR doesn't necessarily bring much by itself, but it's needed to lay the groundwork for DependencyGraph to be used by getDependencyTasks() and getBlockedTasks(). Adding in in-place modifications is a somewhat opportunistic addition for the PR, as it's not strictly necessary for core functionality, but it does improve modification performance when running big batch modifications.

All tests pass and I am pretty sure this doesn't break anything, but there's a small chance bugs have slipped by me.

This would be my first major contribution to a C++ codebase, and I am not a professional software engineer, so feedback on pretty much everything here is accepted gratefully.

ashprice added 7 commits June 15, 2026 12:39
Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>

Typo

Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>

Added note

Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>
Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>
… one to resolve references within all_tasks)

Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>
Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>
Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>
Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>
Signed-off-by: ashprice <gitcommit1@sl.ashprice.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant