Task caching with minimal interface addition

[Napolitain]

Description

What

I believe caching is arguably in-scope of taskfile interface, since it is naturally part of workflows. We could choose to delegate caching to a end user, but then, the user would need to orchestrate the workflow on top of taskfiles, or write very complicated tasks to support this.

In fact, there is little to do for supporting a simple KV store caching solution.

Why

Caching is always a very important functionality in every products. Use case may happen to have back and forth task build (with identical content at times). In that case, KV cache is a obvious solution to checksum based solution in Taskfile.

As a argument against not supporting it, Taskfile currently support :

Prevent unnecessary work

via sources and generates fields in the interface. That is exactly the definition of caching : if input output same, dont do again.

How

Consider

sources:
- file.txt
- blabla.txt
generates:
- output.txt

It then creates already a hash in .task/checksum/myTaskName, and it contains (in other words, cat shows) a hash like 23bc5743a223c33397efb435f7cf59a6.

To implement KV store based cache, we just need to :

• support cache: [false|true] default to false field, to enable the functionality
• use the checksum of the task as the key and the generates/ listed files as the content
• bundle the content in an archive, e.g. checksum.tar.zstd

We would end up with

sources:
- file.txt
- blabla.txt
generates:
- output.txt
cache: true

I am pretty sure I can implement that easily and it would have great impact. We would let backwards compatibility by defaulting to false.

Then, we can expand the cache by adding eviction policies such as LRU as part of .taskrc for example. But that can be implemented in a second PR.

[Napolitain]

changing title from change to addition to better reflect the non breaking nature of the feature request, which I can work on.

[Napolitain]

[trulede]

I don't think its a question of effort (although it is). Rather, caching is not such a generic thing. Its rather specialised.

You can do Key/Value cache patterns with Redis (or anything) and by reading the checksums out of the .taskfile. A simple dep task is all it takes. But you are not thinking about Redis ... probably ... and that is the problem.

Even then ... its worse 🗡️ The Checksum may not be enough, for a C project, the generated files will depend on compiler switches and as a result the cache strategy involves not only a checksum but also build metadata. There is a special tool that integrates directly into the GCC toolchain to manage the caching.

In a C++ project, its probably Conan. And Go does its own thing too.

You can write a cache-task in a few minutes with CoPilot. Certainly a lot less effort.