CLDSRV-955: Cache cloudserver builder yarn/tsc layer with a BuildKit mount#6229
Closed
tcarmet wants to merge 1 commit into
Closed
CLDSRV-955: Cache cloudserver builder yarn/tsc layer with a BuildKit mount#6229tcarmet wants to merge 1 commit into
tcarmet wants to merge 1 commit into
Conversation
…Kit mount The cloudserver production image spends most of its build time in the builder stage running `yarn install --production`, which clones the git dependencies and compiles their TypeScript (arsenal's tsc, etc.). The image layer cache is all-or-nothing and misses whenever yarn.lock changes. Add a BuildKit cache mount on yarn's cache folder so those downloaded and compiled packages are reused even on a layer-cache miss, and persist the mount across CI runs with buildkit-cache-dance backed by actions/cache.
Contributor
Hello tcarmet,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Contributor
Incorrect fix versionThe
Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:
Please check the |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| uses: actions/cache@v4 | ||
| with: | ||
| path: docker-yarn-cache | ||
| key: ${{ runner.os }}-docker-yarn-cache-${{ hashFiles('yarn.lock') }} |
There was a problem hiding this comment.
The cache key is hashFiles('yarn.lock') with no restore-keys, so when yarn.lock changes — the exact scenario the PR targets — the cache misses completely and yarn starts from scratch. Add a restore-keys prefix so a stale cache is reused on lockfile changes:
Suggested change
| key: ${{ runner.os }}-docker-yarn-cache-${{ hashFiles('yarn.lock') }} | |
| key: ${{ runner.os }}-docker-yarn-cache-${{ hashFiles('yarn.lock') }} | |
| restore-keys: | | |
| ${{ runner.os }}-docker-yarn-cache- |
Contributor
Author
|
Pausing this BuildKit cache-mount experiment; not investing further effort for now. (Also blocked at startup by the repo Actions allow-list, which would need buildkit-cache-dance added.) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Speeds up the cloudserver image build by making the expensive dependency-install work reusable across CI runs. Most of that image's build time is the builder stage installing the git dependencies and compiling their TypeScript; today the image layer cache is all-or-nothing and is thrown away whenever the lockfile changes. A BuildKit cache mount keeps the downloaded and already-compiled packages so a build reuses them even on a layer-cache miss, and the mount is persisted between runs. This is a follow-up experiment under CLDSRV-955 aimed at the build-time floor that the job-parallelization work (#6228) can't reach.
How it works
yarn installgets a--mount=type=cacheon yarn's cache folder (which also holds the compiled git deps), so a warm cache skips the re-clone and re-tsc.buildkit-cache-dancebacked byactions/cache(keyed on the lockfile hash).Validation
Draft until CI shows the effect: compare the cloudserver image build time on a warm run against a
development/9.3build. Keeping it draft so we can confirm the mount is actually restored (look for the yarn install reusing cached git deps) before calling it a win.