Skip to content

[Bazel] Support for a custom prebuilt emscripten cache#1620

Open
DoDoENT wants to merge 6 commits into
emscripten-core:mainfrom
DoDoENT:feature/custom-cache
Open

[Bazel] Support for a custom prebuilt emscripten cache#1620
DoDoENT wants to merge 6 commits into
emscripten-core:mainfrom
DoDoENT:feature/custom-cache

Conversation

@DoDoENT
Copy link
Copy Markdown
Collaborator

@DoDoENT DoDoENT commented Oct 18, 2025

This adds support for using a prebuilt cache as an archive instead of building it from scratch every time.

This solves multiple problems:

  • build speed on CI machines that have cold cache
    • instead of building the secondary cache every time, it can simply download it
  • effective support for multiple caches (Bazel support for multiple emscripten caches #1581)
    • emscripten supports having multiple caches on its own, but it builds thinlto, lto, and other cache combinations on demand. This doesn't work in the Bazel world, where the cache needs to be frozen.
    • the current solution allowed building a secondary cache, but only with a single configuration
    • this allows the use of any archive that contain any emscripten cache
    • this PR does not handle how that cache is built - you can build it with embuilder, zip it and serve it from your server or whatever you want
  • secondary cache hermeticity problem
    • the current solution for secondary cache generated emscripten_config file that contained full path to the cache.
    • this was not hermetic as this path could be different on different machines
    • this PR calculates the cache path by using the same trick with environment variables, ensuring that emscripten_config file is always same, regardless of the machine
    • additionally, it ensures that the cache is provided as an input to the toolchain so that Bazel can correctly add it to the sandbox environment

After applying this patch to our internal codebase, the regular builds of WASM code went down from 12-15 minutes to cca 30 seconds, regardless on which machine it runs, as long as remote cache is warm.

@sbc100
Copy link
Copy Markdown
Collaborator

sbc100 commented Oct 18, 2025

@walkingeyerobot can you recommend anyone else who might be good to review this?

@DoDoENT
Copy link
Copy Markdown
Collaborator Author

DoDoENT commented Mar 15, 2026

@walkingeyerobot can you recommend anyone else who might be good to review this?

Since @walkingeyerobot didn't respond for a very long time, I added @mmorel-35 as a reviewer for this. He reviewed other Bazel-related emsdk PRs, so I'm confident he can take a look at this one as well.

We've been using this patch for over 5 months now in our company and haven't found any issues with it so far.

@nickschaap
Copy link
Copy Markdown
Contributor

This is incredibly useful. We added this patch to our usage as well.

@DoDoENT DoDoENT force-pushed the feature/custom-cache branch 2 times, most recently from b2e5dc8 to 82c0cbb Compare March 24, 2026 17:37
@DoDoENT DoDoENT force-pushed the feature/custom-cache branch from 82c0cbb to 52e489a Compare April 3, 2026 19:46
@DoDoENT DoDoENT requested a review from sbc100 April 4, 2026 16:26
@DoDoENT DoDoENT mentioned this pull request Apr 29, 2026
@sbc100
Copy link
Copy Markdown
Collaborator

sbc100 commented Apr 29, 2026

I don't really feel qualified to review this. Maybe @mmorel-35 can chime in?

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for supplying Emscripten’s frozen cache as a prebuilt archive (downloaded during repository setup) and makes the cache available as an explicit toolchain input to improve hermeticity and CI performance.

Changes:

  • Extend emscripten_cache module extension to optionally download/extract a prebuilt cache archive and wire it into emscripten_config.
  • Add a builtin_cache filegroup and an emscripten_cache target so the toolchain always carries cache files into the sandbox (builtin or secondary/prebuilt).
  • Document prebuilt_cache usage in the Bazel README.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
bazel/remote_emscripten_repository.bzl Adds cache filegroup target to toolchain “common files” inputs.
bazel/emscripten_cache.bzl Adds prebuilt_cache support and switches cache path computation to be machine-independent.
bazel/emscripten_build_file.bzl Exposes the builtin Emscripten cache as a filegroup for toolchain inputs.
bazel/README.md Documents configuring a prebuilt cache archive via Bzlmod.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread bazel/emscripten_cache.bzl
Comment thread bazel/emscripten_cache.bzl
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread bazel/emscripten_cache.bzl
Applied suggestion by CoPilot

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants