# Pre-RFC: Sandboxed, deterministic, reproducible, efficient Wasm compilation of proc macros
**URL:** https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359
**Category:** language design
**Created:** [August 20, 2023, 10:33pm UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359 "2023-08-20T22:33:46Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@dtolnay](https://internals.rust-lang.org/u/dtolnay)
#### Post date: [August 20, 2023, 10:33pm UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/1 "2023-08-20T22:33:47Z")
- Feature Name: `proc_macro_sandbox`
- Start Date: 2023-08-20
# Summary
Procedural macro crates can **opt-in** to precompilation to WebAssembly as part of the publish process. The crate compiles locally (as already done by `cargo publish`) and then is securely verified by a crates.io-managed service, conceptually similar to docs.rs, which enforces that sources in the crate exactly reproduce the Wasm artifact before the new release becomes available to any package registry users. Users can **opt-in** to running procedural macros they depend on via Wasm sandbox by installing a suitable Wasm runtime as a Rustup component.
# Motivation
**Auditability:** Rust's current only supported approach to procedural macros exposes anyone who uses them to a variety of unsavory possibilities: macros are arbitrary Rust code that can [spawn processes](https://github.com/serde-rs/serde/issues/2538) and [make network requests](https://twitter.com/ryan_levick/status/1257269963552894979). So can build.rs, but the implementation of a typical procedural macro is 2 orders of magnitude larger than a typical build script. We reduce audit burden by sandboxing macros such that their only possible interaction with the outside world is inspecting tokens and producing tokens. An estimated 99% of macros are amenable to sandboxing; unsandboxed macros remain expressible exactly as before, but would invite great scrutiny, analogous to unsafe code during audit.
**Determinism:** Inadvertent proc macro nondeterminism is a source of pain in build systems. For example, Buck likes to race a local recompilation against a download from distributed cache, taking whichever finishes first. When artifacts from the same source code diverge, builds can fail or take longer than needed. In a recent instance, rustc nondeterminism was found to be the largest contributor to slow builds due to poor cache utilization in a codebase using Remote Execution and a distributed build cache in Meta. These bugs can be as simple as a [proc macro using HashMap](https://github.com/asomers/mockall/pull/330) with randomized iteration order, and are impossible in a Wasm sandbox because there is no possible way to access OS randomness within the sandbox. Hash-based containers use pseudo-randomness with a hardcoded seed.
**Isolation:** For monorepo-scale build systems to work on **b** illion-line codebases, perfect understanding of the inputs of every build step is critically important. A rustc invocation must not result in reading some arbitrary file from the local filesystem in a manner that is not tracked by the build system, for it will not know the crate needs to be rebuilt when that file changes. [Workarounds exist](https://github.com/rust-lang/rust/issues/99515) but it is opt-in by the macro, rather than something that can be enforced codebase-wide through sandboxing.
**0-second compile time:** Today, procedural macros are widely understood to be "bad for compile time". Citing [work by @nnethercote and @lqd](https://hackmd.io/YJQSj_nLSZWl2sbI84R1qA) who have jointly been investigating and improving Rust compile time for years: _"These crates incur a lot of compilation costs, being among the most popular crates and quite slow to compile. Also, they block compilation of proc macro crates that are themselves often slow to compile. This leads to long and slow dependency chains"_. Users opting in to sandboxing the macros they depend on (which, again, 99% of macros would be compatible with) would almost never need to compile `syn` or `quote` or `proc-macro2`, which commonly appear on the critical path before useful parts of an application can begin to compile. 10-15 seconds (depending on syn features) is not an enormous benefit in CI, but is enough to be frustrating in interactive situations such as opening a project in an IDE.
**Macro ecosystem rejuvination:** In response to the previous motivation, compile time for crates like `syn` and `serde_derive` are begrudgingly tolerable to most people today, but this has come at the cost of [brutal concessions to functionality](https://github.com/dtolnay/syn/issues/1037) over many years. A precompiled codepath for those sensitive to build time would open the doors to more complete and powerful macro libraries, for example greater ability to focus on fantastic diagnostics rather than scrutinizing whether a user-facing diagnostics improvement would be worth the build time. Syn-based diagnostics today are far from where they would be if build time were a less pressing concern.
**Even faster incremental builds:** While the "0-second compile time" applies to the macro dependency itself, which is relevant to clean builds and otherwise cached by Cargo, 2 other opportunities exist for this feature to benefit incremental compile times too, and interactive latency (IDE). First, precompiled macros being optimized builds, rather than unoptimized native builds. Even with a 50% overhead from a high-performance Wasm runtime compared to native code, complex macros like `serde_derive` will still expand faster than they do today. Second, architectural advancements which are prohibitively expensive in an unoptimized macro, such as the ["ultimate non-generic DeserializeSeed" concept](https://github.com/serde-rs/serde/pull/2250#pullrequestreview-1564212707) for massively reducing monomorphization cost in Serde-generated code.
**User choice:** Design choices throughout the rest of this document are motivated by an understanding that many users and environments are uninterested in a precompiled artifact for legal or technological reasons, and satisfied with unisolated natively compiled macros. Both macro author and macro user must have opted-in for the sandboxed artifact to kick in; if not, the user will build the macro from source just like today. Furthermore, macro crates containing a precompiled implementation are 100% compatible with old versions of Cargo, with the embedded Wasm artifact simply being ignored and the library built from source.
# Guide-level explanation
#### Guide-level explanation for macro authors:
The vast majority of procedural macros do nothing more than inspect input tokens and emit output tokens. No filesystem access, no subprocesses, no network I/O. For such macros, sandboxing their execution in a WebAssembly-based runtime offers a number of benefits (auditability, determinism, and 0-second compile time for users).
A crate author can precompile their procedural macro to an efficient, reproducible, sandboxable Wasm artifact as part of publishing it to [crates.io](http://crates.io) by introducing the following field in Cargo.toml:
```toml
[package]
name = "serde_derive"
version = "..."
[lib]
proc-macro = true
sandboxed = true # <---
```
Now `cargo publish`, which ordinarily would locally verify a build for the host platform before uploading, will instead (or additionally?) build for a target platform called `wasm32-macro`. The resulting Wasm artifact is uploaded to [crates.io](http://crates.io) in the same .crate file as the crate's source code.
There will be a delay before your new release becomes publicly available on [crates.io](http://crates.io). The server will repeat the build, confirming that it produces precisely the same Wasm binary that your computer did. So, no need to try any funny business.
Restrictions apply to macros that use `sandboxed = true`:
1. Your macro must not have any (enabled) transitive dependency which is a procedural macro.
2. While your macro may have a build script or depend on crates that have a build script, those build scripts will _only_ run if a user of your macro is rebuilding it from source. They will _not_ run when you or [crates.io](http://crates.io) builds your macro for the `wasm32-macro` target, nor when a user's build picks up the precompiled implementation.
3. Your macro must build using the latest stable release of Rust. This is the version that you must publish with. This is the version that [crates.io](http://crates.io) will verify your build with.
4. As a consequence of the previous, your macro must not use any unstable APIs within the `proc_macro` crate.
5. Publish's `--no-verify` setting is incompatible with precompilation.
6. While usage of sandbox-incompatible standard library APIs such as `std::fs` or `std::thread` will not cause your macro to fail to build, such APIs unconditionally return an error or panic during macro expansion. All you get to do in a precompiled macro is inspect tokens and produce tokens/diagnostics.
7. Only one single precompiled build is produced. As part of publish, you control what features of your crate are enabled in that build, using publish's `--features` and `--all-features` and `--no-default-features` flags. Users who depend on any other feature set will not get the precompiled build. (In the future, we may allow specifying a collection of feature sets to precompile for.)
8. Unwinding does not happen. To keep code size in check for precompiled artifacts, builds are equivalent to `-Zbuild-std=panic_abort -Zbuild-std-features=panic_immediate_abort`. Proc macros are never supposed to panic in the first place. If you observe panics, a generic error message will be shown to the user, instructing them that they can temporarily disable the proc macro sandbox and rebuild from source to get the message and line number at which the macro panicked, to report a bug.
#### Guide-level explanation for macro users:
By default, procedural macros you depend on are always built from source. Be aware that macros involve running arbitrary Rust code on your computer at compile time, so there is a large degree of trust in depending on a macro provided by somebody else. If you are sensitive to such issues, you may wish to dedicate extra scrutiny to auditing your proc macro dependencies each time they're added or upgraded.
Many procedural macros publish both source code and a sandboxable precompiled build to [crates.io](http://crates.io). Sandboxing mitigates the "arbitrary code" aspect of macros, making them easier to audit and trust. Sandboxed macros are limited to doing nothing more than inspecting tokens and producing tokens.
They also take 0 seconds to compile, since the compilation has already been done securely by [crates.io](http://crates.io).
To opt-in to using these sandboxed macros in your Cargo builds when one is provided by the macro's author, run `rustup component add proc-macro-sandbox`. To stop using precompiled macros and go back to building all macro dependencies from source without isolation, run `rustup component remove proc-macro-sandbox`.
# Reference-level explanation
#### Reference-level explanation for compiler:
A new Tier 2 target platform is introduced, `wasm32-macro`. Almost everything about it is similar to `wasm32-unknown-unknown`, with the following exceptions:
- Bin crates, including tests, cannot be built for this target.
- A build of libproc\_macro is available for this target.
When one builds a `--crate-type=proc-macro --emit=link --target=wasm32-macro`, the output is a .wasm artifact. The API exported by the Wasm artifact is closely related to the API exported by the natively compiled `.so` of a procedural macro.
Instead of the current `--extern serde_derive=path/to/libserde_derive.so`, rustc can be passed `--extern serde_derive=path/to/libserde_derive.wasm`. If any Wasm macro is passed to a rustc invocation, it must also be passed `-Zproc-macro-sandbox=` containing the path to a suitable Wasm runtime, normally installed by the user as a rustup component. A particular proc-macro-sandbox is specific to a particular version of rustc, just like proc-macro-srv is.
Rather than `dlopen`-ing the macro to dynamically link it into the rustc process and communicate with it over serialization through the proc macro bridge, rustc will:
1. Spawn a subprocess of the given proc-macro-sandbox executable (a single one reused throughout the duration of the rustc execution),
2. Perform IPC to load it with the .wasm artifact and a set of Host Functions performing the function of the proc macro bridge (the subset of it corresponding to stable APIs),
3. Expand macros by dispatching invocations to the subprocess.
The proc-macro-sandbox is developed alongside libproc\_macro. It implements a high-performance Wasm runtime based on Wasmtime, although this remains an implementation detail.
#### Reference-level explanation for Cargo:
Cargo recognizes the `[lib] sandboxed = true` setting described in the guide-level explanation, and enforces the applicable Cargo-related restrictions.
As part of `cargo publish` on a sandboxed proc macro, Cargo first communicates with [crates.io](http://crates.io) to query the current compiler version being used for server-side verification of precompiled artifacts. This will almost always be the most recent stable Rust, or briefly, the one before. If the rustc currently configured for Cargo to use is not that one, Cargo fails with an informative message.
Building the precompiled artifact during publish, and during server-side verification, consists of an invocation similar to:
```console
cargo +stable build \
--release \
--target wasm32-macro \
-Z unstable-options \
-Z build-std=std,panic_abort \
-Z build-std-features=panic_immediate_abort
```
Rustc will produce a `.wasm` output which Cargo must include in the packaged `.crate` archive, inside of a directory called `target/wasm32-macro`.
When building a proc macro as a dependency, Cargo determines whether the crate's author has opted-in to sandboxing based on the Cargo.toml, and determines whether the local user has opted in to sandboxing by looking for a proc-macro-sandbox Rustup component for the current toolchain (the same way that Cargo currently knows to find the current toolchain's `rustfmt` for `cargo fmt`, `clippy-driver` for `cargo clippy`, etc).
If both opt-ins are present, Cargo passes `-Zproc-macro-sandbox=` to rustc, and `--extern` containing the `.wasm` artifact, rather than building the macro and its dependencies as `.so` from source.
#### Reference-level explanation for [crates.io](http://crates.io):
Proc macros published to [crates.io](http://crates.io) containing precompiled artifact do not immediately become available to users. There need not be any indication that the version has been published, except maybe to the crate's logged-in owners.
Asynchronously, a crates.io-managed service that is conceptually similar to docs.rs will fetch these newly published pending proc macro releases, and build them using the latest stable Rust toolchain.
As described in restrictions #1 and #2 in the guide-level explanation, this will _not_ involve arbitrary code execution on the server-side, as proc macro dependencies are not allowed and build scripts do not run. This greatly reduces the surface area for attacking this service. The only binaries running are a stable rustc and a stable Cargo.
If the server-side build does not reproduce a .wasm artifact with the same exact content as the uploaded one, [crates.io](http://crates.io) is notified and the release appears in a permanent yanked-like status. It can be downloaded for forensic analysis using the usual download endpoint, but is impossible to pull into builds, and cannot be unyanked.
If server-side verification successfully reproduces the uploaded .wasm, [crates.io](http://crates.io) is notified and the release becomes instantly available to users (and docs.rs) just as an ordinary upload would be.
There is already some mechanism substantially similar to this in [crates.io](http://crates.io), because "Documentation" links only appear in the UI after the docs.rs build of that crate has succeeded. If a documentation build failed, [crates.io](http://crates.io) does not show a Documentation link.
#### Reference-level explanation for rustup:
Rustup distributes `proc-macro-sandbox` as a new component. Not part of the default profile.
# Drawbacks
Though the user-facing surface area is small, this is undeniably a complex feature with involvement across multiple Rust subteams.
- The current, arbitrary unsandboxed native code execution of procedural macros is a good enough model for most Rust users, even those who never audit the source code of their dependencies.
- Macro compile times are not painfully bad for most people. (The Motivation section suggests why this may be a misleading impression and comes at significant expense to the ecosystem.)
- Non-human-readable package contents are anathema. (This sentiment appears to be elevated in the [crates.io](http://crates.io) ecosystem and is not present to the same extent in other places, like Python's wheels.)
- "Someone else is always auditing the code and will save me from anything bad in a macro before it would ever run on my machines." (At one point serde\_derive ran an untrusted binary for over 4 weeks across 12 releases before almost anyone became aware. This was plain-as-day code in the crate root; I am confident that professionally obfuscated malicious code would be undetected for years.)
- Procedural macros, including their transitive dependency graphs, are normally easy and pleasant to audit.
- Reproducible builds are hard, and will never work as envisioned, or will be onerous to maintain support for.
- High-profile crate publishers like dtolnay probably won't ever get their [crates.io](http://crates.io) credentials hacked by state-sponsored actors, and even if he does, eventually [crates.io](http://crates.io) will be able to do 2FA publishes, which will block any threat to anybody.
- If build scripts won't be sandboxed eventually, sandboxing proc macros is worthless. (Typical macros contain 100× more code than typical build scripts.)
- This will be hard to support in build systems other than Cargo. (They can literally do nothing and everything continues to work.)
# Rationale and alternatives
#### Alternative: black-box vs white-box sandbox model
I think of what's proposed above as the black-box plan. A _lot_ of mechanics are handled under the hood by Cargo and rustc. The interface surface area to users (both macro authors and consumers) is tiny.
I considered a white-box alternative. The only thing Rustup provides is a generic Wasm runtime with no specificity to macros. The whole thing is a single API to hand it your own Wasm blob and your own set of Host Functions. The only thing Cargo provides to crates is a `cfg` for whether this Wasm runtime is installed. Literally nothing else from rustc or [crates.io](http://crates.io) or cargo.
Crates are responsible for producing their own Wasm artifact by whatever means, using the existing `wasm32-unknown-unknown` target most likely, adding conditional compilation in their procedural macro crate root to shuffle wasm into the provided runtime, defining their own host fns that plug everything together with the `proc_macro`'s API.
I think this loses a range of benefits to security and compile performance, but would be comparatively trivial to ship.
#### Alternative: pre- vs post-publish verification
In the RFC, I propose that the macro author runs `cargo publish`, and it returns successfully without the published release appearing on [crates.io](http://crates.io) right away. Server-side verification runs asynchronously, just like docs.rs today.
I know little about [crates.io](http://crates.io) internals, but the following alternative might be easier to implement depending:
The author runs `cargo publish` and it _fails_ because the uploaded .crate has not _already_ been verified server-side. The author runs some other command first. The server-side build occurs, and [crates.io](http://crates.io) commits a checksum of the successfully verified .crate into a database. Only after this point may the author run `cargo publish`. The same .crate is produced again locally (or was cached) and this time the publish can succeed.
#### Alternative: no server-side verification
What's the point? It's sandboxed! Trust the ecosystem to audit, as they do with sources.
Is server-side verification prohibitively expensive in CPU time (unlikely compared to docs.rs which processes vastly more crates and traffic) or maintenance cost (a lot more likely)?
#### Alternative: should any of the restrictions be lifted now or later?
How valuable is it really to deny build scripts and proc macro dependencies during server-side verification? Is it trivial to implement verification in a hardened way? Docs.rs and crater do arbitrary remote code execution, but their blast radius may be small compared to smuggling through a malicious wasm.
#### Alternative: Cargo.toml syntax
This RFC proposes:
```toml
[lib]
proc-macro = true
sandboxed = true # <---
```
A more compact spelling would be:
```toml
[lib]
proc-macro = "sandboxed"
```
with the downside of being instantly incompatible with any pre-existing stable Cargo.
# Prior art
Compiler MCP: ["Build-time execution sandboxing"](https://github.com/rust-lang/compiler-team/issues/475)
**[Watt](https://github.com/dtolnay/watt)**. A 4-year-old working proof-of-concept that compiles procedural macros to WebAssembly and executes in either a from-source slow interpreter or an optimized Wasmtime runtime, complete enough to pass `serde_derive`'s test suite. Unfortunately we never got expansion performance good enough to ship officially in serde\_derive for the following reasons:
- Watt is not coupled with compiler version, so there end up being redundant IPCs: first between the Wasm code and the proc macro dylib, then between the dylib and rustc over proc macro bridge. This RFC proposes developing proc-macro-sandbox in concert with libproc\_macro which enables Wasm to communicate directly with rustc over host fns, eliminating the function of the proc macro bridge that a native proc macro would use. I am confident that expansion performance will be on par with natively compiled debug-mode macros, or better.
- The need for a build script to detect the presence of a suitable Wasmtime runtime on every macro call, and the need to compile all the macros dependencies unconditionally (syn) before Wasmtime detection applies, both eat away significantly from the benefit.
- The developer experience cannot be nearly as seamless as proposed by this RFC. Watt involved juggling a `[patch]` of the proc-macro2 crate to swap out its libproc\_macro-based implementation with Watt's stable Wasm-based one.
# Unresolved questions
- The optional feature situation. This RFC proposes precompiling exactly one feature combination, based on the feature flags passed to `cargo publish`; if someone's build uses the macro with some other choice of features enabled, they'll get a from-source build. Is this good enough? It is good enough for serde\_derive.
-
# Future possibilities
#### Future possibility: delete the whole thing!
If things go wrong, the Wasm spec becomes a disaster over time, stakeholders go bankrupt, Wasmtime goes unmaintained and no suitable replacement runtime emerges, ... what does Rust do?
This RFC has been designed to make this easy and inconsequential to deprecate and delete. `cargo publish` would inform you to remove `sandboxed = true` from your manifest, citing an explanatory blog post. Rustup would no longer ship the proc-macro-sandbox component and all macros would seamlessly resume building from source.
#### Future possibility: opt-in rejection of un-sandboxed macros
In the future, some users may want assurance that _all_ macros in their dependency graph **must** be sandboxed.
Maybe there'd be a setting in `~/.cargo/config.toml` that tells Cargo to refuse to build any unsandboxed macro (with exemption for local path dependencies).
Maybe we'd need a configurable allowlist of vetted unsandboxed macros, with all others being rejected.
---
### Author:  [@matklad](https://internals.rust-lang.org/u/matklad)
#### Post date: [August 20, 2023, 11:27pm UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/2 "2023-08-20T23:27:57Z")
😍
I still remeber how we spend weeks debugging mysterious crashes in rust-analyzer, which were heroically tracked down by [edwin0cheng (Edwin Cheng) · GitHub](https://github.com/edwin0cheng) to be a result of non-deterministic hash-map-iterating proc-macro breaking salsa's assumptions. So, very much in favor!
Two specific comments:
> Only one single precompiled build is produced. As part of publish, you control what features of your crate are enabled in that build, using publish's `--features` and `--all-features` and `--no-default-features` flags.
I think a more logical solution is to say that `--all-features` is used, period. This fits with Cargo's "additive features" model. We could further specify that this --all-features WebAssembly build can be used regardless of the set of actual features specified by the downstream user. I don't know if that's workable and if that's a good idea, but it certainly fits with the originally intended semantics of features.
One reason why it might not be workable is that the sandboxed and non-sandboxed builds could be different, but that's already the case, because proc macro can `cfg()` and just behave differently in wasm.
> Sandboxing mitigates the "arbitrary code" aspect of macros, making them easier to audit and trust.
I think this might want to be a bit more nuanced. Even sandboxed proc macros can inject arbitrary code into user's projects. This is _different_ from "normal" sandboxing security. So, auditing proc-macros is still required. What we guarantee is that the WASM blob is built by the official compiler from the source code uploaded to the [crates.io](http://crates.io). so its enough to validate only source code. And, frankly, the ecosystem could use a bit more auditing, as the recent large scale study very valuably demonstrates 😉
EDIT:
Also, I **really** love how this serves as a forcing function to have bit-for-bit reproducible builds for `.wasm`.
---
### Author:  [@jhpratt](https://internals.rust-lang.org/u/jhpratt)
#### Post date: [August 21, 2023, 12:53am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/3 "2023-08-21T00:53:43Z")
> [@dtolnay](#):
>
> Your macro must build using the latest stable release of Rust. This is the version that you must publish with. This is the version that [crates.io](http://crates.io) will very your build with.
Is this to avoid issues with regard to nightly features, similar to how vendor prefixes on the web were unintentionally and de facto standardized in the past?
> [@dtolnay](#):
>
> Users who depend on any other feature set will not get the precompiled build.
Am I correct in interpreting that this means they will get the macro built from source? How is a user to know which feature set it is precompiled for?
Also — why can't `cfg!` be made evaluated at runtime (by say deferring to a function), which would still allow behavior to change as necessary. That behavior changing could include emitting `compile_error!`, which would effectively be the same as gating the macro.
For `time-macros`, I use two feature gates (controlled entirely by the re-exporting crate). One of these simply enables/disables a macro, while the other controls whether certain inputs are valid or not.
> [@dtolnay](#):
>
> To opt-in to using these sandboxed macros in your Cargo builds when one is provided by the macro's author, run `rustup component add proc-macro-sandbox`. To stop using precompiled macros and go back to building all macro dependencies from source without isolation, run `rustup component remove proc-macro-sandbox`.
I'd vastly prefer there be a way to control this at the project or even dependency-level, rather than globally. I have a ton of Rust projects on my computer, but would not want this enabled for all of them.
> [@dtolnay](#):
>
> (a single one reused throughout the duration of the rustc execution)
Is it necessary to guarantee anything here? I see no immediately reason proc macros couldn't be expanded in parallel.
> [@dtolnay](#):
>
> the release appears in a permanent yanked-like status
This is _really_ unfortunate. Why can't the release be rejected outright? This would effectively pollute the version numbering (admittedly that is a small issue).
> [@dtolnay](#):
>
> - "Someone else is always auditing the code and will save me from anything bad in a macro before it would ever run on my machines." (At one point serde\_derive ran an untrusted binary for over 4 weeks across 12 releases before almost anyone became aware. This was plain-as-day code in the crate root; I am confident that professionally obfuscated malicious code would be undetected for years.)
Respectfully, it's almost certainly best to leave out any mentions of `serde_derive`. It's going to draw _a lot_ of (further) negative attention for little gain.
More generally, I find the layout of the drawbacks section very confusing. What purpose do the parentheticals serve? A rebuttal? It's not clear to me — a native English speaker.
> [@dtolnay](#):
>
> If things go wrong, the Wasm spec becomes a disaster over time, stakeholders go bankrupt, Wasmtime goes unmaintained and no suitable replacement runtime emerges, ... what does Rust do?
It's worth noting that the sandboxing doesn't _necessarily_ have to be through a specific runtime, or even wasm at all.
---
### Author:  [@workingjubilee](https://internals.rust-lang.org/u/workingjubilee)
#### Post date: [August 21, 2023, 1:41am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/4 "2023-08-21T01:41:06Z")
> "Someone else is always auditing the code and will save me from anything bad in a macro before it would ever run on my machines." (At one point serde\_derive ran an untrusted binary for over 4 weeks across 12 releases before almost anyone became aware. This was plain-as-day code in the crate root; I am confident that professionally obfuscated malicious code would be undetected for years.)
The first comment on this was over 3 weeks ago. People had indeed noticed. The onslaught of remarks followed your rejection of their pleas to reconsider, which was regarded as insulting by some. I detect a hint of mockery in this comment, which is unwarranted as it is based fundamentally on a lie. You should not misinterpret community outrage with your refusal as synonymous with people "noticing".
---
### Author:  [@kayabaNerve](https://internals.rust-lang.org/u/kayabaNerve)
#### Post date: [August 21, 2023, 2:00am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/5 "2023-08-21T02:00:54Z")
> [@dtolnay](#):
>
> At one point serde\_derive ran an untrusted binary for over 4 weeks across 12 releases before almost anyone became aware. This was plain-as-day code in the crate root; I am confident that professionally obfuscated malicious code would be undetected for years.
This is extremely misleading IMO. Multiple issues were opened weeks ago about your usage of binaries. It was solely the community-at-large unaware. If there was a reason to believe the binaries directly malicious, I personally believe the community-at-large would've been made aware with it much sooner.
According to pinkforest, you also did trip `cackle`, a tool for automatically checking if crates exceed their claimed scope.
* * *
If [crates.io](http://crates.io) wants to precompile macros in a reproducible environment, and offer them _as an opt-in feature_, I'd support it despite my complete and continued objections to what happened with serde\_derive on a professional level and the maintainer's actions on a personal level.
My sole notable objection to this RFC/pre-RFC has nothing to do with its content, yet rather the process of introducing what's widely considered a security issue into the ecosystem to then further justify changes to the toolchain, holding the security concerns over the ecosystem (RFC commentators, project members, implementers) in the process. It's effectively impossible to fairly review this on its merit now, nor to say it isn't being reviewed on an accelerated time span than it would otherwise have been.
For actual RFC feedback, I'd like to object to
> [@dtolnay](#):
>
> If the server-side build does not reproduce a .wasm artifact with the same exact content as the uploaded one, [crates.io](http://crates.io) is notified and the release appears in a permanent yanked-like status. It can be downloaded for forensic analysis using the usual download endpoint, but is impossible to pull into builds, and cannot be unyanked.
I believe this should be an error, not a yanked publication. There are several ways non-reproducible builds can be triggered. We shouldn't waste version numbers finding out a reproducible build isn't working when there's no benefit to existing with a yanked status _unless there's some intricacy to the [crates.io](http://crates.io) backend I'm unaware of_. I'll admit inexperience with it. jhpratt seems to have raised the same comment.
Then as a question, I'd like to ask how you plan to achieve reproducible wasm builds. From my understanding, depending on the platform built from, different wasm outputs will be created. Is there a proposed mechanism other than always building from x86\_64 (requiring CPU emulation, and not just containerization)?
I'd, personally, insist [crates.io](http://crates.io) does verification (not that my personal insistence means anything) in order to ensure publishers don't each setup their own build processes each needing their own replication. I'd also like to note the value in [crates.io](http://crates.io) rebuilding an uploaded artifact (not just performing the build) to ensure reproducible builds are possible (without multiple server-side runs).
As for watt, I do not believe it resolved reproducible builds from different host architectures.
---
### Author:  [@alice-i-cecile](https://internals.rust-lang.org/u/alice-i-cecile)
#### Post date: [August 21, 2023, 2:06am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/6 "2023-08-21T02:06:42Z")
> Both macro author and macro user must have opted-in for the sandboxed artifact to kick in
The motivation for macro users to have to opt-in is well-explained and clear, but after reading the existing proposal I don't fully understand why macro authors would choose not to (perhaps security reasons for exceptionally sensitive crates?). Obviously not all proc macros can be sandboxed, but this seems like something that can be automatically tried. Swapping to an opt-out mechanism for authors (not users!) seems like a sensible choice to me.
---
### Author:  [@ekuber](https://internals.rust-lang.org/u/ekuber)
#### Post date: [August 21, 2023, 2:07am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/7 "2023-08-21T02:07:53Z")
> [@dtolnay](#):
>
> Asynchronously, a crates.io-managed service that is conceptually similar to docs.rs will fetch these newly published pending proc macro releases, and build them using the latest stable Rust toolchain.
>
> As described in restrictions #1 and #2 in the guide-level explanation, this will _not_ involve arbitrary code execution on the server-side, as proc macro dependencies are not allowed and build scripts do not run. This greatly reduces the surface area for attacking this service. The only binaries running are a stable rustc and a stable Cargo.
>
> If the server-side build does not reproduce a .wasm artifact with the same exact content as the uploaded one, [crates.io](http://crates.io) is notified and the release appears in a permanent yanked-like status. It can be downloaded for forensic analysis using the usual download endpoint, but is impossible to pull into builds, and cannot be unyanked.
>
> If server-side verification successfully reproduces the uploaded .wasm, [crates.io](http://crates.io) is notified and the release becomes instantly available to users (and docs.rs) just as an ordinary upload would be.
What's the benefit of having the crate author provide a wasm file if [crates.io](http://crates.io) will have to rebuild them anyways? Why not publish the [crates.io](http://crates.io) generated binary?
> [@dtolnay](#):
>
> The optional feature situation. This RFC proposes precompiling exactly one feature combination, based on the feature flags passed to `cargo publish`; if someone's build uses the macro with some other choice of features enabled, they'll get a from-source build. Is this good enough? It is good enough for serde\_derive.
[crates.io](http://crates.io) has the entire reverse dependency graph for crates, including the features they set for their direct dependencies. Providing pre-compilation only for the top N crates/feature-set would make the compute cost bounded while maximizing the effect of "reduction in compile time of 'Can't Believe It's Not Std!' crates" while making it so crate authors don't have to do anything special (beyond meeting the restrictions in the reference-level explanation for compiler).
---
### Author:  [@kayabaNerve](https://internals.rust-lang.org/u/kayabaNerve)
#### Post date: [August 21, 2023, 2:13am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/8 "2023-08-21T02:13:21Z")
> [@ekuber](#):
>
> What's the benefit of having the crate author provide a wasm file if [crates.io](http://crates.io) will have to rebuild them anyways? Why not publish the [crates.io](http://crates.io) generated binary?
In order for [crates.io](http://crates.io) to practically ensure they're reproducible, it'd have to do multiple runs or have a run provided by another user (the publisher). While the other user could cheat, they can't cheat with security impact, only w.r.t. reproducibility (by forcing a match with [crates.io](http://crates.io) they couldn't actually achieve). The amount of users who'd modify cargo to do this is negligible.
---
### Author:  [@ekuber](https://internals.rust-lang.org/u/ekuber)
#### Post date: [August 21, 2023, 2:16am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/9 "2023-08-21T02:16:33Z")
Multiple compilations (from different environments) merely doubles the cost of this check, it doesn't materially change the costs of having [crates.io](http://crates.io) having to build. If almost every proc-macro crate would opt-in automatically, that would most likely be an even bigger set than the one that [crates.io](http://crates.io) would select.
---
### Author:  [@kpreid](https://internals.rust-lang.org/u/kpreid)
#### Post date: [August 21, 2023, 2:35am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/10 "2023-08-21T02:35:08Z")
Verified identical builds as part of `crates.io`'s duties sound possibly a bit difficult, but I would really like to see sandboxed proc-macros _even if_ they aren't precompiled and no performance benefits result.
It would reduce “opening a Rust project in an IDE can execute malware” to “opening a Rust project can execute malware if there's an exploitable bug in the compiler or if it includes a build script”. (Sandboxed build scripts are also desirable, but a much harder problem. Small steps forward.)
---
### Author:  [@kayabaNerve](https://internals.rust-lang.org/u/kayabaNerve)
#### Post date: [August 21, 2023, 2:36am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/11 "2023-08-21T02:36:22Z")
> [@ekuber](#):
>
> Multiple compilations (from different environments) merely doubles the cost of this check,
You can't check an item is reproducible if you only produce it once. You _need_ multiple complications (at least 2). This has [crates.io](http://crates.io) perform one and the publisher perform one.
---
### Author:  [@bb010g](https://internals.rust-lang.org/u/bb010g)
#### Post date: [August 21, 2023, 2:38am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/12 "2023-08-21T02:38:35Z")
> [@kayabaNerve](#):
>
> Then as a question, I'd like to ask how you plan to achieve reproducible wasm builds. From my understanding, depending on the platform built from, different wasm outputs will be created. Is there a proposed mechanism other than always building from x86\_64 (requiring CPU emulation, and not just containerization)?
Building with a sandboxed, `wasm32-wasi` or `wasm64-wasi` stable release of rustc & Cargo & friends would likely be ideal.
---
### Author:  [@steffahn](https://internals.rust-lang.org/u/steffahn)
#### Post date: [August 21, 2023, 4:13am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/13 "2023-08-21T04:13:20Z")
I haven't understood the story of rust versions from this RFC. Is the binary future-compatible with later Rust versions? Does this imply some sort of stable (WASM-) ABI for proc macros? Or will older crates eventually (silently?) fall back to from-source compilation? Surely [crates.io](http://crates.io) won't go ahead and the-compile all existing macros for all version. I feel like this should be explained explicitly in the RFC.
Also I don't like the failure conditions around using "the latest stable Rust version" around times of new Rust releases. Rust releases are often, we don't want to break people's publishes for a few days every 6 weeks because either they or [crates.io](http://crates.io) haven't updated yet (and the other one has) or because their crate waited in the queue post an update. Also what if the latest stable Rust release has some regression that wasn't noticed? Now suddenly it may have become _impossible_ to properly publish your proc macro crate until that's fixed (unless you work around the regression). So: How about supporting the latest 2 stable versions instead?
---
### Author:  [@jrose](https://internals.rust-lang.org/u/jrose)
#### Post date: [August 21, 2023, 4:13am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/14 "2023-08-21T04:13:54Z")
This is very neat and I’ll let people with better perspective on the necessary issues than me speak to it…except for one editorial note:
> [@dtolnay](#):
>
> Proc macros are never supposed to panic in the first place.
There are “panics that terminate the app” and “panics that unwind for a bit and then get caught”. I’ve never heard that the latter aren’t supposed to be used in proc-macros. I think it’s fine to say “it’s a limitation of the sandbox that there’s no unwinding”, but, well, I don’t think we know who’s using `catch_unwind` in a macro, and I don’t think I would categorically say it’s wrong to do so.
---
### Author:  [@jmjoy](https://internals.rust-lang.org/u/jmjoy)
#### Post date: [August 21, 2023, 6:03am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/15 "2023-08-21T06:03:06Z")
I think the root of the problem is that `build.rs` affects compile time and is not safe.
The wasm sandbox can solve compile time and security issues, but the source code is not visible.
If rust can be used as a sandboxed interpreter, and `build.rs` can be run as a script in the sandbox, it may perfectly solve all the above problems, but the cost is that the workload is huge.
---
### Author:  [@bjorn3](https://internals.rust-lang.org/u/bjorn3)
#### Post date: [August 21, 2023, 7:01am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/16 "2023-08-21T07:01:23Z")
> [@dtolnay](#):
>
> Proc macros are never supposed to panic in the first place.
For many proc macros it is the way that errors are reported.
> [@dtolnay](#):
>
> A new Tier 2 target platform is introduced, `wasm32-macro`. Almost everything about it is similar to `wasm32-unknown-unknown`, with the following exceptions:
Please also make the `extern "C"` ABI match the official C ABI for wasm like for wasm32-unknown-emscripten and wasm32-wasi rather than the weird ABI of wasm32-unknkwn-unknown.
> [@dtolnay](#):
>
> If any Wasm macro is passed to a rustc invocation, it must also be passed `-Zproc-macro-sandbox=` containing the path to a suitable Wasm runtime, normally installed by the user as a rustup component. A particular proc-macro-sandbox is specific to a particular version of rustc, just like proc-macro-srv is.
Any reason to not embed the wasm runtime in rustc itself?
> [@dtolnay](#):
>
> It implements a high-performance Wasm runtime based on Wasmtime, although this remains an implementation detail.
What about all the targets without wasmtime support?
Could the wasm file contain the exact rustc version it was built with as well as the Cargo.lock to allow reproduction by third parties?
> [@dtolnay](#):
>
> As described in restrictions #1 and #2 in the guide-level explanation, this will _not_ involve arbitrary code execution on the server-side, as proc macro dependencies are not allowed and build scripts do not run.
`proc_macro2` requires a build script, right? Wouldn't that make it impossible to use pretty much any proc macro with this?
---
### Author:  [@edward](https://internals.rust-lang.org/u/edward)
#### Post date: [August 21, 2023, 7:57am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/17 "2023-08-21T07:57:02Z")
> [@dtolnay](#):
>
> If the server-side build does not reproduce a .wasm artifact with the same exact content as the uploaded one, [crates.io](http://crates.io) is notified and the release appears in a permanent yanked-like status. It can be downloaded for forensic analysis using the usual download endpoint, but is impossible to pull into builds, and cannot be unyanked.
From what I understand reading both reference-level sections for authors and [crates.io](http://crates.io), I think what's being suggested is that publishing a sandboxed proc macro stages it for verification by the async service, and only when it gets reproduced does a new version get committed and published (using version control concepts). On failure then does an artifact download link becomes available to only the author for debugging purposes?
Or is the intent here that if reproducibility failed, it publishes regardless, perhaps with a "Reproducibility failed" marker?
I think the former is a reasonable approach, avoids versioning pollution, and allows [crates.io](http://crates.io) to limit the time these debugging artifacts can exist for (and thus keep costs down). It might be better to mention that `publish` is really a `stage` command in this case.
The latter interpretation I feel has a couple unresolved issues:
- Failed reproducibility artifacts has less value over time. If we assume the ideal scenario that the majority of users will opt-in to reproducible versions only and that non-reproduction is a bug, then these artifacts are likely only useful for the author to assist in resolving failures. I agree with the sentiment that in this failure mode it's better to reject the package all together.
- If there's a bug in the service where it begins to reject certain or all builds, then we're going to see a wide spread release of versions where reproducibility fails. This adds noise to any potential metrics while likely going to cause confusion/scare it brings to any less informed user or community. A failed publish instead would only notify the author, which would lead to a much smaller publicity blast zone.
> [@dtolnay](#):
>
> While usage of sandbox-incompatible standard library APIs such as `std::fs` or `std::thread` will not cause your macro to fail to build, such APIs unconditionally return an error or panic during macro expansion. All you get to do in a precompiled macro is inspect tokens and produce tokens/diagnostics.
Why not panic on build? I expect panic-on-build to be both better for the iterative development experience and remove ambiguity on if a function is permitted or not.
As maybe an interesting idea, would it be possible to add feature flags to `core`, or split `core` into even smaller? It sounds like the best solution here would be to have a subset of the `core` API, e.g. `proc-macro-core`.
* * *
Might be out of scope, but how is the signalling going to be for this feature? Once implemented there's going to be tension between promoting sandboxed proc macros as strictly better than non-sandboxed macros while also conveying that there might be legitimate reasons to not be sandboxed. In other words, is this RFC also asserting a stance that there is no legitimate reason to not sandbox?
---
### Author:  [@bjorn3](https://internals.rust-lang.org/u/bjorn3)
#### Post date: [August 21, 2023, 8:28am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/18 "2023-08-21T08:28:37Z")
> [@edward](#):
>
> As maybe an interesting idea, would it be possible to add feature flags to `core`, or split `core` into even smaller? It sounds like the best solution here would be to have a subset of the `core` API, e.g. `proc-macro-core`.
You mean splitting std? Splitting is impossible in many cases due to coherency preventing existing trait impls. I don't think anything in libstd other than the default allocator makes sense to be used in samdboxed proc macros. File reading should be done through a new proc macro specific api to allow rustc to declare to cargo that it needs to rebuild when the file changes. Intercepting `std::fs` wouldn't work when doing a non-sandboxed build.
---
### Author:  [@thomaseizinger](https://internals.rust-lang.org/u/thomaseizinger)
#### Post date: [August 21, 2023, 9:11am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/19 "2023-08-21T09:11:24Z")
The above proposal requires **opt-in** from both parties, the author and the user. Why do we need the opt-in from the author? As an analogy, I can attempt to compile a crate for a certain platform, without the author opting in. Can we apply the same concept to this and make it entirely a user-decision whether the macro is pre-compiled?
The pre-compilation could be done by a remote service that is invoked by [crates.io](http://crates.io). Until that remote service commits back the WASM artifact, users will only download the source and compile it locally.
The remote service would compile the crate (at least twice to ensure reproducibility) and notify [crates.io](http://crates.io) of the result. From then on, [crates.io](http://crates.io) can embed or link to the WASM artifact.
If it is the author's explicit intention to make the macro WASM compatible, they should have a CI job that verifies that, similar to compatibility with any other target.
This would reduce the surface area of this feature further. It might also be easier to experiment with because it could be built as a custom registry and a nightly flag in cargo to use the new WASM blobs.
---
### Author:  [@dlight](https://internals.rust-lang.org/u/dlight)
#### Post date: [August 21, 2023, 9:19am UTC](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359/20 "2023-08-21T09:19:35Z")
Here's my two cents.
Rustup distributes precompiled stdlib
Why can't rustup also distribute precompiled syn (for example?) The infrastructure to do so is already in place.
[Next page](https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros/19359.md?page=2)