Pre-RFC: Add macros target to Cargo manifest

language design
1

Summary

Add a new target called [macros] to the cargo manifest that creates a procedural macro library target inside the crate. Allowing you to create, use, and export procedural macros without creating additional crates or needing a cargo workspace.

Context

With the stabilisation of attribute and function procedural macros, authors are writing and using more and more macros as part of their APIs.

Currently if you as a library author want to provide a derive macro implementation for your trait, or a compile time version of your API (e.g. we wanted to build our own std::env::var and env!) you have to convert your project into a cargo workspace, which means you'll have two separate peer crates (e.g. env and env-macro).

A common pattern has also emerged of exporting the proc macros through a top level crate (e.g. serde = { version = "*", features =["derive"] }). When you write a compile time version of your API you'll usually want to make a subset of your API available to both the proc macro and the top level crate so this requires three separate crates (e.g. unic-locale, unic-locale-impl, and unic-locale-macros).

Publishing 2–3 separate crates on crates.io can be a lot of small maintenance work, especially compared to just one crate. It would be nice if instead of having to specify proc-macro = true in [lib], we had a [macros] target that allowed for specifying procedural macros from inside a single crate.

Configuration

Macro specific dependencies can be specified with the [macro-dependencies].

Having a macros target also enables a "macros" feature flag.

Manifest example

[macros]
# Path to macro crate root.
# Default: `src/macros.rs` or `src/macros/lib.rs` 
path = "src/macros.rs"

Importing in Rust

The new macros target can be imported from the crate root as macros (e.g. use macros::{Deserialize, Serialize};)

use macros::{Deserialize, Serialize};
// You can publicly re-export macros with pub use.
pub use macros::{Deserialize, Serialize};

#[cfg_attr(feature = "macros", derive(Serialize))]
struct Point {
  x: u32,
  y: u32,
}

Unresolved Questions

  • Are there other options that could/should be added to a macros target?
  • In addition to the macros feature flag should there be a macros key in dependencies? This would be harder to mistype as cargo can warn on an unused key, but not on an unused feature. E.g.
serde = { version = "*", features = ["macros"] }
# With macros key
serde = { version = "*", macros = true }
15 Likes
2

One proc-macro lib can export as many macros as it wants. Is there a reason that you want to support multiple macro crate roots as opposed to just one?

3

If I understand correctly, each crate currently published to crates.io is built as exactly one crate by Cargo (e.g. you can't publish the bin or example` crates associated with a crate.

With this RFC, would a crate from crates.io now potentially 'expand' to multiple crates (the 'main' crate, and an internal proc-macro crate)?

Can dependencies be specified in the [[macros]] section?

4

Well with the additional configuration options, you'd want more control over which macros are affected by them. Such as which are publicly available and which are private.

I think this is aimed at reducing the "crate sprawl" that occurs because you have to split your crates along the implementation details rather than logic, and if it was limited to one, you'd still have to use a cargo workspace when you wanted to split them, and then you have to ask yourself whether to use this new format or use the existing proc-macro = true option, which would be worse in my opinion.

I would certainly like that, as that's my biggest gripe with [[bin]] is that you can't specify individual dependencies so you bloat the whole download size when downloading the library. However that's proabbly a larger RFC, and could be future work.

That's a good point. I was thinking of there also being a macros key in dependencies, so you can specify which your using from the Cargo.toml. So you can still get the dependency graph from the Cargo.toml as if it was another package on crates.io like it would be today.

[dependencies.serde]
version = "*"
macros = ["serialize", "deserialize"]
5

I believe that's:

2 Likes
6

One major issue with not being able to specify separate dependencies between proc-macros and the library crate is that you will end up building very big crates like syn twice, once for the target and once for the host.

You will also have the existing issue with target/host features being mixed continued on. AFAIK with the new resolver target/host features can be fully disambiguated (by either a marker on the edge with dependencies vs build-dependencies or a marker on a node with proc-macro = true), but if they are mixed into a single package's dependencies and shared between a normal crate and a proc-macro = true crate you lose that disambiguation.

5 Likes
7

Couldn't macros be handled similarly to tests, benchmarks and examples? Then we could simply add the macros to a macros folder in the root directory, and add dependencies to the [dev-dependencies] section.

2 Likes
8

I like the concept of this in general.

I would prefer to be able to group macros together in modules, rather than naming each individual macro. Perhaps it would make sense to have [[macros]] name a macro module (which may contain multiple macros), rather than an individual macro? If someone wants to break their macros up into one macro per module, they still can.

After an edition, we could potentially start inferring this from directory structure (as we do with bins).

1 Like
9

I think it'd be equally bad to mix it up with dev-dependencies as it would be to mix it up with regular dependencies, for the opposite reason. Mainly, right now crates can have all sorts of stuff in dev-dependencies that don't get compiled for their consumers, and thus don't matter much. But if macro dependencies were to be included there, then anyone depending on the crate would have to compile all of dev-dependencies, and that'd be a lot more test/benchmark/etc. crates which don't actually need to be compiled for the final product.

1 Like
10

One idea might be to mix proc-macro dependencies into build-dependencies, I'm not sure if the new resolver does this with existing proc-macros, or if it has separate "build-host" and "proc-macro-host" dependency sets, but I can't think of any downsides of mixing these together other than maybe confusion about the naming and interaction with the optionality of macros.

2 Likes
11

Or, we could just introduce [macro-dependencies] with no implied semantics wrt existing types of dependencies: they're dependencies for macros, no more, no less. I'll argue that that would be conceptually a bit cleaner, too.

5 Likes
12

I think macro targets should work exactly like build scripts, except they are compiled as a proc macro library instead of an executable, and exposed as a dependency to the main targets instead of run before building the targets.

To be concrete about what I mean:

  • Each package can have exactly one macros target, not many. There is a default path to find it at, which can be overriden with a packages.macros keys.
  • These packages are exposed to the library and binary targets as a dependency under a standard name (probably macros::). Users can rexport from this crate to expose them to downstream users. No magic re-export is done.
  • The macros target either has access to build-dependencies or to a new macros-dependencies section which functions from a resolution perspective just like build-dependencies except that one is exposed to the build script and one is exposed to the macros.

By making macros function just like build scripts except exposed as a library instead of executed before building, this is a reasonably scoped feature to add. But adding a new variadic target with its own configuration, sets of dependencies, etc, and adding new magic paths that have to be coordinated between cargo and rustc, greatly increase the scope of this feature for implementation & design. We should start with the easy version and can consider adding these bells and whistles later.

25 Likes
13

A possible solution is to give the macros a separate version, so you can increment the crate version without incrementing the macro version, for example:

[package]
name = "my_library"
version = "0.2.0"
edition = "2018"

[dependencies]
# dependency `my_macros` is implied

[features]
default = ["my_macros"]

[macros]
name = "my_macros"  # can be imported with `use my_macros::*`
version = "0.1.0"
path = "./macros"   # macros should be exported from `./macros/lib.rs`
optional = true     # this is an optional dependency

[macros.dependencies]
syn = "1"
14

Thank you everyone for your interest in this feature! Going through the feedback I'm inclined to agree that it should be a single target, and that dependencies should be specified with [macro-dependencies]. At least in cargo, you can achieve selectively enabled macros with feature flags.

So that I'm clear, a hypothetical serde import with this syntax would look like the following? If so I much prefer that. I believe “macros” isn’t currently reserved, though I’ll admit I don’t know if it needs to be to work.

use macros::serde::{Serialize, Deserialize};
use macro::serde::{Serialize, Deserialize};
15

No. The serde crate would have a new dependencies called macros. If it wants to expose those macros to other crates, it contains the line:

pub use macros::{Serialize, Deserialize};

And other users can now use serde::Serialize. But no one but serde can directly access serde's macro crate.

I suggested macros because its not a keyword; absolutely no change to rustc is necessary to implement this design. It's just cargo building another crate and then exposing it as a dep. It is, happily, a reserved name on crates.io, so there is and will be no macros crate in the public registry.

It's fine if someone who has a private dependency on a crate called macros can't use this feature until they rename that dependency, though we have to be careful not to break them either.

2 Likes
16

Want to explicitly mention that there are no IDE concerns about making proc-macros work roughly like build scripts.

5 Likes
17

Okay, I like that even more. I’ll update the top post in a bit with the all the feedback.

1 Like
18

Okay I’ve updated the top post with all the feedback.

19

The way I currently read the original post is that the macro doesn't have a particular version, as it's functionally private and only (potentially) exposed by the parent crate. Is that correct?

With regard to auto-enabling a feature flag, Given that it can already be done for other built-in crates, and this proposal otherwise behaves like a private built-in crate, I think that it makes sense to allow for a feature flag of the same name.

1 Like
20

I really like this Pre-RFC and am looking forward to simplify the amount of crate needed to be uploaded on crates.io. Is there any update on this, is this going to become a real RFC?

3 Likes