Pre-issue: Feature Request: Give me the option to not build a target

tools and infrastructure cargo
1

I wrote out this Github issue, but then I realized since there is no single solution for this problem (I propose two below), I should take the Github-given advice of posting here first. Which approach below would be most likely to be seriously considered as an issue?

TLDR I want [target.'cfg(target_arch = "wasm32")'.lib].

Problem

If a Cargo.toml contains definitions for both an executable and a lib form, both targets must be run on every build. This is a problem because there are sensible situations, for example any application-shaped WebASM embed, where some targets (executable, static library, dynamic library) are only relevant on some platforms; and on the platforms where they are irrelevant, building them can result in spurious warnings, wasted time, or hypothetically even errors.

Scenario: I have a in-progress game/webgpu-graphics-demo app (current commit c6dcc33, simpler version here commit 3a0cb37). It is intended to run either as a standalone exe on desktop, or embedded in a browser window via wasm (where the build is done with wasm-pack and winit creates then draws to a canvas). Wasm embeds of this type always build as libs. So I have two platform targets: Windows 10, where the app is built as an exe and only an exe; and wasm, where the app is built as a lib and only a lib. There are no substantial code differences between the two platforms and both versions have a single entry point, fn main().

However, the mere presence of [lib] crate-type = ["cdylib"], necessary for wasm-pack to work, means that on desktop platforms I get an enormous number of warnings, for example this alarming one twice on every build:

warning: output filename collision.
The bin target `wgpu-hello` in package `wgpu-hello v0.1.0 (C:\Users\Andi\work\r\zap)` has the same output filename as the lib target `wgpu-hello` in package `wgpu-hello
 v0.1.0 (C:\Users\Andi\work\r\zap)`.
Colliding filename is: C:\Users\Andi\work\r\zap\target\debug\deps\wgpu_hello.pdb
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.

In addition there are unused-symbol warnings on every single global symbol, since although on a wasm-pack build it understands main is a used symbol in the lib, on the desktop lib build formally there are no used symbols at all.

I have surveyed the various methods of suppressing or "correcting" these warnings; all of them are very complicated, and all to some degree inadequate (see below). All workarounds for this feel unnecessary because it would be much simpler if Cargo gave me a way to do the thing I really want: Don't build a desktop lib target. After all, even if I could fix the lib/exe collision warnings, the desktop lib target would still be happening, and would still be undesirable.

Proposed Solution

The simplest and best solution, for my use case, would be if [target.'cfg(whatever)'.lib] worked in Cargo, the way that you can make target-specific "dependencies"/etc fields. (Currently such qualified lib fields do nothing; on desktop they result in warning: unused manifest key: target.cfg(target_arch = "wasm32").lib, in wasm-pack you don't even get this warning, just the failure.) This would also address some other noteworthy use cases such as wanting to suppress dynamic library builds on certain platforms.

Another workable solution would be if there were some command line I could pass to cargo that would suppress certain builds. For example, in cargo build --help one finds documented:

      --lib                     Build only this package's library
      --bin [<NAME>]            Build only the specified binary

It's a little unfortunate this requires me to look up the name of the executable before I build, but never mind that; this would resolve my problem if it worked. I would feel fine simply including build instructions with my app that specify a particular cargo invocation line. However, if I run cargo.exe build --bin wgpu-hello on either of my sample programs above, the --bin argument does not prevent a lib build. I get the "output filename collision" warning, I get the unused symbol warnings (warning: function `main` is never used), and I get an explicit notification the lib target was built (warning: `wgpu-hello` (lib) generated 29 warnings). (In my most recent clean-build test I do see no final rlib product shows up in target/, but the lib is clearly still being built, somewhere.) My current theory is that contra the documented behavior in --help, the [lib] in the Cargo.toml is somehow overriding the --bin request on the command line. But I have to include the [lib] because as far as I know that's the only way to set [lib]path=, required for wasm-pack/a lib build to work.

I would consider a solution based on .cargo inadequate, because it seems to me .cargo should not be checked into source control, and I would feel uncomfortable giving someone build instructions that require them to create .cargo files before building on every checkout.

Notes

While researching this problem, I believe I found a description of someone who had a project that was in some configurations built as a dynamic library and in other configurations built as a standalone binary for some platform (I believe embedded) which didn't support dynamic libraries at all. This is the basis of my claim above that the absence of this feature could result in errors. However, I am not sure I understood the posting I read correctly.

I am using Rust/cargo 1.69.0 on Windows 10.

Can this be achieved with existing features?

I asked for help with this in some Rust/Rust-adjacent discussion fora. I was given a lot of different suggestions as to how to work around the problem, which I would categorize into two buckets:

  • Various magic combinations of command line flags, environment variables etc— the problem is I tried all of these and they did not work. (In the sense that they did not result in me being able to build only the exe on desktop and lib for wasm, and/or fully avoid warnings for "irrelevant" targets). I could list the things I tried exhaustively if it would help. Note it's possible one of the stranger one of these tricks actually works and I was just doing it wrong, and if that turns out to be the case I apologize, but I would consider the ergonomics of all the stranger approaches I saw as poor so new feature work here might still be worthwhile. (In particular I did not fully investigate all the .cargo based suggestions I received— but as I described above, .cargo is a poor solution for a configuration which needs to run consistently and on every vcs checkout.)

  • "Do it correctly". The canonical "correct" way to do what I'm doing here is to split my project into two or three packages, each with their own Cargo.toml. Either there's a top-level rlib which is included into subdirectory projects that build an exe and a cdylib respectively, or a top level rlib+cdylib and a subdirectory project for an exe that includes the parent's rlib (and you just ignore the cdylib).

    Why do I resist this? Well, because I think it's fundamentally unreasonable to ask me to change the on-disk project structure to support a new hardware platform. Please excuse me if I get a bit argumentative here; I'm trying to anticipate objections I got to this sentiment in other places.

    The multi-package build approach is designed for (and in my opinion would surely be reasonable for) projects which actually have both a lib and an exe target: For example, a project which can be run standalone but is also designed to be incorporated into another program as a library. But my project only has one target on any given platform, it's just that whether that target is a lib or a exe depends on which platform (and I would consider the fact wasm binaries are libs to be something akin to an implementation detail; a single-entry-point wasm binary is formally a lib, but it sure doesn't act like one). If a program is conceived in a way that a single target platform would never have a reason to build both the lib and the exe target, the safety rails that encourage people to strictly separate lib and exe targets no longer make as much sense.

    The subdirectories are the problem here. If supporting both lib and exe targets required me to name my notional main function something other than "main", and then create a little fake stub "main" that does nothing but call into android_main() or whatever, I'd be a little annoyed but I'd do it. If I was required to create a realmain.rs or something so that the file main.rs could be reserved for my little fake exe-only stub "main", I'd be more than a little annoyed (I would like the option of writing simple programs with only a single source file) but I'd probably do it. If I was required to put all of my code for my executable, including the notional main function, into a file named lib.rs this would be like nails on a chalkboard to me, but I'd still probably do it. But the point I'm balking at, and the reason I'm running my project for the moment with thirty-ish disregarded warnings, would moving my project src/ directory or primary (executable) build into a subdirectory of its own repo (and because 1 Cargo.toml = 1 subdirectory that is what the "correct" solution here currently requires). That's just a weird requirement, and although my objections to it are primarily aesthetic there are situations where it could cause real problems. For example, imagine a program which is developed for three years as a desktop-only app and then has to add a wasm target; if this app is required to move existing files into subdirectories, this could create merge conflict problems for outstanding branches.

    Reshaping my repo in strange ways to support the build system or putting "the real project" in a subdirectory of a subdirectory is the kind of kludge I am used to from C or Java, and I've found Rust refreshing because this is rarely required. Usually Cargo allows simple projects to have simple structures. Meanwhile, Wasm/web is a major platform that's only going to become more important with time; if merely adding a wasm cross compile to a project requires moving files between directories, something went wrong somewhere.

Note, I think "trunk" somehow sidesteps this entire cluster of problems entirely. I'm not sure how as I haven't fully investigated trunk.

3 Likes
2

Last time I proposed a tiny improvement for lib + bin support, it was rejected, and the answer was that Cargo doesn't want to encourage this pattern any more:

lib + bin also creates problems for specifying dependencies, features, and required-features. So I suppose the preferred solution currently is to split the project into separate lib and bin crates.

And the issue 6313 is IMHO just a Cargo bug. Cargo chose to build executable and a DLL with the same name and put them in the the same directory, with debug information that has to have the same base filename. Despite what the warning suggests, there's no user error here. It's Cargo being Linux-centric in its design and trying to do something that is not appropriate on Windows. Windows won't budge, so Cargo has to change layout of its target dir, or dll naming scheme, or require an explicit target type selection instead of building all by default.

3

the answer was that Cargo doesn't want to encourage this pattern any more:

This was in 2018. Was it clear at this time that this pattern would be (from everything I can tell) required for wasm targets? Because that seems to create a legitimate use case for something that might otherwise be convincingly dubious.

To stress, the situation I'm in is more "lib | bin" than "lib + bin". My project is building both lib + bin now, and I don't want that. I want to suppress one of the two.

4

The --bin flag implicitly means "build lib, and then build the bin linked with the lib". This is to support the design where most of the code is in the lib, and the binary just adds a CLI UI and invokes the lib.

Cargo has autobins = false, but not autolibs. So I don't think it ever intended to support disabling the library part. And given the existing reluctance to support lib+bin, I would not expect proposal for "lib | bin" in a single crate to be accepted.

For WASM support I've had to separate crates myself, e.g. dssim bin+lib crate had to become a pair of dssim + dssim-core crates, cavif became cavif + ravif.

1 Like
5

The --bin flag implicitly means "build lib, and then build the bin linked with the lib".

That does make sense, but then why does the help text explicitly say "Build only the specified binary"?

6

It means "…as opposed to building all binaries of a multi-binary crate or workspace". I've proposed to update the help text.

1 Like
7

The fact that cdylib is a type of [lib] is a bit weird, in many ways it's closer to being a [[bin]] since it's a final artifact (in Cargo's POV) instead of an intermediate one.

1 Like
8

…huh.

Okay, so this is very weird to me, but

In contravention of what wasm-pack recommends, I changed my [lib] to

[lib]
crate-type = ["cdylib"]
path = "src/main.rs"

Once I had done this, cargo.exe build --bin wgpu-hello started working as expected; I get only:

warning: file `C:\Users\Andi\work\r\zap\src/main.rs` found to be present in multiple build targets:
  * `lib` target `wgpu-hello`
  * `bin` target `wgpu-hello`

…but this is much easier to ignore than the other warnings, and I can even address this by just adding one additional file. So that's very helpful!

However, this is very surprising! If this behavior is documented somewhere I don't know where I'd find it, and it seems to contravene some of the immediately-available documentation (EG, --bin [<NAME>] Build only the specified binary actually meaning "Build only the specified binary as well as any rlib targets, but NOT cdylib targets). And because I spoke to at least a dozen Rust people none of whom were aware of this loophole, this seems to be kind of deep secret magic here…

I think I will still file a version of the [target.'cfg(target_arch = "wasm32")'.lib]. bug…

9

I highly recommend Issue 8628 for an exploration of cdylib and [lib] vs [[bin]].

2 Likes
10

That link says

We've discussed this at the Cargo meeting, and the conclusion was that, while today src/lib.rs + src/main.rs is the default pattern for applications, we are actually not sure that this is the proper way to structure code, and not just a tradition and a historical accident.

That's from 2018. In the 5 years after that, many large projects didn't migrate into separating lib and main crates and many new projects are still being created unifying those crates. Learning materials don't mention this as an antipattern either. So I guess this feature is here to stay.

In this case maybe those shortcomings should just be fixed? Like, [features] applies to both, [features.lib] only to the lib, etc.

11

Note that you are still double-building the crate if you have the bin and lib target pointed at the same source file. The standard way to set up a dual mode crate is that you'd have pub fn main in src/lib.rs, and that src/main.rs is just a single fn main() { libname::main() }. It's not ideal to need the extra bit of boilerplate to make a dual-mode executable, but it works and doesn't end up building your code twice and throwing away one copy as unused/unreachable. (Not that it's any help on name collisions, though.) But it doesn't need any major restructuring of your project into multiple packages, either.

I think the real end goal resolution is that the bin target should eventually just do the right thing for wasm. It's just that what the right thing is isn't fully standard yet, so in the interim we're stuck doing it manually with cdylibs and shims.

1 Like
12

you are still double-building the crate if you have the bin and lib target pointed at the same source file

But why did it stop printing the target's warnings, then? (And only when I hit the very precise combination of "configured as cdylib only, no rlib" + " --bin wgpu-hello argument")

1 Like
13

I think I remember it working the last time I poked at wasm? But that's probably be experimenting with raw --target wasm32... not wasm-pack.

14

Except in a browser or a plugin or a game mod you don't just call main, you normally have an expected public API you must expose. For wasm32-wasi which does have the normal executable expectation of calling via main binary targets just work.

I'm not sure what "the right thing" even could be for wasm32-unknown-unknown, it's not a target that defines much standardized interaction with the host environment (it has just enough to make some bits of std work, a way to allocate memory + a way to park a thread).

15

Btw, OP, why not structure your project like this:

[workspace]

members = [
    # Here all your code.
    # Exposes single function that used in WASM
    "game_library", 
    # Game executable
    # Links with "game_library" and just calls its single public function.
    "game",

    # For any benches you used to control game.
    # In separate dir to avoid building criterion every time
    "benches"
]

I almost always end up with such structure even if I build an application and don't support WASM. Last time I ended with such structure because I wanted to make a procedural macro.