Easier access to files generated by `--emit foo`

pacak · March 14, 2024, 7:13pm

You can pass --emit asm, for rustc to save generated asm into a file or a set of files. With --emit asm=foo.s rustc will save it into a file called foo.s but this doesn't work with multiple compilation units.

At the same time rustc takes --json=artifacts - with that it prints information about generated artifacts such as binaries or rlib files, but not files generated from --emit.

Does it make sense to extend definition of artifacts to files created with --emit? Or put them into a separate category such as byproducts?

Figuring out what was generated is very fragile without it.

bjorn3 · March 14, 2024, 9:29pm

--emit as whole doesn't support multiple codegen units. Even if you don't pass an explicit path, using it with a value other than link, dep-info or metadata, will force a single codegen unit.

pacak · March 14, 2024, 10:37pm

--emit as whole doesn't support multiple codegen units.

Hmm... I'm pretty sure it does at least for asm and llvm. Here's me running it with --emit asm and -Ccodegen-units=16. This is stable 1.76.0

/home/pacak/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc
 --crate-name sample                                     
 --edition=2021 src/lib.rs
 --error-format=json                                      
 --json=diagnostic-rendered-ansi,artifacts,future-incompat
 --diagnostic-width=239
 --crate-type lib             
 --emit=dep-info,metadata,link                            
 -C opt-level=3      
 -C embed-bitcode=no          
 -C debuginfo=line-tables-only
 --emit asm              
 -Ccodegen-units=16                
 -C llvm-args=-x86-asm-syntax=intel
 -Cdebuginfo=2             
 --cfg 'feature="default"'    
 --cfg 'feature="superbanana"'
 -C metadata=7aab9f7e7ab26e00       
 -C extra-filename=-7aab9f7e7ab26e00                                      
 --out-dir /home/pacak/ej/cargo-show-asm/sample/target/release/deps            
 -C incremental=/home/pacak/ej/cargo-show-asm/sample/target/release/incremental
 -L dependency=/home/pacak/ej/cargo-show-asm/sample/target/release/deps                                         
 --extern hashbrown=/home/pacak/ej/cargo-show-asm/sample/target/release/deps/libhashbrown-f438d6c1297e2efd.rmeta
 --extern rand_core=/home/pacak/ej/cargo-show-asm/sample/target/release/deps/librand_core-714dc04807e9dfc0.rmeta
 -C link-arg=-fuse-ld=mold   
 -C target-feature=+fma,+avx2      
 -C link-arg=-fuse-ld=mold                                          
 -C target-feature=+fma,+avx2

and here it's producing 12 files

% ls -lha sample/target/release/deps/*.s                                                                                                                                                                        
-rw-rw-r-- 1 pacak pacak  27K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.1bl96mmtwdnudhuu.rcgu.s
-rw-rw-r-- 1 pacak pacak  19K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.1hd1bp0zq09oqd1u.rcgu.s
-rw-rw-r-- 1 pacak pacak 295K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.1xvylk4b4eo4tugv.rcgu.s
-rw-rw-r-- 1 pacak pacak 6.6K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.3sihs2gbuvnioedi.rcgu.s
-rw-rw-r-- 1 pacak pacak  41K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.4377b0evtjly8432.rcgu.s
-rw-rw-r-- 1 pacak pacak 8.5K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.443ls6cv1jr14fkm.rcgu.s
-rw-rw-r-- 1 pacak pacak 3.9K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.474czdvrmobyb3yk.rcgu.s
-rw-rw-r-- 1 pacak pacak 290K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.4yrh78sryt68d3e.rcgu.s
-rw-rw-r-- 1 pacak pacak  12K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.53xe9n61hxojleor.rcgu.s
-rw-rw-r-- 1 pacak pacak  79K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.p5kssscqpxqjft.rcgu.s
-rw-rw-r-- 1 pacak pacak  54K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.q2h3w5hrtkqi50f.rcgu.s
-rw-rw-r-- 1 pacak pacak 6.3K Mar 14 18:28 sample/target/release/deps/sample-7aab9f7e7ab26e00.vemkzhxlqpx6apm.rcgu.s

If I run it with single codegen unit - it produces a single file with a different name and a bigger size:

-rw-rw-r-- 1 pacak pacak 584K Mar 14 18:35 sample/target/release/deps/sample-7aab9f7e7ab26e00.s

bjorn3 · March 15, 2024, 10:25am

Looks like the code to force a single codegen unit for certain --emit flags only runs when -o is used to specify a specific output file:

github.com

rust-lang/rust/blob/ee03c286cfdca26fa5b2a4ee40957625d2c826ff/compiler/rustc_session/src/config.rs#L2082-L2111


      
          // Issue #30063: if user requests LLVM-related output to one
          // particular path, disable codegen-units.
          let incompatible: Vec<_> = output_types
              .0
              .iter()
              .map(|ot_path| ot_path.0)
              .filter(|ot| !ot.is_compatible_with_codegen_units_and_single_output_file())
              .map(|ot| ot.shorthand())
              .collect();
          if !incompatible.is_empty() {
              match codegen_units {
                  Some(n) if n > 1 => {
                      if matches.opt_present("o") {
                          for ot in &incompatible {
                              early_dcx.early_warn(format!(
                                  "`--emit={ot}` with `-o` incompatible with \
                                       `-C codegen-units=N` for N > 1",
                              ));
                          }
                          early_dcx.early_warn("resetting to default -C codegen-units=1");

This file has been truncated. show original

It does result in a warning though: "warning: ignoring emit path because multiple .s files were produced"

system · June 13, 2024, 10:25am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Interaction between incremental compilation and --emit compiler	1	300	June 26, 2024
RFC: --emit check?	5	995	August 15, 2021
Make it easier to include generated files as module? language design	2	1781	March 25, 2019
Rustc can be "tricked" into generating exponentially large debug info	5	1499	March 25, 2019
Project idea: datalog output from rustc compiler	19	4915	March 25, 2019

Easier access to files generated by `--emit foo`

Related topics