Problems With Using include!() in build.rs for .rs Code


#1

Problems With Using include!() in build.rs for .rs Code

Introduction

This is an overview of problems I’ve been facing when using include!() from build.rs to build Rust code. I don’t have any good suggestions on how to resolve these problems, so this is more of a bug report than anything else.

Most of the problems in this issue have been encountered while using build.rs to create external files for CLI apps. Examples include: generating man pages, generating shell completions, and updating README.md ## Usage headings.

I’ve split this report up into 3 distinct interactions that I have had problems with. I hope that with sharing my experiences I can help people understand why this is problematic, and kick off a discussion on how to improve some of these interactions.

Feedback and questions would be very welcome!

Dependency Duplication

Summary

When using include!() in build.rs to import a file from src/, dependencies must be declared twice in Cargo.toml: once under [dependencies], and once under [build-dependencies]. If this is not done, the rustc will refuse to compile, and mention there are missing dependencies.

Expected Behavior

We would expect the following code to work:

build.rs

Here we take a structopt instance, and iterate over it to write shell completions to disk. We can later include those in our release tarballs.

use std::str::FromStr;
use structopt::{clap::Shell, StructOpt};

include!("src/lib.rs");

fn main() -> std::result::Result<(), Box<dyn std::error::Error>> {
  let outdir = ::std::env::var_os("OUT_DIR").expect("OUT_DIR not found.");
  let mut app = cli::Cli::clap();
  for shell in &Shell::variants() {
    let shell = Shell::from_str(*shell)?;
    app.gen_completions(env!("CARGO_PKG_NAME"), shell, &outdir);
  }
  Ok(())
}

Cargo.toml

Because build.rs uses structopt directly, under current dependency rules we might expect it to be required in [build-dependencies]. Coming from Node.js, however, I would intuitively have expected structopt to only need to be declared once in Cargo.toml, and be available main.rs, lib.rs and build.rs.

[package]
name = "test"
version = "0.3.2"
license = "MIT OR Apache-2.0"

[dependencies]
failure = "0.1.2"
structopt = "0.2.10"
clap_flags = "0.3.0"
git2 = "0.7.5"

[dev-dependencies]

[build-dependencies]
structopt = "0.2.10"

Current Behavior

In order to get the build.rs file above to compile, all dependencies must be duplicated under [build-dependencies].

This not only makes it harder to distinguish between build dependencies, and regular dependencies. It also makes it hard to add new dependencies (even with automation such as cargo-edit), and update existing dependencies. For example dependabot only upgrades one dependency at a time, which makes automation harder to apply.

Cargo.toml

[package]
name = "test"
version = "0.3.2"
license = "MIT OR Apache-2.0"

[dependencies]
failure = "0.1.2"
structopt = "0.2.10"
clap_flags = "0.3.0"
git2 = "0.7.5"

[dev-dependencies]

[build-dependencies]
failure = "0.1.2"
structopt = "0.2.10"
clap_flags = "0.3.0"
git2 = "0.7.5"

Hygiene Problems

Summary

If a dependency is declared in a source file, it can break the build file without warning.

Expected Behavior

When we use include!() in build.rs code should just work.

lib.rs

extern crate structopt;

pub mod cli {
  pub struct Cli {}
}

build.rs

extern crate structopt;

use std::str::FromStr;
use structopt::{clap::Shell, StructOpt};

include!("src/lib.rs");

fn main() -> std::result::Result<(), Box<dyn std::error::Error>> {
  let outdir = ::std::env::var_os("OUT_DIR").expect("OUT_DIR not found.");
  let mut app = cli::Cli::clap();
  for shell in &Shell::variants() {
    let shell = Shell::from_str(*shell)?;
    app.gen_completions(env!("CARGO_PKG_NAME"), shell, &outdir);
  }
  Ok(())
}

Current Behavior

Instead the code declared above will throw an error, stating dependencies are being imported twice. This seems to be because include!() simply concatenates a target file into build.rs, skipping any isolation mechanisms as are the case with mod and extern crate.

This also means that changing lib.rs code might suddenly cause build.rs to start failing, making the setup brittle.

Error Reporting Problems

Summary

When an error occurs in build.rs because of dependencies, the resulting error is unclear that this is happening because of a problem in build.rs.

Expected Behavior

When an error occurs inside build.rs using include!(), the output should point us to build.rs as the root of the error.

Current Behavior

Instead because of how include!() simply concatenates files, the error can highlight the included file instead, causing some confusion as to what is going on.

References


#2

Why not split the code you want to include into a separate crate?


#3

That’s a good question! Let me pull up an example here: changelog(1). It includes the following key files:

Together these files form a coherent whole. lib.rs stores all the core logic, and main.rs exists to front all the logic and create a CLI interface. The CLI struct needs to be part of lib.rs's tree so it can be called from both main.rs, and build.rs.

The CLI struct here also has methods on it, which expose errors as defined in src/errors.rs. That means that any refactoring would require duplicating most of the error handling code.

In other applications I’ve also been exploring a pattern of adding more application-specific methods to the CLI struct. This is quite nice, as it can make main.rs more readable, and opens up exploration for further code reuse.

Now to answer your question: yes, it’s probably possible to move code to a separate crate. But it comes at a cost. Which I guess is the main point of this issue: if you want to use a build.rs script to generate assets from your source code, there’s a cost involved.

I think it would be great if we could explore ways to remove that cost.