By default minimize build progress log

I'm starting a SDK which has a few dependencies that rely on other several dependencies, whether small or large dependencies, and I've been imagining that future users of my SDK will get annoyed to see the build progress logging the name of every dependency under the hood.

The dependency names are only logged by Cargo the first the project is built; however this occurs for every project.

It doesn't harm the SDK API, but it harms the eyes when building. For example, to get started, you'll typically run this in the command line:

> agera new photoshop
> cd photoshop
> cargo run

And then you'll see the name of every minor crate being compiled, usually common dependencies such as lazy_regex, once_cell, futures, tokio, maplit and hundreds more.

Small suggestions in case we were to hide these names during crate compilation:

Progress

Use a progress bar or icon based on crates_compiled_count / crate_total. Diagnostics such as compilation errors are still reported after progress is done.

Split progress

Reproduce something similiar to:

Downloading dependencies...
Compiling dependencies... (50%)
✓ Done compiling dependencies.

Compiling `actual_crate`...
✓ Done compiling `actual_crate`.

Therefore you will see only the name of the crate being compiled and not the dependencies, reducing the look of "dependency hell" (in my opinion).

When building in verbose mode, every individual dependency will log its name in the build.

1 Like

You can use the -q argument to suppress all messages about which crates are being compiled. Seems that it also suppresses the progress bar though.

2 Likes

I would be very much in favor of shrinking the output. I especially think of it with "cargo script" where a lot of cargo's output gets mxied with the users application.

In my ideal world, all output should be erased that isn't errors, warnings, or a summary line for the command being run.

Console output should better reflect the build process · Issue #8889 · rust-lang/cargo · GitHub is the closest issue for tracking this. The biggest concern is how well how well the fancier terminal control codes are supported across terminals. This isn't a question of "works in top 5 terminals" but the long tail and how to best gracefully degrade for that long tail.

4 Likes

Somewhat relatedly I just happened to be playing around with an old tool that renders the build-graph during the build, which is not something I think works well with typical rust projects with >500 dependencies, but is sort of interesting to play with.

asciicast

7 Likes

If you are trying to debug a CI build using only the logs, full details of every operation are frequently essential, so you want the exact opposite of this: you want the verbosity cranked up as far as it will go. (To a point. Only once in my career have I had to inject a wrapper script into the build environment to force gcc -v mode on. But it did happen once.)

Okay, sure, there's already a knob for that, right? But here's the catch: suppose your boss dumps a bug on you and it involves a broken CI build that ran with the default verbosity and that verbosity is not at least as high as "every single command is logged exactly as executed". Now you are, as they say, up shit creek without a paddle. I've had to deal with many such bugs in my time and my expectation is that each one will take me multiple days worth of failed manual CI invocations just to get the verbosity high enough that I can start debugging the actual problem. (N.B. I've mostly worked on very large systems, ones where a CI cycle typically takes an hour or more, and the build involves multiple nested layers of build drivers. If that number seems ridiculous, that's probably why.)

Is this a problem we should have in the first place? Absolutely not. Sourcehut's CI system obliterates it by letting you ssh into a failed build VM. But that's the only CI system I have ever seen with that feature, and there are a lot of other features it doesn't have, so it's not reasonable to expect everyone switch to Sourcehut right away.

Therefore: The default verbosity should be "every single command is logged exactly as executed". I consider it a bug that "cargo build" doesn't already do that. This does pose a small cognitive speedbump for new contributors, but one that can be easily mitigated by documenting that you should type "cargo build --quiet" most of the time (but never, ever, ever in CI).

2 Likes

imo we should be focused on the interactive case because having to pass a flag for every invocation of the interactive case is onerous while setting a flag in CI once means its set for all future runs.

t. This does pose a small cognitive speedbump for new contributors, but one that can be easily mitigated by documenting that you should type "cargo build --quiet" most of the time (but never, ever, ever in CI).

That is not a small cognitive bump but being actively hostile to users. That is in addition that --quiet is too quiet in most cases. I tried to experiment with "Cargo script" implying --quiet and I had to immediately turn it off because I could never tell if it was taking a long time to compile or my program was hung.

7 Likes

Cirrus ci has an option to rerun a build on debug mode where you can access a terminal while the build is running and after a build failed it will keep the VM up for another hour I believe to run commands. It is not automatic for every failed build like sourcehut, but still really useful.

1 Like

It would be possible to set a global config for this once as user, right?

1 Like

If this could be set in the config, yes. That would still be a terrible out of box experience if the first thing we tell a user after installing rust is to change a setting in cargo.

3 Likes

I want to second the view that the default mode should be the interactive one. Anything that’s typed in a script can be typed a little bit longer in a script, but anything someone types by hand should do something reasonable with the shortest invocation. (There are sometimes tricks one can play with tty-detection, but I would never want to rely on that.)

5 Likes

How are you eyes harmed? It seems more like an aesthetic preference than a harm. If anything I occasionally need more output (-v) to diagnose build issues.

I don't understand the argument. If dependencies are a real problem then hiding them from a build don't solve the problem, it merely ignores it.

5 Likes

I didn't say dependencies are a problem; I wanted to mean that running cargo build and getting something like this everytime for the first time in the terminal:

Downloading spaghetti-loader...
Downloading alface-num-....
Downloading beterraba-lazy-trash...
Downloading cell-vortex-ko...
Downloading dead-maze-xf...
Downloading blah-bindgen...
Downloading spaghetti-loader...
Downloading alface-num-....
Downloading beterraba-lazy-trash...
Downloading cell-vortex-ko...
Downloading dead-maze-xf...
Downloading blah-bindgen...
Downloading spaghetti-loader...
Downloading alface-num-....
Downloading beterraba-lazy-trash...
Downloading cell-vortex-ko...
Downloading dead-maze-xf...
Downloading blah-bindgen...
Downloading spaghetti-loader...
Downloading alface-num-....
Downloading beterraba-lazy-trash...
Downloading cell-vortex-ko...
Downloading dead-maze-xf...
Downloading blah-bindgen...
Downloading spaghetti-loader...
Downloading alface-num-....
Downloading beterraba-lazy-trash...
Downloading cell-vortex-ko...
Downloading dead-maze-xf...
Downloading blah-bindgen...
Downloading spaghetti-loader...
Downloading alface-num-....
Downloading beterraba-lazy-trash...
Downloading cell-vortex-ko...
Downloading dead-maze-xf...
Downloading blah-bindgen...
Compiling spaghetti-loader...
Compiling alface-num-....
Compiling beterraba-lazy-trash...
Compiling cell-vortex-ko...
Compiling dead-maze-xf...
Compiling blah-bindgen...
Compiling spaghetti-loader...
Compiling alface-num-....
Compiling beterraba-lazy-trash...
Compiling cell-vortex-ko...
Compiling dead-maze-xf...
Compiling blah-bindgen...
Compiling spaghetti-loader...
Compiling alface-num-....
Compiling beterraba-lazy-trash...
Compiling cell-vortex-ko...
Compiling dead-maze-xf...
Compiling blah-bindgen...
Compiling spaghetti-loader...
Compiling alface-num-....
Compiling beterraba-lazy-trash...
Compiling cell-vortex-ko...
Compiling dead-maze-xf...
Compiling blah-bindgen...
Compiling spaghetti-loader...
Compiling alface-num-....
Compiling beterraba-lazy-trash...
Compiling cell-vortex-ko...
Compiling dead-maze-xf...
Compiling blah-bindgen...
Compiling spaghetti-loader...
Compiling alface-num-....
Compiling beterraba-lazy-trash...
Compiling cell-vortex-ko...
Compiling dead-maze-xf...
Compiling blah-bindgen...

Is not pleasing.

It's true that it's more an aesthetic preference, but it does also avoid cluttering the user terminal, which is what most developers want.

1 Like

So.... only sometimes. Unless you're constantly busting your build cache, in case there is an underlying problem that might be worth fixing.

2 Likes

You get started with a new project, and you'll see this useless trash log.

It's more than aesthetic, information overload makes important information less obvious.

3 Likes

I want to agree with this position but, given my experiences, I can't. I just cannot trust anyone to explicitly select verbose modes for CI builds. I don't think I've ever seen CI builds changed away from the default log verbosity in real life.

Therefore, I have to insist that the default mode needs to be verbose, that it is not just a cosmetic problem but a bug that cargo's default mode does not print every single sub-command invocation exactly as executed, despite the negative consequences for interactive use.

That seems like a separate issue -- we should either adjust --quiet or add a new level of detail that's tuned for interactive builds.

1 Like

When getting started with a new project it can be useful to know what you're pulling in. Granted, it would be safer to run cargo tree before actually compiling anything, but I think it's good to know whether your database tool accidentally tries to build native ffmpeg or other funny things.

And calling things "trash" is... quite subjective and disparaging. I don't consider this a convincing argument.

3 Likes

However there are beginners who have no interest on seeing what's under the hood, so if one really wants, they use -v.

"must be removed/changed" does not follow from "someone momentarily happens to have no interest in some particular aspect".

There's plenty of information out there that's posted in case someone needs it but can be otherwise ignored.

1 Like

Wouldn't a minimal output in interactive mode and verbose mode in non-interactive mode solve the issue? (obviously with a way to force it with command line arguments and/or settings in cargo.toml)