# Pre-RFC: \`cargo-script\` for everyone

**URL:** <https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639>\
**Category:** cargo\
**Created:** [April 7, 2023, 8:20pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639 "2023-04-07T20:20:19Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![epage](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/epage/32/3171_2.png) [@epage](https://internals.rust-lang.org/u/epage)\
**Post date:** [April 7, 2023, 8:20pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/1 "2023-04-07T20:20:19Z")

</div>

- Feature Name: cargo-script
- Start Date: 2023-03-31
- [Source](https://github.com/epage/cargo-script-mvs/blob/main/0000-cargo-script.md) ([edit history](https://github.com/epage/cargo-script-mvs/compare/1bf0dfec25b7543d03ab53765b39c1a40d2f32ef...main))
- [Demo](https://crates.io/crates/cargo-script-mvs)

# Summary

This adds support for so called single-file packages in cargo. Single-file packages are `.rs` files with an embedded manifest. These will be accepted with just like `Cargo.toml` files with `--manifest-path`. `cargo` will be modified to accept `cargo <file>.rs` as a short-cut to `cargo run --manifest-path <file>.rs`. This allows placing `cargo` in a `#!` line for directly running these files.

Example:

````rust
#!/usr/bin/env cargo

//! ```cargo
//! [dependencies]
//! clap = { version = "4.2", features = ["derive"] }
//! ```

use clap::Parser;

#[derive(Parser, Debug)]
#[clap(version)]
struct Args {
    #[clap(short, long, help = "Path to config")]
    config: Option<std::path::PathBuf>,
}

fn main() {
    let args = Args::parse();
    println!("{:?}", args);
}

````

```console
$ ./prog --config file.toml
Args { config: Some("file.toml") }

```

See [`cargo-script-mvs`](https://crates.io/crates/cargo-script-mvs) for a demo.

# Motivation

**Collaboration:**

When sharing reproduction cases, it is much easier when everything exists in a single code snippet to copy/paste. Alternatively, people will either leave off the manifest or underspecify the details of it.

This similarly makes it easier to share code samples with coworkers or in books / blogs when teaching.

**Interoperability:**

One angle to look at including something is if there is a single obvious solution. While there isn't in the case for single-file packages, there is enough of a subset of one. By standardizing that subset, we allow greater interoperability between solutions (e.g. [playground could gain support](https://users.rust-lang.org/t/call-for-contributors-to-the-rust-playground-for-upcoming-features/87110/14?u=epage) ). This would make it easier to collaborate..

**Prototyping:**

Currently to prototype or try experiment with APIs or the language, you need to either

- Use the playground
  - Can't access local resources
  - Limited in the crates supported
  - _Note:_ there are alternatives to the playground that might have fewer restrictions but are either less well known or have additional complexities.

- Find a place to do `cargo new`, edit `Cargo.toml` and `main.rs` as necessary, and `cargo run` it, then delete it
  - This is a lot of extra steps, increasing the friction to trying things out
  - This will fail if you create in a place that `cargo` will think it should be a workspace member

By having a single-file project,

- It is easier to setup and tear down these experiments, making it more likely to happen
- All crates will be available
- Local resources are available

**One-Off Utilities:**

It is fairly trivial to create a bunch of single-file bash or python scripts into a directory and add it to the path. Compare this to rust where

- `cargo new` each of the "scripts" into individual directories
- Create wrappers for each so you can access it in your path, passing `--manifest-path` to `cargo run`

# Guide-level explanation

### Creating a New Package

_(Adapted from [the cargo book](https://doc.rust-lang.org/cargo/guide/creating-a-new-project.html))_

To start a new [package](https://doc.rust-lang.org/cargo/guide/../appendix/glossary.html#package ""package" (glossary entry)") with Cargo, create a file named `hello_world.rs`:

```rust
#!/usr/bin/env cargo

fn main() {
    println!("Hello, world!");
}

```

Let's run it

```console
$ chmod +x hello_world.rs
$ ./hello_world.rs
Hello, world!

```

### Dependencies

_(Adapted from [the cargo book](https://doc.rust-lang.org/cargo/guide/dependencies.html))_

[crates.io](https://crates.io/) is the Rust community's central [_package registry_](https://doc.rust-lang.org/cargo/guide/../appendix/glossary.html#package-registry ""package-registry" (glossary entry)") that serves as a location to discover and download [packages](https://doc.rust-lang.org/cargo/guide/../appendix/glossary.html#package ""package" (glossary entry)"). `cargo` is configured to use it by default to find requested packages.

#### Adding a dependency

To depend on a library hosted on [crates.io](https://crates.io/), you modify `hello_world.rs`:

````rust
#!/usr/bin/env cargo

//! ```cargo
//! [dependencies]
//! time = "0.1.12"
//! ```

fn main() {
    println!("Hello, world!");
}

````

The `cargo` section is called a [_ **manifest** _](https://doc.rust-lang.org/cargo/guide/../appendix/glossary.html#manifest ""manifest" (glossary entry)"), and it contains all of the metadata that Cargo needs to compile your package. This is written in the [TOML](https://toml.io/) format (pronounced /tɑməl/).

`time = "0.1.12"` is the name of the [crate](https://doc.rust-lang.org/cargo/guide/../appendix/glossary.html#crate ""crate" (glossary entry)") and a [SemVer](https://semver.org) version requirement. The [specifying dependencies](https://doc.rust-lang.org/cargo/guide/../reference/specifying-dependencies.html) docs have more information about the options you have here.

If we also wanted to add a dependency on the `regex` crate, we would not need to add `[dependencies]` for each crate listed. Here's what your whole `hello_world.rs` file would look like with dependencies on the `time` and `regex` crates:

````rust
#!/usr/bin/env cargo

//! ```cargo
//! [dependencies]
//! time = "0.1.12"
//! regex = "0.1.41"
//! ```

fn main() {
    let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap();
    println!("Did our date match? {}", re.is_match("2014-01-01"));
}

````

You can then re-run this and Cargo will fetch the new dependencies and all of their dependencies. You can see this by passing in `--verbose`:

```console
$ cargo eval --verbose ./hello_world.rs
      Updating crates.io index
   Downloading memchr v0.1.5
   Downloading libc v0.1.10
   Downloading regex-syntax v0.2.1
   Downloading memchr v0.1.5
   Downloading aho-corasick v0.3.0
   Downloading regex v0.1.41
     Compiling memchr v0.1.5
     Compiling libc v0.1.10
     Compiling regex-syntax v0.2.1
     Compiling memchr v0.1.5
     Compiling aho-corasick v0.3.0
     Compiling regex v0.1.41
     Compiling hello_world v0.1.0 (file:///path/to/package/hello_world)
Did our date match? true

```

## Package Layout

_(Adapted from [the cargo book](https://doc.rust-lang.org/cargo/guide/project-layout.html))_

When a single file is not enough, you can separately define a `Cargo.toml` file along with the `src/main.rs` file. Run

```console
$ cargo new hello_world --bin

```

We’re passing `--bin` because we’re making a binary program: if we were making a library, we’d pass `--lib`. This also initializes a new `git` repository by default. If you don't want it to do that, pass `--vcs none`.

Let’s check out what Cargo has generated for us:

```console
$ cd hello_world
$ tree .
.
├── Cargo.toml
└── src
    └── main.rs

1 directory, 2 files

```

Unlike the `hello_world.rs`, a little more context is needed in `Cargo.toml`:

```toml
[package]
name = "hello_world"
version = "0.1.0"
edition = "2021"

[dependencies]

```

Cargo uses conventions for file placement to make it easy to dive into a new Cargo [package](https://doc.rust-lang.org/cargo/guide/../appendix/glossary.html#package ""package" (glossary entry)"):

```plaintext
.
├── Cargo.lock
├── Cargo.toml
├── src/
│ ├── lib.rs
│ ├── main.rs
│ └── bin/
│ ├── named-executable.rs
│ ├── another-executable.rs
│ └── multi-file-executable/
│ ├── main.rs
│ └── some_module.rs
├── benches/
│ ├── large-input.rs
│ └── multi-file-bench/
│ ├── main.rs
│ └── bench_module.rs
├── examples/
│ ├── simple.rs
│ └── multi-file-example/
│ ├── main.rs
│ └── ex_module.rs
└── tests/
    ├── some-integration-tests.rs
    └── multi-file-test/
        ├── main.rs
        └── test_module.rs

```

- `Cargo.toml` and `Cargo.lock` are stored in the root of your package (_package root_).
- Source code goes in the `src` directory.
- The default library file is `src/lib.rs`.
- The default executable file is `src/main.rs`.
  - Other executables can be placed in `src/bin/`.

- Benchmarks go in the `benches` directory.
- Examples go in the `examples` directory.
- Integration tests go in the `tests` directory.

If a binary, example, bench, or integration test consists of multiple source files, place a `main.rs` file along with the extra [_modules_][def-module] within a subdirectory of the `src/bin`, `examples`, `benches`, or `tests` directory. The name of the executable will be the directory name.

You can learn more about Rust's module system in [the book](https://doc.rust-lang.org/cargo/guide/../../book/ch07-00-managing-growing-projects-with-packages-crates-and-modules.html).

See [Configuring a target](https://doc.rust-lang.org/cargo/guide/../reference/cargo-targets.html#configuring-a-target) for more details on manually configuring targets. See [Target auto-discovery](https://doc.rust-lang.org/cargo/guide/../reference/cargo-targets.html#target-auto-discovery) for more information on controlling how Cargo automatically infers target names.

# Reference-level explanation

## Single-file packages

In addition to today's multi-file packages (`Cargo.toml` file with other `.rs` files), we are adding the concept of single-file packages which may contain an embedded manifest. There is no required distinguishment for a single-file `.rs` package from any other `.rs` file.

A single-file package may contain an embedded manifest. An embedded manifest is stored using `TOML` in a markdown code-fence with `cargo` at the start of the infostring inside a target-level doc-comment. It is an error to have multiple `cargo` code fences in the target-level doc-comment. We can relax this later, either merging the code fences or ignoring later code fences.

Supported forms of embedded manifest are:

````rust
//! ```cargo
//! ```

````

````rust
/*!
```cargo
```
*/

````

````rust
/*!
 * ```cargo
 * ```
 */

````

Inferred / defaulted manifest fields:

- `package.name = <slugified file stem>`
- `package.version = "0.0.0"` to [call attention to this crate being used in unexpected places](https://matklad.github.io/2021/08/22/large-rust-workspaces.html#Smaller-Tips)
- `package.publish = false` to avoid accidental publishes, particularly if we later add support for including them in a workspace.
- `package.edition = <current>` to avoid always having to add an embedded manifest at the cost of potentially breaking scripts on rust upgrades
  - Warn when `edition` is unspecified. While with single-file packages this will be silenced by default, users wanting stability are also likely to be using other commands, like `cargo test` and will see it.

Disallowed manifest fields:

- `[workspace]`, `[lib]`, `[[bin]]`, `[[example]]`, `[[test]]`, `[[bench]]`
- `package.workspace`, `package.build`, `package.links`, `package.autobins`, `package.autoexamples`, `package.autotests`, `package.autobenches`

As the primary role for these files is exploratory programming which has a high edit-to-run ratio, building should be fast. Therefore `CARGO_TARGET_DIR` will be shared between single-file packages to allow reusing intermediate build artifacts.

A single-file package is accepted by cargo commands as a `--manifest-path`

- This is distinguished by the file extension (`.rs`) and that it is a file.
- This allows running `cargo test --manifest-path single.rs`
- `cargo package` / `cargo publish` will normalize this into a multi-file package
- `cargo add` and `cargo remove` may not support editing embedded manifests initially
- Path-dependencies may not refer to single-file packages at this time (they don't have a `lib` target anyways)

The lockfile for single-file packages will be placed in `CARGO_TARGET_DIR`. In the future, when workspaces are supported, that will allow a user to have a persistent lockfile.

## `cargo <file>.rs`

`cargo` is intended for putting in the `#!` for single-file packages:

```rust
#!/usr/bin/env cargo

fn main() {
    println!("Hello world");
}

```

This command will have the same behavior as running

```console
$ RUST_BACKTRACE=1 cargo run --quiet --manifest-path <file.rs> -- <args>`.

```

- `--release` is not passed in because the primary use case is for exploratory programming, so the emphasis will be on build-time performance, rather than runtime performance
- `RUST_BACKTRACE=1` will be enabled by default (allowing the caller to override it) to help exploratory programming by making it quicker to debug panics.
- `--quiet` is enabled by default so as to not mix cargo and user output. On success, `cargo` will print nothing while error messages will be shown on failure. In the future, we can explore showing progress bars if `stdout` is interactive but they will be cleared by the time cargo is done. A single `--verbose` will restore normal output and subsequent `--verbose`s will act l ike normal.

Most other flags and behavior will be similar to `cargo run`.

# Drawbacks

At the moment, the doc-comment parsing is brittle, relying on regexes, to extract it and then requires a heavy dependency (a markdown parser) to get the code fence.

The implicit content of the manifest will be unclear for users. We can patch over this as best we can in documentation but the result won't be ideal.

The `bin.name` assigned to the script included a hash as an implementation detail of the shared cache (for improving build times). This makes programmatic choices off of `argv[0]` not work like normal (e.g. multi-call binaries). We could settings [`argv[0]` on unix-like systems](https://doc.rust-lang.org/std/os/unix/process/trait.CommandExt.html#tymethod.arg0) but could not find something similar for Windows.

This increases the maintenance and support burden for the cargo team, a team that is already limited in its availability.

Like with all cargo packages, the `target/` directory grows unbounded. Some prior art include a cache GC but that is also to clean up the temp files stored in other locations (our temp files are inside the `target/` dir and should be rarer).

Syntax is not reserved for `build.rs`, `[lib]` support, proc-maros, or other functionality to be added later with the assumption that if these features are needed, a user should be using a multi-file package.

# Rationale and alternatives

Guidelines used in design decision making include

- Single-file packages should have a first-class experience
  - Provides a higher quality of experience (doesn't feel like a hack or tacked on)
  - Transferable knowledge, whether experience, stackoverflow answers, etc
  - Easier unassisted migration between single-file and multi-file packages
  - The more the workflows deviate, the higher the maintenance and support costs for the cargo team
  - Example implications:
    - Workflows, like running tests, should be the same as multi-file packages rather than being bifurcated
    - Manifest formats should be the same rather than using a specialized schema or data format

- Friction for starting a new single-file package should be minimal
  - Easy to remember, minimal syntax so people are more likely to use it in one-off cases, experimental or prototyping use cases without tool assistance
  - Example implications:
    - Embedded manifest is optional which also means we can't require users specifying `edition`
    - See also the implications for first-class experience
    - Workspaces for single-file packages should not be auto-discovered as that will break unless the workspaces also owns the single-file package which will break workflows for just creating a file anywhere to try out an idea.

- Cargo/rustc diagnostics and messages (including `cargo metadata`) should be in terms of single-file packages and not any temporary files
  - Easier to understand the messages
  - Provides a higher quality of experience (doesn't feel like a hack or tacked on)
  - Example implications:
    - Most likely, we'll need single-file packages to be understood directly by rustc so cargo doesn't have to split out the `.rs` content into a temp file that gets passed to cargo which will cause errors to point to the wrong file
    - Most likely, we'll want to muck with the errors returned by `toml_edit` so we render manifest errors based on the original source code which will require accurate span information.

## Embedded Manifest Format

Considerations for embedded manifest include

- How obvious it is for new users when they see it
- How easy it is for newer users to remember it and type it out
- How machine editable it is for `cargo add` and friends
- Needs to be valid Rust code based on the earlier stated design guidelines
- Lockfiles might also need to reuse how we attach metadata to the file

**Option 1: Doc-comment**

````rust
#!/usr/bin/env cargo

//! ```cargo
//! [package]
//! edition = "2018"
//! ```

fn main() {
}

````

- This has the advantage of using existing, familiar syntax both to read and write.
- Could use `syn` to parse to get the syntax correct
- Might be a bit complicated to do edits (translating between location within `toml_edit` spans to the location within `syn` spans)
- Requires pulling in a full markdown parser to extract the manifest

**Option 2: Macro**

```rust
#!/usr/bin/env cargo

cargo! {
[package]
edition = "2018"
}

fn main() {
}

```

- The `cargo` macro would need to come from somewhere (`std`?) which means it is taking on `cargo`-specific knowledge
- A lot of tools/IDEs have problems in dealing with macros
- Free-form rust code makes it harder for cargo to make edits to the manifest

**Option 3: Attribute**

```rust
#!/usr/bin/env cargo

#![cargo(manifest = r#"
[package]
edition = "2018"
"#)]

fn main() {
}

```

- `cargo` could register this attribute or `rustc` could get a generic `metadata` attribute
- I posit that this syntax is more intimidating to read and write for newer users
- As an alternative, `manifest` could a less stringly-typed format but that makes it harder for cargo to parse and edit, makes it harder for users to migrate between single and multi-file packages, and makes it harder to transfer knowledge and experience

**Option 4: Presentation Streams**

YAML allows several documents to be concatenated together variant [presentation streams](https://yaml.org/spec/1.2.2/#323-presentation-stream) which might seem familiar as this is frequently used in static-site generators for adding frontmatter to pages. What if we extended Rust's syntax to allow something similar?

```rust
#!/usr/bin/env cargo

fn main() {
}

---Cargo.toml
[package]
edition = "2018"

```

- Easiest for machine parsing and editing
- Flexible for manifest, lockfile, and other content
- Being new syntax, there would be a lot of details to work out, including
  - How to delineate and label documents
  - How to allow escaping to avoid conflicts with content in a documents
  - Potentially an API for accessing the document from within Rust

- Unfamiliar, new syntax, unclear how it will work out for newer users

**Option 5: Regular Comment**

The manifest can be a regular comment with a header. If we were to support multiple types of content (manifest, lockfile), we could either use multiple comments or HEREDOC.

Open questions

- Which syntax to use
- Which comment types would be supported

Simple header:

```rust
#!/usr/bin/env cargo
/* Cargo.toml:
[package]
edition = "2018"
*/

fn main() {
}

```

HEREDOC:

```rust
#!/usr/bin/env cargo
/* Cargo.TOML >>>
[package]
edition = "2018"
<<<
*/

fn main() {
}

```

- Unfamiliar syntax
- New style of structured comment for the ecosystem to support with potential compatibility issues
- Assuming it can't be parsed with `syn` and either we need to write a sufficiently compatible comment parser or pull in a much larger rust parser to extract and update comments.

## Lockfile

[Lockfiles](https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html) record the exact version used for every possible dependency to ensure reproducibility. In particular, this protects against upgrading to broken versions and allows continued use of a yanked version. As this time, the recommendation is for [`bin`s to persist their lockfile while `lib`s do not](https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries).

With multi-file packages, `cargo` writes a `Cargo.lock` file to the package directory. As there is no package directory for single-file packages, we need to decide how to handle locking dependencies.

Considerations

- Sharing of single-file projects should be easy
  - In "copy/paste" scenarios, like reproduction cases in issues, how often have lockfiles been pertinent for reproduction?

- There is an expectation of a reproducible Rust experience
- Dropping of additional files might be frustrating for users to deal with (in addition to making it harder to share it all)
- We would need a way to store the lockfile for `stdin` without conflicting with parallel runs
- `cargo` already makes persisting of `Cargo.lock` optional for multi-file packages, encouraging not persisting it in some cases
- Newer users should feel comfortable reading and writing single-file packages
- A future possibility is allowing single-file packages to belong to a workspace at which point they would use the workspace's `Cargo.lock` file. This limits the scope of the conversation and allows an alternative to whatever is decided here.
- Read-only single-file packages (e.g. running `/usr/bin/package.rs` without root privileges)

**Location 1: In `CARGO_TARGET_DIR`**

The path would include a hash of the manifest to avoid conflicts.

- Transient location, lost with a `cargo clean --manifest-path foo.rs`
- Hard to find for sharing on issues, if needed

**Location 2: In `$CARGO_HOME`**

The path would include a hash of the manifest to avoid conflicts.

- Transient location though not lost with `cargo clean --manifest-path foo.rs`
- No garbage collection to help with temporary source files, especially `stdin`

**Location 3: As `<file-stem>.lock`**

Next to `<file-stem>.rs`, we drop a `<file-stem>.lock` file. We could add a `_` or `.` prefix to distinguish this from the regular files in the directory.

- Users can discover this file location
- Users can persist this file to the degree of their choosing
- Users might not appreciate file "droppings" for transient cases
- When sharing, this is a separate file to copy though its unclear how often that would be needed
- A policy is needed when the location is read-only
  - Fallback to a user-writeable location
  - Always re-calculate the lockfile
  - Error

**Location 4: Embedded in the source**

Embed in the single-file package the same way we do the manifest. Resolving would insert/edit the lockfile entry. Editing the file should be fine, in terms of rebuilds, because this would only happen in response to an edit.

- Users can discover the location
- Users are forced to persist the lock content if they are persisting the source
- This will likely be intimidating for new users to read
- This will be more awkward to copy/paste and browse in bug reports as just a `serde_json` lockfile is 89 lines long
- This makes it harder to resolve conflicts (users can't just checkout the old file and have it re-resolved)
- A policy is needed when the location is read-only
  - Fallback to a user-writeable location
  - Always re-calculate the lockfile
  - Error

**Location 5: Minimal Versions**

Instead of tracking a distinct lockfile, we can get most of the benefits with [`-Zminimal-versions`](JEP 330: Launch Single-File Source-Code Programs).

- Consistent runs across machines without a lockfile
- More likely to share versions across single-file packages, allowing more reuse within the shared build cache
- Deviates from how resolution typically happens, surprising people
- Not all transitive dependencies have valid version requirements

**Configuration 1: Hardcoded**

Unless as a fallback due to a read-only location, the user has no control over the lockfile location.

**Configuration 2: Command-line flag**

`cargo generate-lockfile --manifest-path <file>.rs` would be special-cased to write the lockfile to the persistent location and otherwise we fallback to a no-visible-lockfile solution.

- Passing flags in a `#!` doesn't work cross-platform

**Configuration 3: A new manifest field**

We could add a `workspace.lock` field to control some lockfile location behavior, what that is depends on the location on what policies we feel comfortable making. This means we would allow limited access to the `[workspace]` table (currently the whole table is verboten).

- Requires manifest design work that is likely specialized to just this feature

**Configuration 4: Exitence Check**

`cargo` can check if the lockfile exists in the agreed-to location and use it / update it and otherwise we fallback to a no-visible-lockfile solution. To initially opt-in, a user could place an empty lockfile in that location

## `edition`

[The `edition` field controls what variant of cargo and the Rust language to use to interpret everything.](https://doc.rust-lang.org/edition-guide/introduction.html)

A policy on this needs to balance

- Matching the expectation of a reproducible Rust experience
- Users wanting the latest experience, in general
- Boilerplate runs counter to experimentation and prototyping
- There might not be a backing file if we read from `stdin`

**Option 1: Fixed Default**

Multi-file packages default the edition to `2015`, effectively requiring every project to override it for a modern rust experience. People are likely to get this by running `cargo new` and could easily forget it otherwise.

````rust
#!/usr/bin/env cargo

//! ```cargo
//! [package]
//! edition = "2018"
//! ```

fn main() {
}

````

**Option 2: Latest as Default**

Default to the `edition` for the current `cargo` version, assuming single-file packages will be transient in nature and users will want the current `edition`.

Longer-lived single-file packages are likely to be used with

- other cargo commands, like `cargo test`, so warning when `edition` is defaulted can raise awareness
- workspaces (future possibility), where single-file packages will implicitly inherit `workspace.edition`

```rust
#!/usr/bin/env cargo

fn main() {
}

```

**Option 3: No default**

It is invalid for an embedded manifest to be missing `edition`, erroring when it is missing.

The minimal single-package file would end up being:

````rust
#!/usr/bin/env cargo

//! ```cargo
//! [package]
//! edition = "2018"
//! ```

fn main() {
}

````

This dramatically increases the amount of boilerplate to get a single-file package going.

**Option 4: Auto-insert latest**

When the edition is unspecified, we edit the source to contain the latest edition.

```rust
#!/usr/bin/env cargo

fn main() {
}

```

is automatically converted to

````rust
#!/usr/bin/env cargo

//! ```cargo
//! [package]
//! edition = "2018"
//! ```

fn main() {
}

````

This won't work for the `stdin` case.

**Option 5: `cargo --edition <YEAR>`**

Users can do:

```rust
#!/usr/bin/env -S cargo --edition 2018

fn main() {
}

```

The problem is this does not work on all platforms that support `#!`

**Option 6: `cargo-<edition>` variants**

Instead of an extra flag, we embed it in the binary name like:

```rust
#!/usr/bin/env -S cargo-2018

fn main() {
}

```

single-file packages will fail if used by `cargo-<edition>` and `package.edition` are both specified.

On unix-like systems, these could be links to `cargo` can parse `argv[0]` to extract the `edition`.

However, on Windows the best we can do is a proxy to redirect to `cargo`.

Over the next 40 years, we'll have dozen editions which will bloat the directory, both in terms of the number of files (which can slow things down) and in terms of file size on Windows.

## Scope

The `cargo-script` family of tools has a single command

- Run `.rs` files with embedded manifests
- Evaluate command-line arguments (`--expr`, `--loop`)

This behavior (minus embedded manifests) mirrors what you might expect from a scripting environment, minus a REPL. We could design this with the future possibility of a REPL.

However

- The needs of `.rs` files and REPL / CLI args are different, e.g. where they get their dependency definitions
- A REPL is a lot larger of a problem, needing to pull in a lot of interactive behavior that is unrelated to `.rs` files
- A REPL for Rust is a lot more nebulous of a future possibility, making it pre-mature to design for it in mind

Therefore, this RFC proposes we limit the scope of the new command to `cargo run` for single-file rust packages.

## Naming

Considerations:

- The name should tie it back to `cargo` to convey that relationship
- The command that is run in a `#!` line should not require arguments (e.g. not `#!/usr/bin/env cargo <something>`) because it will fail. `env` treats the rest of the line as the bin name, spaces included. You need to use `env -S` but that wasn't supported on macOS at least, last I tested.
- Either don't have a name that looks like a cargo-plugin (e.g. not `cargo-<something>`) to avoid confusion or make it work (by default, `cargo something` translates to `cargo-something something` which would be ambiguous of whether `something` is a script or subcommand)

Candidates

- `cargo-script`:
  - Out of scope
  - Verb preferred

- `cargo-shell`:
  - Out of scope
  - Verb preferred

- `cargo-run`:
  - This would be shorthand for `cargo run --manifest-path <script>.rs`
  - Might be confusing to have slightly different CLI between `cargo-run` and `cargo run`
  - Could add a positional argument to `cargo run` but those are generally avoided in cargo commands

- `cargo-eval`:
  - Currently selected proposal
  - Might convey REPL behavior
  - How do we describe the difference between this and `cargo-run`?

- `cargo-exec`
  - How do we describe the difference between this and `cargo-run`?

- `cargo`:
  - Mirror Haskell's `cabal`
  - Could run into confusion with subcommands but only
    - the script is in the `PATH`
    - the script doesn't have a file extension
    - You are trying to run it as `cargo <script>` (at least on my machine, `#!` invocations canonicalize the file name)

  - Might affect the quality of error messages for invalid subcommands unless we just assume
  - Restricts access to more complex compiler settings unless a user switches over to `cargo run` which might have different defaults (e.g. setting `RUST_BACKTRACE=1`)
  - Forces us to have all commands treat these files equally (e.g. `--<edition>` solution would need to be supported everywhere).
  - Avoids the risk of overloading a `cargo-script`-like command to do everything special for single-file packages, whether its running them, expanding them into multi-file packages, etc.

## First vs Third Party

As mentioned, a reason for being first-party is to standardize the convention for this which also allows greater interop.

A default implementation ensures people will use it. For example, `clap` received an issue with a reproduction case using a `cargo-play` script that went unused because it just wasn't worth installing yet another, unknown tool.

This also improves the overall experience as you do not need the third-party command to replicate support for every potential feature including:

- `cargo test` and other built-in cargo commands
- `cargo expand` and other third-party cargo commands
- `rust-analyzer` and other editor/IDE integration

While other third-party cargo commands might not immediately adopt single-file packages, first-party support for them will help encourage their adoption.

This still leaves room for third-party implementations, either differentiating themselves or experimenting with

- Alternative caching mechanisms for lower overhead
- Support for implicit `main`, like doc-comment examples
- Template support for implicit `main` for customizing `use`, `extern`, `#[feature]`, etc
- Short-hand dependency syntax (e.g. `//# serde_json = "*"`)
- Prioritizing other workflows, like runtime performance

## File association on Windows

We would add a non-default association to run the file. We don't want it to be a default, by default, to avoid unintended harm and due to the likelihood someone is going to want to edit these files.

## File extension

Should these files use `.rs` or a custom file extension?

Reasons for a unique file type

- Semantics are different than a normal `.rs` file
  - Except already a normal `.rs` file has context-dependent semantics (rest of project, `Cargo.toml`, etc), so this doesn't seem too far off

- Different file associations for Windows
- Better detection by tools for the new semantics (particularly `rust-analyzer`)

Downsides to a custom extension

- Limited support by different tools (rust-analyzer, syntax highlighting, non-LSP editor actions) as adoptin rolls out

At this time, we do not see enough reason to use a custom extension when facing the downsides to a slow roll out.

While `rust-analyzer` needs to be able to distinguish regular `.rs` files from single-file packages to look up the relevant manifest to perform operations, we propose that be through checking the `#!` line (e.g. [how perl detects perl in the `#!`](https://stackoverflow.com/questions/38059830/how-does-perl-avoid-shebang-loops). While this adds boilerplate for Windows developers, this helps encourage cross-platform development.

If we adopted a unique file extensions, some options include:

- `.crs` (used by `cargo-script`)
- `.ers` (used by `rust-script`)
  - No connection back to cargo

- `.rss`
  - No connection back to cargo
  - Confused with RSS

- `.rsscript`
  - No connection back to cargo
  - Unwieldy

- `.rspkg`
  - No connection back to cargo but conveys its a single-file package

# Prior art

Rust, same space

- [`cargo-script`](https://github.com/DanielKeep/cargo-script)
  - Single-file (`.crs` extension) rust code
    - Partial manifests in a `cargo` doc comment code fence or dependencies in a comment directive
    - `run-cargo-script` for she-bangs and setting up file associations on Windows

  - Performance: Shares a `CARGO_TARGET_DIR`, reusing dependency builds
  - `--expr <expr>` for expressions as args (wraps in a block and prints blocks value as `{:?}` )
    - `--dep` flags since directives don't work as easily

  - `--loop <expr>` for a closure to run on each line
  - `--test`, etc flags to make up for cargo not understanding thesefiles
  - `--force` to rebuild`and`--clear-cache`
  - Communicates through scrpts through some env variables

- [`cargo-scripter`](https://crates.io/crates/cargo-scripter)
  - See above with 8 more commits

- [`cargo-eval`](https://crates.io/crates/cargo-eval)
  - See above with a couple more commits

- [`rust-script`](https://crates.io/crates/rust-script)
  - See above
  - Changed extension to `.ers` / `.rs`
  - Single binary without subcommands in primary case for ease of running
  - Implicit main support, including `async main` (different implementation than rustdoc)
  - `--toolchain-version` flag

- [`cargo-play`](https://crates.io/crates/cargo-play)
  - Allows multiple-file scripts, first specified is the `main`
  - Dependency syntax `//# serde_json = "*"`
  - Otherwise, seems like it has a subset of `cargo-script`s functionality

- [`cargo-wop`](https://crates.io/crates/cargo-wop)
  - `cargo wop` is to single-file rust scripts as `cargo` is to multi-file rust projects
  - Dependency syntax is a doc comment code fence

Rust, related space

- [Playground](https://play.rust-lang.org/)
  - Includes top 100 crates

- [Rust Explorer](https://users.rust-lang.org/t/rust-playground-with-the-top-10k-crates/75746)
  - Uses a comment syntax for specifying dependencies

- [`runner`](https://github.com/stevedonovan/runner/)
  - Global `Cargo.toml` with dependencies added via `runner --add <dep>` and various commands / args to interact with the shared crate
  - Global, editable prelude / template
  - `-e <expr>` support
  - `-i <expr>` support for consuming and printing iterator values
  - `-n <expr>` runs per line

- [`evcxr`](https://github.com/google/evcxr)
  - Umbrella project which includes a REPL and Jupyter kernel
  - Requires opting in to not ending on panics
  - Expressions starting with `:` are repl commands
  - Limitations on using references

- [`irust`](https://github.com/sigmaSd/IRust)
  - Rust repl
  - Expressions starting with `:` are repl commands
  - Global, user-editable prelude crate

- [papyrust](https://crates.io/crates/papyrust)
  - Not single file; just gives fast caching for a cargo package

D

- [rdmd](https://dlang.org/rdmd.html)
  - More like `rustc`, doesn't support package-manager dependencies?
  - `--eval=<code>` flag
  - `--loop=<code>` flag
  - `--force` to rebuild
  - `--main` for adding an empty `main`, e.g. when running a file with tests

- [dub](https://dub.pm/advanced_usage)
  - `dub hello.d` is shorthand for `dub run --single hello.d`
  - Regular nested block comment (not doc-comment) at top of file with `dub.sdl:` header

Java

- [JEP 330: Launch Single-File Source-Code Programs](https://openjdk.org/jeps/330)
- [jbang](https://www.jbang.dev/)
  - `jbang init` w/ templates
  - `jbang edit` support, setting up a recommended editor w/ environment
  - Discourages `#!` and instead encourages looking like shell code with `///usr/bin/env jbang "$0" "$@" ; exit $?`
  - Dependencies and compiler flags controlled via comment-directives, including
    - `//DEPS info.picocli:picocli:4.5.0` (gradle-style locators)
      - Can declare one dependency as the source of versions for other dependencies (bom-pom)

    - `//COMPILE_OPTIONS <flags>`
    - `//NATIVE_OPTIONS <flags>`
    - `//RUNTIME_OPTIONS <flags>`

  - Can run code blocks from markdown
  - `--code` flag to execute code on the command-line
  - Accepts scripts from `stdin`

Kotlin

- [kscript](https://github.com/holgerbrandl/kscript) (subset is now supported in Kotlin)
  - Uses an annotation/attribute-like syntqx

.NET

- [dotnet-script](https://github.com/dotnet-script/dotnet-script)
  - [`#` repl directives](https://github.com/dotnet-script/dotnet-script#repl-commands) can appear on lines following `#!`

Haskell

- [`runghc` / `runhaskell`](https://downloads.haskell.org/ghc/latest/docs/users_guide/runghc.html)
  - Users can use the file stem (ie leave off the extension) when passing it in

- [cabal's single-file haskel script](https://cabal.readthedocs.io/en/stable/getting-started.html#run-a-single-file-haskell-script)
  - Command is just `cabal`, which could run into weird situations if a file has the same name as a subcommand
  - Manifest is put in a multi-line comment that starts with `cabal:`
  - Scripts are run with `--quiet`, regardless of which invocation is used
  - Documented in their "Getting Started" and then documented further under `cabal run`.

- [`stack script`](https://www.wespiser.com/posts/2020-02-02-Command-Line-Haskell.html)
  - `stack` acts as a shortcut for use in `#!`
  - Delegates resolver information but can be extended on the command-line
  - Command-line flags may be specified in a multi-line comment starting with `stack script`

Bash

- `bash` to get an interactive way of entering code
- `bash file` will run the code in `file,` searching in `PATH` if it isn't available locally
- `./file` with `#!/usr/bin/env bash` to make standalone executables
- `bash -c <expr>` to try out an idea right now
- Common configuration with rc files, `--rcfile <path>`

Python

- `python` to get an interactive way of entering code
- `python -i ...` to make other ways or running interactive
- `python <file>` will run the file
- `./file` with `#!/usr/bin/env python` to make standalone executables
- `python -c <expr>` to try out an idea right now
- Can run any file in a project (they can have their own "main") to do whitebox exploratory programming and not just blackblox

Go

- [`gorun`](https://github.com/erning/gorun/) attempts to bring that experience to a compiled language, go in this case
  - `gorun <file>` to build and run a file
  - Implicit garbage collection for build cache
  - Project metadata is specified in HEREDOCs in regular code comments

Perl

- [Re-interprets the `#!`](https://stackoverflow.com/questions/38059830/how-does-perl-avoid-shebang-loops)

Ruby

- [`bundler/inline`](https://bundler.io/guides/bundler_in_a_single_file_ruby_script.html)
  - Uses a code-block to define dependencies, making them available for use

Cross-language

- [`scriptisto`](https://github.com/igor-petruk/scriptisto)
  - Supports any compiled language
  - Comment-directives give build commands

- [nix-script](https://github.com/BrianHicks/nix-script)
  - Nix version of scriptisto, letting you use any Nix dependency

See also [Single-file scripts that download their dependencies](https://dbohdan.com/scripts-with-dependencies)

# Unresolved questions

- Can we have both script stability and make it easy to be on the latest edition?
- Could somehow "lock" to what is currently in the shared script cache to avoid each script getting the latest version of a crate, causing churn in `target/`?
- Since single-file packages cannot be inferred and require an explicit `--manifest-path`, is there an alternative shorthand we should provide, like a short-flag for `--manifest-path`?
  - `p` is taken by `--package`
  - `-m`, `-M`, and `-P` are available, but are the meanings clear enough?

- Is there a way we could track what dependency versions have been built in the `CARGO_TARGET_DIR` and give preference to resolve to them, if possible.
- `.cargo/config.toml` and rustup-toolchain behavior
  - These are "environment" config files
  - Should `cargo` run like `cargo run` and use the current environment or like `cargo install` and use a consistent environment from run-to-run of the target?
  - It would be relatively easy to get this with `.cargo/config.toml` but doing so for rustup would require a new proxy that understands `cargo <file.rs>` CLI.
  - This would also reduce unnecessary rebuilds when running a personal script (from `PATH`) in a project that has an unrelated `.cargo/config.toml`

# Future possibilities

## Executing `<stdin>`

We could extend this to allow accepting single-file packages from stdin, either explicitly with `-` or implicitly when `<stdin>` is not interactive.

## Implicit `main` support

Like with doc-comment examples, we could support an implicit `main`.

Ideally, this would be supported at the language level

- Ensure a unified experience across the playground, `rustdoc`, and `cargo`
- `cargo` can directly run files rather than writing to intermediate files
  - This gets brittle with top-level statements like `extern` (more historical) or bin-level attributes

Behavior can be controlled through editions

## A REPL

See the [REPL exploration](https://github.com/epage/cargo-script-mvs/discussions/102)

In terms of the CLI side of this, we could name this `cargo shell` where it drops you into an interactive shell within your current package, loading the existing dependencies (including dev). This would then be a natural fit to also have a `--eval <expr>` flag.

Ideally, this repl would also allow the equivalent of `python -i <file>`, not to run existing code but to make a specific file's API items available for use to do interactive whitebox testing of private code within a larger project.

## Workspace Support

Allow scripts to be members of a workspace.

The assumption is that this will be opt-in, rather than implicit, so you can easily drop one of these scripts anywhere without it failing because the workspace root and the script don't agree on workspace membership. To do this, we'd expand `package.workspace` to also be a `bool` to control whether a workspace lookup is disallowed or whether to auto-detect the workspace

- For `Cargo.toml`, `package.workspace = true` is the default
- For cargo-script, `package.workspace = false` is the default

When a workspace is specified

- Use its target directory
- Use its lock file
- Be treated as any other workspace member for `cargo <cmd> --workspace`
- Check what `workspace.package` fields exist and automatically apply them over default manifest fields
- Explicitly require `workspace.dependencies` to be inherited
  - I would be tempted to auto-inherit them but then `cargo rm`s gc will remove them because there is no way to know they are in use

- Apply all `profile` and `patch` settings

This could serve as an alternative to [`cargo xtask`](https://github.com/matklad/cargo-xtask) with scripts sharing the lockfile and `target/` directory.

## Scaling up

We provide a workflow for turning a single-file package into a multi-file package, on `cargo-new` / `cargo-init`. This would help smooth out the transition when their program has outgrown being in a single-file.

---

<div class="post-metadata">

**Author:** ![Aloso](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/aloso/32/5039_2.png) [@Aloso](https://internals.rust-lang.org/u/Aloso)\
**Post date:** [April 7, 2023, 9:42pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/2 "2023-04-07T21:42:25Z")

</div>

Seems interesting.

Notes:

- `time = "0.1.12"` and `regex = "0.1.41"` are both very outdated, I wouldn't mention versions this old in a tutorial.

- 

> You can then re-run this and Cargo will fetch the new dependencies and all of their dependencies. You can see this by passing in `--verbose`:
> 
> ```rust
> $ cargo-eval --verbose ./hello_world.rs
> 
> ```

It should be possible to call the program as a subcommand (`cargo eval`, without the dash), like `cargo clippy` and `cargo fmt`. I don't like that `--verbose` is needed for seeing cargo output. It would be best if `-q` could just be added to the first line:

```
#!/usr/bin/env cargo-eval -q

```

But as you mentioned, this doesn't work. Another option is to implicitly add the `-q` flag, but only when the file is executed directly (as `./hello_world.rs` rather than `cargo eval hello_world.rs`).

- 

> Is there a way we could track what dependency versions have been built in the `CARGO_TARGET_DIR` and give preference to resolve to them, if possible.

Yes, and it is called `Cargo.lock`. But when there are multiple scripts in the same directory, creating a `Cargo.lock` for one script may overwrite a lockfile for another script, so the lockfiles need to have different names. For example, running `cargo eval hello_world.rs` could create `hello_world.lock`.

I believe your intention was to create the lockfile _in the target directory_. I don't think that's a good idea, because lockfiles should be committed into version control; otherwise they aren't very useful. The alternative is to tell everyone to use absolute version requirements such as `regex = "=1.7.0"`, so no lockfile is needed. Or we accept that Rust scripts may occasionally break if dependencies don't strictly adhere to semantic versioning. That might be an acceptable tradeoff for simple scripts.

- There's a mistake in the reference-level documentation: If the manifest has to be in an inner documentation comment, that's `//! ...` or `/*! ... */` (note the missing exclamation mark)

- You refer to the program as `cargo-shell` instead of `cargo-eval` once in "Unresolved questions"

- How about adding `cargo init --script` (or `cargo eval --init`) to quickly create a Rust script with all the boilerplate?

---

<div class="post-metadata">

**Author:** ![epage](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/epage/32/3171_2.png) [@epage](https://internals.rust-lang.org/u/epage)\
**Post date:** [April 7, 2023, 9:57pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/3 "2023-04-07T21:57:37Z")

</div>

> [@Aloso](#):
>
> `time = "0.1.12"` and `regex = "0.1.41"` are both very outdated, I wouldn't mention versions this old in a tutorial.

I was intentionally mirroring what is in the cargo book today.

> [@Aloso](#):
>
> For example, running `cargo eval hello_world.rs` could create `hello_world.lock`.

This is for locking within a script. The comment you quoted is about trying to do best-effort lock between scripts.

As a user, I'm not happy when running tools leave random breadcrumbs in my regular files. The current demo does create a `Cargo.lock` per script but its in the shared `target/` directory. I forgot to specify how this would work within the RFC

As for committing the lock files,

- Embedded manifests are already a little sloppy in that the default `package.edition` is the latest
- I felt this compromise was acceptable because these are for more throwaway uses
- If people want this for more long term uses, the "Workspace" future possibility provides locking and fixes the edition to `workspace.edition` by default.

> [@Aloso](#):
>
> There's a mistake in the reference-level documentation: If the manifest has to be in an inner documentation comment, that's `//! ...` or `/*! ... */` (note the missing exclamation mark)

Thanks, you can tell how much I use them. I likely would have forgotten to even include them if it weren't for building off of `rust-script`

> [@Aloso](#):
>
> You refer to the program as `cargo-shell` instead of `cargo-eval` once in "Unresolved questions"

Thanks, fixed. I originally had a wider scope but decided that it really should be separate programs.

> [@Aloso](#):
>
> How about adding `cargo init --script` (or `cargo eval --init`) to quickly create a Rust script with all the boilerplate?

This is what I'm referring to in the "Scaling up" section under "Future Possibilities". I had originally intended to include it but I realized there was enough UX to decide on that I wanted to push it off in hopes that a narrowed discussion would make this easier to progress.

---

<div class="post-metadata">

**Author:** ![epage](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/epage/32/3171_2.png) [@epage](https://internals.rust-lang.org/u/epage)\
**Post date:** [April 7, 2023, 10:01pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/4 "2023-04-07T22:01:00Z")

</div>

> [@Aloso](#):
>
> But as you mentioned, this doesn't work. Another option is to implicitly add the `-q` flag, but only when the file is executed directly (as `./hello_world.rs` rather than `cargo eval hello_world.rs`).

Is there a way to detect when being run with `#!`?

In thinking over this, it actually got me really excited about the idea of refactoring cargo so you `cargo-eval` is able to show a progress bar during compilation that then gets out of the way when done.

---

<div class="post-metadata">

**Author:** ![Aloso](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/aloso/32/5039_2.png) [@Aloso](https://internals.rust-lang.org/u/Aloso)\
**Post date:** [April 7, 2023, 10:11pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/5 "2023-04-07T22:11:49Z")

</div>

> Is there a way to detect when being run with `#!`?

Upon checking, I don't think there is. I was hoping that it can be observed by looking at the first argument, but I was wrong.

P.S. But I do like the idea of the progress bar!

---

<div class="post-metadata">

**Author:** ![matklad](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/matklad/32/12266_2.png) [@matklad](https://internals.rust-lang.org/u/matklad)\
**Post date:** [April 7, 2023, 10:55pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/6 "2023-04-07T22:55:05Z")

</div>

This is potentially quite big. The biggest problem this solves is bootstrapping. It's very easy to write and read Python, but it's very hard to actually run it. If we had something like this, then Rust would be a very compelling tool to use for scripts that always just run on whichever machine. So to me it seems like this goes quite a bit beyond exploratory programming, and adds a very significant public API surface, which we should carefully design.

Some questions:

Where do we put the lockfile? For scripts, its invaluable to guarantee that they are frozen in time. Can we `cat` the lockfile as a comment at the end of the file itself? Actually, can we optionally move the manifest to the end as well, so that it doesn't get in a way?

What's the startup time here? For scripts, it's important to start fast. If I run such a script for the second time, what's the minimum amount of work that Cargo needs to do? Do I get a guarantee that it won't, eg, try to ping [crates.io](http://crates.io) index?

The `#!` works for UNIX. Windows is a first-class platform for us, so we need to make sure that `./my-script.rs` works there just as well. I think we can setup a blanket file association for all `.rs` files? Or would it break double-clicking on the file to open it in an IDE?

---

<div class="post-metadata">

**Author:** ![chrisd](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/chrisd/32/7232_2.png) [@chrisd](https://internals.rust-lang.org/u/chrisd)\
**Post date:** [April 7, 2023, 11:37pm UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/7 "2023-04-07T23:37:22Z")

</div>

> [@matklad](#):
>
> I think we can setup a blanket file association for all `.rs` files? Or would it break double-clicking on the file to open it in an IDE?

IMHO, it should really be a separate file type for an executable script. Maybe `.rss` (aka, rust script file)?

Also, semantically I think it is a different file type if it includes metadata that's only understood by this feature.

---

<div class="post-metadata">

**Author:** ![epage](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/epage/32/3171_2.png) [@epage](https://internals.rust-lang.org/u/epage)\
**Post date:** [April 8, 2023, 12:11am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/8 "2023-04-08T00:11:49Z")

</div>

> [@matklad](#):
>
> Where do we put the lockfile? For scripts, its invaluable to guarantee that they are frozen in time. Can we `cat` the lockfile as a comment at the end of the file itself? Actually, can we optionally move the manifest to the end as well, so that it doesn't get in a way?

I realized I left out lockfiles and created [an issue](https://github.com/epage/cargo-script-mvs/issues/145) to track getting them referenced in the RFC.

The summary is I've been punting on reproducible runs of scripts (lock file, edition) and instead see the workspace support (future possibility) as the main avenue for reproducibility. I am open though to alternative approaches.

One person mentioned dropping lockfiles nearby but I expect users won't be happy with us polluting the file system and having two files to delete now.

As for embedding lockfiles, at first I was concerned about changing the mtime but if we have to make an edit to the block, then we already will have to rebuild it anyways. We just need to make sure we don't accidentally cause extra rebuilds.

I would also like to find a way to nudge the resolver to reuse existing intermediate build artifacts from other scripts.

The main challenge I see to embedding the lockfile is deciding how to do so syntactically. Similar if we moved the manifest to below.

> [@matklad](#):
>
> What's the startup time here? For scripts, it's important to start fast. If I run such a script for the second time, what's the minimum amount of work that Cargo needs to do? Do I get a guarantee that it won't, eg, try to ping [crates.io](http://crates.io) index?

I just ran the demo implementation on code using `serde` w/ `derive` and `serde_json`. A second run (populated cache) took 156ms.

My personal stance on performance (personal, organizational, code) is to implement the ideal case to identify the big problems and then identify how things can improve rather than blocking on performance and only aiming for smaller, incremental improvements that might not make enough. I see this as a positive opportunity to re-evaluate cargo's no-op build times to see where we can improve them.

For people who want faster performance, I could see this also being an opportunity for alternative implementations. `rust-script` is a superset of `cargo-eval` that doesn't even talk to cargo if the source is unchanged but this requires a `--clear-cache` and `--force` flags to deal with corner cases. By us standardizing on the format, we can make sure we can interoperate with alternative implementations that take shortcuts cargo is likely to never take.

> [@matklad](#):
>
> The `#!` works for UNIX. Windows is a first-class platform for us, so we need to make sure that `./my-script.rs` works there just as well. I think we can setup a blanket file association for all `.rs` files? Or would it break double-clicking on the file to open it in an IDE?

I think that even for these scripts we should probably prefer the IDE as the default association but I assume we can register multiple associations so users can still run it through a right click menu.

I've added some research tasks

- [Investigate Python Launcher for Windows](https://github.com/epage/cargo-script-mvs/issues/146)
- [Investigate PATHEXT](https://github.com/epage/cargo-script-mvs/issues/147)

btw file associations on windows is referenced in the Unresolved Questions section.

---

<div class="post-metadata">

**Author:** ![chrisd](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/chrisd/32/7232_2.png) [@chrisd](https://internals.rust-lang.org/u/chrisd)\
**Post date:** [April 8, 2023, 12:22am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/9 "2023-04-08T00:22:20Z")

</div>

Actually, thinking about it, "default executable" might not be that great. Just from an "oops" perspective.

---

<div class="post-metadata">

**Author:** ![epage](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/epage/32/3171_2.png) [@epage](https://internals.rust-lang.org/u/epage)\
**Post date:** [April 8, 2023, 12:39am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/10 "2023-04-08T00:39:27Z")

</div>

> [@matklad](#):
>
> This is potentially quite big. The biggest problem this solves is bootstrapping. It's very easy to write and read Python, but it's very hard to actually run it. If we had something like this, then Rust would be a very compelling tool to use for scripts that always just run on whichever machine.

I real-world use case to back up your comment: Heroku buildpacks are being ported from bash to Rust and there is interest in this exact workflow

See

- [Add inventory automation by joshwlewis · Pull Request #74 · heroku/buildpacks-go · GitHub](https://github.com/heroku/buildpacks-go/pull/74)
- [Richard Schneeman: "@epage@hachyderm.io I heard your rustacean statio…" - Ruby.social](https://hachyderm.io/@Schneems@ruby.social/110097424353566765)

---

<div class="post-metadata">

**Author:** ![8573](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/8573/32/4913_2.png) [@8573](https://internals.rust-lang.org/u/8573)\
**Post date:** [April 8, 2023, 1:03am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/11 "2023-04-08T01:03:23Z")

</div>

> [@epage](#):
>
> The manifest would be a module-level doc comment with a code fence with `cargo` as the type.

Especially if the lock file also gets included as another comment, could it use the normal file names as the code types: `Cargo.toml` and `Cargo.lock`? This would make it very clear what the comments are, more so than using just "cargo" as the type.

---

<div class="post-metadata">

**Author:** ![kpreid](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kpreid/32/8484_2.png) [@kpreid](https://internals.rust-lang.org/u/kpreid)\
**Post date:** [April 8, 2023, 1:11am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/12 "2023-04-08T01:11:01Z")

</div>

> [@epage](#):
>
> ````rust
> #!/usr/bin/env cargo-eval
> 
> //! ```cargo
> //! [dependencies]
> //! clap = { version = "4.2", features = ["derive"] }
> //! ```
> 
> ````

I'd like to propose a slightly different syntax. Instead of embedding TOML inside a code block inside markdown, use a new attribute:

```rust
#!/usr/bin/env cargo-eval

#![cargo(manifest = r#"
    [dependencies]
    clap = { version = "4.2", features = ["derive"] }
"#)]

```

Advantages:

- Parser does not need to know about Markdown or about `#[doc]` implicit concatenation rules; only Rust and TOML
- Separated from documentation; does not appear in it.
  - Can even be automatically edited without disturbing documentation at all (e.g. for the lock-file discussion above)

- Does not require typing `//!` on each line, particularly if pasting between a regular `Cargo.toml` and this inline style
- Does not get doc-comment syntax highlighting which is often faded and conveys "not part of code, just an example" to the reader
- Obviously an error to implementations that don't support it (unknown attribute) or which cannot support it in the current invocation
  - e.g. what if one of these scripts is pasted into a regular multi-file project? This should probably be at least a warning.

- Extensible using the provisions of attribute syntax rather than the less expressive code block tag namespace.
  - `#[cargo(lock = r#"..."#)]`, anyone?

Disadvantages:

- Less familiar syntax for multi-line non-Rust text
- Is an error, rather than being skipped, if the compiler doesn't support it, so it might prevent code from being compilable in certain circumstances
- Might be harder for tools to add TOML syntax highlighting (since doc comment code blocks already desire it), or easier (if they have trouble with `//!` prefixes interspersed with the inner language text)

---

<div class="post-metadata">

**Author:** ![davidsk](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/davidsk/32/11544_2.png) [@davidsk](https://internals.rust-lang.org/u/davidsk)\
**Post date:** [April 8, 2023, 1:22am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/13 "2023-04-08T01:22:38Z")

</div>

I tried cargo-script a year back as a medium size `bash` script replacement. I didn't end of using it. I loved the idea and I have a need for such a tool as illustrated by the various tiny Rust projects with `target/<some_bin>` linked into my `path`.

There where two reasons why I stopped using cargo-script:

- There was no syntax highlighting due to the file extension not being `.rs`. If the file keeps `.rs` as an extension that would no longer an issue. If we end up picking something else (`.rss` for example) a lot of IDE's/text editors will need (trivial) updates.

- Rust analyzer (RA) speeds up development thanks to semantically correct auto-completion and documentation in the editor/IDE. While rust-script saved me a project dir, a toml and a syslink that was not worth losing RA. There has been some discussion on solving using a proxy language server: [github](https://github.com/fornwall/rust-script/issues/11). It would be way easier if this got supported 'natively' and at the very least it should be possible in the future.

How would RA recognize a rust 'script' file, can it do so without a unique extension (even on windows)?

---

<div class="post-metadata">

**Author:** ![quinedot](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/quinedot/32/7294_2.png) [@quinedot](https://internals.rust-lang.org/u/quinedot)\
**Post date:** [April 8, 2023, 1:24am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/14 "2023-04-08T01:24:24Z")

</div>

It took me awhile to absorb the proposal, and others have made these points better than I. But to voice my support for them...

* * *

I'd rather see more of a distinct and first-class syntax for the embedded manifest (and potentially lockfile and other future possibilities) than a particularly-shaped `doc` attribute. While necessary to compile the following Rust program, it's not part of the following Rust program, and doesn't need to pretend to be. \[1\]

I do understand there are some advantanges to being valid Rust after the `#!` (probably less work for `Cargo`, if not other parties; easier support for editors). But even if the syntax end up being valid Rust, I'd rather see something like a distinct attribute than specially crafted doc comments.

I'd rather see a separate extension than `.rs`. It embeds a Rust source code file, but isn't one. And my Rust source code files aren't executables (or packages).

With all the default-behavior overrides, there should probably be a `.cargo/eval.toml` or the like to tweak `cargo-eval` configuration separately from the whole of `cargo`. If you want them to behave differently, I should be able to configure it independently.

* * *

1. Incidentally your multiline comment examples are not `doc` comments/attributes.

---

<div class="post-metadata">

**Author:** ![steffahn](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/steffahn/32/13288_2.png) [@steffahn](https://internals.rust-lang.org/u/steffahn)\
**Post date:** [April 8, 2023, 1:38am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/15 "2023-04-08T01:38:20Z")

</div>

I don't like the idea of not specifying the edition. I would be happy to disable the default edition being `2018` because (from my personal experience with [godbolt.org](http://godbolt.org), which uses 2018 unless you add a `--edition` command line argument) I'm convinced that will just lead to _lots_ of confusion. So an error when no edition is specified seems desirable.

I believe that tooling can help with making mandatory edition information less of a churn. If the edition is mandatory, since the manifest information is a lot of typing work anyways, I suppose a `cargo new` equivalent to create a script file that includes an empty dependencies list should be useful. Additionally, there could be, either a separate command or integrated into `cargo-eval`, a way of automatically add in the edition information to the manifest in the script.

On a somewhat related note, for me when prototyping `cargo add` is one of the most important tools, so we definitely need a way of achieving the same or even better for scripts: My experience with [rustexplorer.com](http://rustexplorer.com) is that adding a short `foo = "*"` line to the manifest is often a very convenient way of avoiding the need to go to the terminal and touching a tool entirely, however... this approach is naturally very prone to future breakage. If `cargo-eval` can be made to fix my missing edition field in my file, editing the file to use the latest available edition, maybe it could also fix a missing dependency by adding the latest available version. Maybe something like a naked line `foo`, without any subsequent `= …` or maybe some other stand-in syntax that waits to be completed by tooling (perhaps we _do_ want the stand in to still be valid `.toml`).

* * *

Back to editions: If the ability to write small neat files without dependencies in a way that avoids the whole manifest is a goal, another, perhaps complementary, approach could be to offer multiple commands with different edition defaults. So `cargo-eval` could error and offer to fix the file when editions are missing, but something like `cargo-eval-2021` could have a 2021 edition default, and the shebang could thus this way optionally contain the edition information in a terse manner.

A simple hello world

```rust
#!/usr/bin/env cargo-eval-2021

fn main() {
    println!("Hello, world!");
}

```

might then be modified by a `cargo add` invocation, to something like

````rust
#!/usr/bin/env cargo-eval

//! ```cargo
//! [package]
//! edition = "2021"
//!
//! [dependencies]
//! clap = "4.2.1"
//! ```

fn main() {
    println!("Hello, world!");
}

````

or maybe the `2021` would rather be kept as part of the `#!` by default to keep the file slightly shorter, I don't know.

---

<div class="post-metadata">

**Author:** ![Aloso](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/aloso/32/5039_2.png) [@Aloso](https://internals.rust-lang.org/u/Aloso)\
**Post date:** [April 8, 2023, 1:47am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/16 "2023-04-08T01:47:32Z")

</div>

> [@kpreid](#):
>
> Is an error, rather than being skipped, if the compiler doesn't support it, so it might prevent code from being compilable in certain circumstances

That could be easily solved by adding `cargo` to the [tool prelude](https://doc.rust-lang.org/reference/attributes.html#tool-attributes), so `#![cargo::manifest(...)]` would be ignored by rustc. But I would prefer a less "stringy" syntax:

```rust
#![cargo::manifest(dependencies(
    regex = "=1.7.0",
))]

```

This works with rustfmt out of the box, has syntax highlighting, and auto-completion can be easily added to rust-analyzer.

---

<div class="post-metadata">

**Author:** ![kpreid](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kpreid/32/8484_2.png) [@kpreid](https://internals.rust-lang.org/u/kpreid)\
**Post date:** [April 8, 2023, 1:50am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/17 "2023-04-08T01:50:12Z")

</div>

I agree that embedding languages as text in string literals is undesirable if it can be avoided, but in this case, I think there is more value in keeping the current TOML syntax for manifests exactly unchanged. A distinct syntax would be an obstacle to, for example, preparing a repro case from a larger project.

---

<div class="post-metadata">

**Author:** ![Aloso](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/aloso/32/5039_2.png) [@Aloso](https://internals.rust-lang.org/u/Aloso)\
**Post date:** [April 8, 2023, 1:53am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/18 "2023-04-08T01:53:54Z")

</div>

Then there may be value in supporting both: `#![cargo::manifest(rust syntax)]` and `#![cargo::manifest = "toml syntax"]`

Though that would make implementing `cargo add` support more complicated.

---

<div class="post-metadata">

**Author:** ![davidsk](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/davidsk/32/11544_2.png) [@davidsk](https://internals.rust-lang.org/u/davidsk)\
**Post date:** [April 8, 2023, 1:55am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/19 "2023-04-08T01:55:55Z")

</div>

A long attribute might be intimidating to new users. The _comment format_ is easily recognized as an inline `Cargo.toml` even if we leave out the package section.

---

<div class="post-metadata">

**Author:** ![DanielKeep](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/danielkeep/32/1332_2.png) [@DanielKeep](https://internals.rust-lang.org/u/DanielKeep)\
**Post date:** [April 8, 2023, 2:08am UTC](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639/20 "2023-04-08T02:08:40Z")

</div>

I think it's a good idea to include something like this in the standard toolchain. I've nothing to do with this proposal, but here are some thoughts.

> [@epage](#):
>
> When sharing reproduction cases, it is much easier when everything exists in a single code snippet to copy/paste. Alternatively, people will either leave off the manifest or underspecify the details of it.
> 
> This similarly makes it easier to share code samples with coworkers or in books / blogs when teaching.

I think it's very valuable to be able to have completely self-contained chunks of runnable code whose use is just "shove it in a file". The only way running code could be easier would be to do it on the playpen, but as noted, that doesn't run on the user's machine, so it pretty limited.

(Deleted section about `/*! .. */` since someone else beat me to it.)

> [@epage](#):
>
> Inferred / defaulted manifest fields:

You should probably also note that the package _name_ should be inferred (to something derived from the name of the file), and there needs to be a version of some kind (at least, I _think_ you need to specify a version).

Not allowing publishing by default is a great idea! That could be messy...

As for `edition`, I'm not a fan of script behaviour changing implicitly over time. "This script worked, but now it doesn't? I guess Rust isn't very stable..." But defaulting to "old" Rust for users who maybe don't know what's going on would probably be confusing.

Perhaps it would be worth having a visible warning when running a script for which things like the edition are being defaulted in a way that might change over time. Something like:

```plaintext
> ./hello.rs
note: this script doesn't specify an edition of Rust, defaulting to the latest.
This may change in the future, which could break this script.
To specify the edition, add the following to the script's manifest:

    [package]
    edition = "2024"

```

> [@epage](#):
>
> A single-file package is accepted by cargo commands as a `--manifest-path`

This feels a bit messy. Single-file packages will be _real_ packages... except where they aren't. It might be simpler to have either another command or an option/subcommand to take a single-file package and "explode" it into a real package.

> [@epage](#):
>
> The implicit content of the manifest will be unclear for users. We can patch over this as best we can in documentation but the result won't be ideal.

With the exception of `package.name` and `package.publish`, I feel like it would be sufficient to let the user know that some parts of the manifest are implied and tell them what to change to make the warning go away. Rust, in general, has a good habit of being explicit when silence might promote confusion.

> [@epage](#):
>
> The command run in a `#!` line should not require arguments [...]

If I remember correctly, this is partly why `cargo-script` installed a secondary program `run-cargo-script` that was meant to be used as the hashbang target. The leading `run-` meant it could _never_ be mistaken for a cargo plugin.

> [@epage](#):
>
> Candidates

I worry that "eval" feels like a command that should be either a REPL or take code on the command line, not something that runs code from files. Maybe `cargo-exec`?

Also, although you have other reasons for not using `cargo-script`, it hasn't been updated in absolutely ages and it's probably not widely used, so it might be okay to just co-opt the name.

> [@epage](#):
>
> [...] rather than the third-party tool trying to cover every `cargo` workflow in a slightly different way than the regular cargo workflow.

That's definitely a point in favor of treating these files like regular packages, although it still _feels_ a little gross. To that end, you _could_ allow for: `cargo eval --test package.rs` and `cargo eval --doc package.rs`.

> [@epage](#):
>
> Should rustup associate `.rs` files with `cargo-eval`?

(Apologies for wall of text.)

So, the problem here is Windows. From what I recall, you can set it up so that scripts (be they `.py` or `.rs`) behave like any other program on the system (_i.e._ `.exe`, `.cmd`, _etc._). The issue is that the "interpreter" is _not_ what is listed on the hashbang, it's whatever the default "verb" defined in the file association is. So, if the default action is to open the file in an editor, then that's what "running" the script file will do. Likewise, if you set the default verb to execute the code, then that's also what double-clicking on the file in the file explorer will do.

To be less circumspect: naming the file on the command line and double-clicking on the file in the GUI _do the same thing_ on Windows. Insofar as I'm aware, you can't change or avoid that.

The tension here is that it's not surprising that double-clicking on a `.py` file executes it by default, because that's what it always does. But by now there's probably a very strong association with "double-click `.rs` to open in editor", so changing that default is a _really_ bad idea (in my opinion).

The `.crs` extension lets you have that different default, letting you treat Rust scripts as "real" scripts on Windows. It would also make the "this file is meant to be treated as a single-file package" distinction clearer from just regular old Rust code.

The _downside_ is that none of the editors know what a `.crs` file is, so I suspect most `cargo-script` scripts just used `.rs`—a problem that wouldn't exist if this is part of the standard toolchain.

As an aside, there's also some precedent here: Python allows `.pyw` files to distinguish between "scripts run in console mode" and "scripts run in graphics mode", which again is because Windows just _has_ to be different. So, using the file extension to change what the "default verb" is doesn't come completely out of the blue.

> [@epage](#):
>
> Since single-file packages cannot be inferred and require an explicit `--manifest-path`, is there an alternative shorthand we should provide, like a short-flag for `--manifest-path`?

To reiterate, I feel like having those shortcuts as part of `cargo-eval` might be more natural. `cargo eval --test glarbits` ⇒ `cargo test --manifest-path glarbits`

> [@epage](#):
>
> `.cargo/config.toml` and rustup-toolchain behavior

I think you could equally make the case either way. I feel like, as a user, my expectation would be that the single-file package behaves exactly as a "full" package would in that same location. So, if I put a script in `$HOME/bin/do-thing.rs`, but run it from `$HOME/code/some-project`, the script should not be affected by anything in `$HOME/code/some-project` or `$HOME/code`.

> [@Aloso](#):
>
> I don't like that `--verbose` is needed for seeing cargo output.

Ah, forgot to mention this before: I feel like the most user-friendly behaviour would be for `cargo-eval` to eat the build output and not show it to the user _unless_ the build fails for some reason. You might also want to show it if, say, 5 seconds elapses so that the user knows that it _is_ doing something, it's just taking a while.

> [@chrisd](#):
>
> Maybe `.rss` (aka, rust script file)?

Look, I know Google Reader has been dead a long time, but some people still use RSS feeds, y'know. 😛

If this proposal really wants to go down the "single file package" route, maybe `.rspkg` would do? Longer, but at least it should be relatively safe from collision.

> [@kpreid](#):
>
> Does not require typing `//!` on each line, particularly if pasting between a regular `Cargo.toml` and this inline style

I have just the thing for you: `/*! .. */` block comments! 😃

[Next page](https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639.md?page=2)