[Pre-RFC]: cargo new templates (v2)

tools and infrastructure cargo
29

I want to turn attention to how templates are supposed to function for a moment. Basically, what is the underlying engine going to do, and going to be able to do. Is the engine going to be allowed to do arbitrary things? Or is it going to be sandboxed? How will it be sandboxed? Are we going to develop a new domain specific language (DSL) for templates alone? Can we use a language that is already available?

My vote is to use the absolute simplest method I can think of; copy/paste/text substitution. This would involve making the convention that each child of the template directory is a complete template. Thus:

- mycrate
    - templates
        - A 
            - Template.toml
            - src
                - Cargo.toml
                - src
                    - file1.rs
                    - file2.rs
        - B
            - Template.toml
            - src
                - file1.rs

cargo --template would interpret this as there being two templates, A and B. Within a given template is the Template.toml file. This file enumerates all of the keys that are available within that template, along with any help information that is needed. E.g. A/Template.toml might contain:

[package]
name = "A"
version = "0.1.0"
authors = ["Alice B Caring <abc@place.com>"]
edition = "2018"
description = """A small, complete template for the [foo-bar-baz](https://crates.io/crates/foo-bar-baz) crate.

This template will create a small, but complete, project that uses the [foo-bar-baz](https://crates.io/crates/foo-bar-baz) crate.  
"""
keywords = ["foo", "bar", "baz"]
categories = ["template"]
license = "MIT OR Apache-2.0" # It would probably be better to require that templates use the same license as the original crate.

[badges]
maintenance = {status = "experimental"}

[dependencies]
foo-bar-baz = {version = ">= 0.4, <= 0.6", features = ["serde"], uuid = "B23901D91F984C39888E468F531E0F97"}

[keys]
authors = {description = "A list of strings of authors like 'Alice B Caring <abc@place.com>'", default = ""}
date = {description = "The date that the file was generated.  Can be any form you want, it will be directly copied into the template.", default = ""}

Most of those keys are directly copied from the Cargo docs, and so should already be familiar to users of the tool. The only new keys are uuid, which contains a uuid generated by the trick I mentioned above (by the way, uuid would have to become a new key for Cargo.toml files), and the whole keys section, which (I hope) should be relatively obvious.

When using cargo --template, you might have subcommands like:

  • cargo --template list, which would return A and B to you.
  • cargo --template show A would give you a pretty-printed version of A/Template.toml.
  • cargo --template generate --authors "Alice B Caring <abc@place.com>, Donald E Francis <def@place.com>" --date "Thursday, 29 February 2024" A ~/Documents/repositories/my_project/ which actually generates your file(s) from the template.

The generate command would have two required arguments; the first is the name of the template to use (A) in the example above, and the location where you want the template to be copied to (~/Documents/repositories/my_project/) above. The complete contents of a template's src directory would be copied directly to that path; thus, when using template A I would expect to see a directory structure like:

- ~/Documents/repositories/my_project
    - Cargo.toml
        - src
            - file1.rs
            - file2.rs

but if I used template B, I'd see something like this:

- ~/Documents/repositories/my_project
    - file1.rs

This actually solves the 'what is a template?' question; to use template B correctly, I would need to use cargo --template generate --authors "Alice B Caring <abc@place.com>, Donald E Francis <def@place.com>" --date "Thursday, 29 February 2024" A ~/Documents/repositories/my_project/src/, which would place file1.rs where it belongs.

The engine would be a simple text substitution engine. It would search for text strings of the form {{key}} and replace them with either what is on the command line, or the specified default in the Template.toml file. If a key doesn't have a default and is not specified on the command line, then the engine would issue an error about the missing key (maybe with the key's description copied from the Template.toml file), and require that the template user fix the errors before continuing.

This solves the major security problems because the template isn't being executed, and you can't really do a DOS attack on text substitution (notice how the method above doesn't permit re-evaluation of the generated text, which means you don't get a billion-laughs style attack).

This leaves two issues; how do template authors indicate that they really want the string {{ or }}, and how do you reuse a template for the same destination path multiple times in a row?

  • How do template authors indicate that they really want to have {{ or }} in the source text? A method that could work is to pre-define the keys {{left double curly brackets}} and {{right double curly brackets}}, which template authors could then put into their templates as needed. The engine wouldn't need any modification as it would substitute the appropriate strings in directly whenever it encountered those keys. The only tricky part would be binary blobs (e.g., audio or image files that were part of the template). My suggestion is to have a cargo --template from <source path> <dest path> command. It does the reverse of the template engine, copying everything from <source path> to <dest path> almost verbatim; the difference obviously being that whenever the engine encountered {{ it would substitute {{left double curly brackets}} in, and similarly for }}.
  • What happens when a user has the same destination path multiple times in a row? The trivial answer is that the engine simply and blindly overwrites the contents of the destination path. This would be a horrible user experience, so let's not do that. Instead, cargo --template generate could have additional switches, maybe -f, --force and -i, --interactive, which would be mutually exclusive. If the user selected -f, then the destination path would be overwritten. If -i were selected, then the engine would ask what to do for each destination file that was going to be overwritten (probably overwrite, don't overwrite, compare with the user's diff tool, etc.; whatever you'd expect from a good VCS when merging in conflicts). If neither switch is selected, then the engine would quit with an error warning about the conflict. Note that it would probably be nice to have a key argument to rename the file to make this easier to use.

I think that something like the above would be sufficient for most templates; can anyone think of something where it wouldn't work?

2 Likes