I am writing this post here on the Internals Forum in order to start a discussion on the matter as well as gather more information. I'm not a toolchain developer or a member of any of the Working Groups, so I have little knowledge on whether this is something that is actively being worked on. The latest semi-relevant discussion on the forum was, as far as I am aware, here, but perhaps I have missed something. In my eyes we lack a good place to discuss this.
Problem and use cases
The reason why I would like this to be discussed here is multifaceted.
The lack of grammar information in proc macros affects many areas of the toolchain, and there have been isolated attempts at bringing this up in projects like rust-analyzer
and rustfmt
, but what I believe we need is a unified interface that both of these projects - and any other that would benefit - can use. I will focus on these two purely because that's as far as my knowledge on the topic can reach.
This is neither the responsibility of an IDE integration nor any formatting tool. If rust-analyzer
were to implement something like this for its purposes, rustfmt
would need to have its own implementation, due to rustfmt
not being dependent on the other. The rustfmt
developer team also understandably has little interest in implementing this themselves. It's a heavy effort that is outside the scope of the project.
Function-like macros with custom syntax appear in crates that have wide usage throughout the ecosystem. Examples include serde_json
with 310 million downloads (although usage of the json!
macro isn't extremely common), sqlx
with 19 million, delegate
with 2.5 million, yew
with 1.3 million, leptos
with 0.6 million, and so on (as of writing). This does to my knowledge currently mostly impact frontend frameworks, since basically all of them use some flavor of JSX or RSX written using macros, but it is not limited to those either.
rustfmt
rustfmt
already has partial support for formatting some function-like macros (Does rustfmt format macro calls? · rust-lang/rustfmt · Discussion #5437 · GitHub), but the user experience while working with macros that use non-compliant syntax would be significantly improved by, for example, implementing a hook that libraries could use to define their own formatting rules and parsers for syntax trees.
This requires additional design work but would improve Rust toolchain's ability to handle other kinds of syntax and in turn make function-like macros much more appealing to use. It would also remove the need for library developers to maintain their own CLI tools for formatting macros specifically. This would both remove the pain of having to configure formatting differently for every project (these CLI tools do not have a unified interface) on the user's side, while also making it easier for developers to offer a positive experience for library consumers.
Examples of such CLI tools include dioxus-cli
, leptosfmt
(this one is a drop-in for rustfmt
thankfully), yew-fmt
(also compatible, but an unofficial tool!).
A solution to this problem is something that has been requested and mentioned on multiple occasions in different projects. See examples:
-
let proc-macro's provide a formatting hook for rustfmt · Issue #5656 · rust-lang/rustfmt · GitHub
(Suggests
rustfmt
implements said hook - low priority since last year.) -
format rsx using rustfmt · Issue #747 · DioxusLabs/dioxus · GitHub
(Links to
rust-lang/rustfmt#3705
andrust-lang/rustfmt#5254
, falls back todioxus-cli
as a workaround.) -
format macros · Issue #8 · rust-lang/rustfmt · GitHub
(Still no satisfying solution since it was proposed in 2015.)
-
(Needs formatting for Rust syntax in a specific scenario.)
-
Does rustfmt format macro calls? · rust-lang/rustfmt · Discussion #5437 · GitHub
(Explains current approach and comments on the issue at hand.)
-
How to format custom proc macro? - help - The Rust Programming Language Forum
(Expresses interest in having formatting in custom-syntax macros.)
rust-analyzer
The IDE integration could potentially benefit from proper syntax highlighting, attributing the correct documentation items to selected keywords, as well as code completion, in the context of custom-syntax macros.
You can develop IDE extensions for libraries as a workaround to adding grammar metadata to macros.
Those suffer from the same problems as external formatting tools. They are often unofficial hacks that become outdated over time and are of lower quality than something that could be provided by library developers. Leaving that task to library maintainers is an option, but in practice that just doesn't happen. Major frontend frameworks, all except for dioxus
, don't have official extensions, and the one provided by dioxus
- dioxuslabs.dioxus
- only wraps functionality from dioxus-cli
. It doesn't provide any of the things mentioned earlier.
This has been considered, among others, here:
-
(Asks for this exact feature.)
-
(Current workarounds, in practice this doesn't really offer what we want to the extent it could. See any of the frontend frameworks for results.)
-
(Points to
rust-lang/rust-analyzer#5389
andrust-lang/rust-analyzer#7402
.) -
(Dead since 2022, hoping for
rust-lang/rust-analyzer#7402
.) -
VSCode plugin for Hydroflow/Mermaid · Issue #340 · hydro-project/hydroflow · GitHub
(VSCode extension - low priority, no comment since last year. Many more like this one.)
-
(Same as
hydro-project/hydroflow#340
- waiting onrust-lang/rust-analyzer#5389
.) -
setup with rust and leptos? · Issue #42 · ThePrimeagen/htmx-lsp · GitHub
(Rightfully claims
rust-lang/rust-analyzer#5389
is unlikely to go through due to labelsE-hard
andS-unactionable
.) -
(Expresses interest in having code completion. Points to
intellij-rust/intellij-rust#6367
.) -
(Links to related
rust-analyzer
issues and workarounds.)
No one seems to want to or know how to deal with this, while I see it is a viable improvement. Addressing this lacking could greatly enhance library development experience and streamline tool integration. Being able to expose the expected input syntax would be a great tool in the hands of library developers. I encourage everyone to share their thoughts and experiences to help us potentially move towards a solution. It is not the first time this is brought up, and unlikely to be the last.
I would love to elaborate on the exact details of what kind of interface would need to be exposed for the use cases here, but I have very limited information on the internal design of the toolchain. I implore you to share your thoughts, though. I did what I could with the information I had. There are still many questions to be asked!
Hopefully this is of high enough quality for my first interaction with the Forum...