A small step towards this that would be a large improvement would be some way to turn a file into a TokenStream with correct Span information.
Or in other words, give us the power to create Spans that refer to our own files. The ability to run a macro on “the whole crate” is great but unnecessary if we allow preprocessors to effectively pass rustc a processed AST.
Just as an example, consider a compile-to-Rust preprocessor that just does mod discovery. Then you could have access go through a singular procedural macro:
// lib.rs
rustx::mod_discovery!();
That procedural macro would then crawl the source directory for .rs files, parse them with the TokenStream API, insert inline mods as created by the file system, then spit out one whole collected TokenStream. So long as it keeps the Span information accurate, errors can remain accurate.
If someone can outline the likely changes needed to implement this, I’d love to take a stab at implementing it. (I have multiple projects that could benefit from it.)