We have started asking ourselves about the various editions Rust-GCC was expected to support and were wondering what you were thinking.
This became more of a focus recently, as we started our Imports and Visibility milestone which contains a lot of edition-specific behavior regarding path resolution (among other issues). These differences directly affect the progress we are currently making on metadata and path resolution for visibilities/macros/ use statements.
What minimum edition would you expect gccrs to support? Are there any issues you could think of with us supporting “only” editions 2018 and up? We have not yet made any decision on the matter, and are simply looking for feeback.
Part of the backwards-compatibility guarantee of rust's edition system is that the compiler will always continue to support all previous editions. Also, that editions will remain inter-compatible so that you can (eg.) use a 2015 crate as a dependency of a 20XX crate and vice-versa. You can even mix editions in a single module via macros.
I'm not trying to give you extra work, but I don't think gccrs can be considered a complete rust compiler until it supports all editions.
Yes, and that doesn't mean that pre-2018 support is higher priority than other work you might be doing. If I was on your position I would likely target a single edition first, with full knowledge that some foundational crates might not be supported until I got around to add the complexity needed for the logic of editions. I would certainly incorporate the edition switching logic from the start, as you will need it either way.
I think you might be right about targeting a version. Currently, so much of my focus has been on rustc 1.49.0 since rustc 1.50.0 introduces const-generics, so for example generating a slice from an array in 1.49.0 completely changes in 1.50.0 which is really neat.
My goal here was if we target 1.49.0 libcore then we can figure out where to go from there, It seems new enough but old enough that there is a decent amount of information on it out there. I assume that 1.49.0 defaults to rust 2018?
I agree with @ekuber. There's two different things at play here: short and long term. Short term, it's probably easiest/best to focus on 2018 and 2021. Long term, 2015 is essential.
Yup, we have -frust-edition=2015|2018|2021. Our cargo-gccrs wrapper will simply use the value of the --edition flag and turn it into an -frust-edition= flag for gccrs
There are various discussions going on about "what is Rust", and defining what qualifies as a Rust implementation. One point that's been pretty universal in those discussions has been that Rust includes all previous editions, not just the current edition.
As much as I tend to appreciate a strong commitment to backwards compatibility, I can’t help but think the stance that ‘you have to support all Rust editions forever’ is a bit excessive.
I thought the whole editions mechanism was designed so that people would be expected to eventually migrate code to newer editions. Refusing support for sufficiently old editions might provide some additional incentive for crate maintainers; the stick to the carrot of new features.
This is not JavaScript where people often ship source code directly to production environments of varying freshness, which forces you to maintain perfect backwards compatibility as you evolve the language. Even C and C++ have deprecated and repurposed features from time to time (gets, register, auto, std::auto_ptr).
Speaking of C++ and JavaScript: one of the things that I cannot fail to appreciate enough (cannot depreciate enough?) in those languages is the profusion of redundant features that have been very hard to get rid of (pointers vs references and C-style arrays vs std::array in C++; var vs let vs const and function vs => vs class in JS). I would be very unhappy if Rust ended up in a similar situation, where each time I open a new Rust project I had to wonder whether I should use one syntax or the other depending on which edition the project uses. Those languages at least have pretty good excuses for not removing redundant syntaxes; Rust should do better.
And Rust does, too. Another point of Rust's edition system is that (unlike, say, C++17) Rust editions are not feature frozen. Even on edition 2015, 99.99% of new features work and are still considered idiomatic and should be used. Basically the only exception is new strong keywords (e.g. try) and edition-dependent name resolution (e.g. array.into_iter()), both of which are still expected to be rare in practice.
In other words, unless you're targeting an old rustc version (which is technically an unsupported use case still, until we have some sort of LTS policy) you're expected to have the latest and greatest with very few exceptions. (And because of the strong requirements for rustfix w.r.t. editions, actively maintained code should (unless it is targeting an old rustc version) thus be able to near-painlessly[1] upgrade to the most recent edition.)
One of the core promises of editions is that old code isn't going to be broken. In the C world, this is typically understood to apply at the ABI level: you can still use the old object files even if a new compiler refuses to compile the source[2]. In Rust, this is only a source-level guarantee, but the source-level guarantee is more important due to the lack of object stability to fall back onto.
If you want (or more likely, need) to continue using some legacy library code, you should be able to continue to use it. Perhaps if these libraries are proprietary and internal, you could argue that locking these libraries to the compiler(s) they were originally built against is "okay." But in actuality we live in a world of open source and freely licensed libraries (e.g. through crates-io) and it's important that these libraries continue to work with any compiler that wants to call itself a Rust compiler.
Perhaps in 20 years with more hindsight we'll find some misfeature in early editions that we don't want to burden new compilers with supporting. But at the time being, this seems unlikely enough.
(Now, cargo features on the other hand, such as resolver = "1', I could see a new implementation deciding not to support. But while cargo and Rust are linked, we're discussing what it takes to replace rustc with a new "Rust compiler," not also the build tooling.)
IIRC rustfix doesn't touch documentation tests, so those can still break, and there's a weak initiative to loosen the strictly-machine-applicable rustfix requirement for "we strongly expect no code will actually encounter this" edge cases to just linting and not fixing. ↩︎
While I agree with the point that many have made that a rust compiler cannot be considered complete until it supports all editions, there may be a simple way forwards. Assume that gccrs targets 2021, and is unable to handle earlier editions at first. In that case, it could error out if it encounters a crate that depends on an earlier edition, aborting the compilation. gccrs could also have a flag that forces the error to be a warning instead, which ensures that the choice is logged. As gccrs adds more editions that it supports, eventually getting all of them in place, the flag will become a historical note subject to deprecation and removal.
The most important rule for editions is that crates in one edition can interoperate seamlessly with crates compiled in other editions. This ensures that the decision to migrate to a newer edition is a "private one" that the crate can make without affecting others.
The next paragraph, I think, gives some insight into the level of maintenance effort that should be expected from supporting all editions, both within rustc and for alternate implementations:
The requirement for crate interoperability implies some limits on the kinds of changes that we can make in an edition. In general, changes that occur in an edition tend to be "skin deep". All Rust code, regardless of edition, is ultimately compiled to the same internal representation within the compiler.
I can't remember if this "same internal representation" is referring to HIR or MIR, but either way, there's a fairly well-documented (albeit not yet guaranteed-stable) representation of Rust code that is edition-agnostic. I'm not a compiler writer, but I think that even for non-rustc implementations, having an intermediate code representation that "erases" edition differences is almost certainly the right approach. As the quote above implies, there should never be fundamental differences between editions that would invalidate the approach of "erasing" editions fairly early in the compilation process. (Note: this erasure must happen after macro expansion, since macros are guaranteed to work "across" editions in the sense that a macro written for one edition may be imported and used by code written for another edition.)
Of course, it may even be possible to simply use HIR or MIR as defined for rustc as an intermediate representation for code within the new implementation. That would have an interesting advantage in that you could use rustc as a front-end to generate code from editions that the new implementation does not yet support. As I noted before, although MIR isn't yet stable, there is an intent to make it stable.
I haven't followed the discussions closely, but from what I understand, the intent is not to stabilize rustc's MIR but to stabilize some MIR that rustc can convert into and (maybe) from.
I don't think the Rust-GCC project wants to use HIR/MIR, though.
We have not planned any integration with rustc's HIR or MIR. We currently lower our AST into our own HIR which we then lower again to gcc's GENERIC/GIMPLE representation.
That's definitely what we're going for! I understood the comment I was replying to as "I don't think the Rust-GCC project wants to use rustc's future stabilized subset of HIR/MIR, though".
When handling the various editions, we will simply have different codepaths in our path resolution pass or macro visibility checker, etc. Similary to how rustc does it in some places:
let parent = match parent {
// ::foo is mounted at the crate root for 2015, and is the extern
// prelude for 2018+
kw::PathRoot if self.session.edition() > Edition::Edition2015 => {
"the list of imported crates".to_owned()
}
kw::PathRoot | kw::Crate => "the crate root".to_owned(),
_ => format!("`{}`", parent),
};
let mut msg = format!("could not find `{}` in {}", ident, parent);
(small example, focused on error reporting and not HIR/MIR lowering, but you get the point)
Once everything is lowered to our HIR, the representation will be the same. It does not matter to our HIR if we do "2015 path resolution" or "2018 path resolution". We will not have any weird classes such as an HIR::QualifiedPath2015 and HIR::QualifiedPath2018 which end up being compiled/typechecked differently or anything.
Support for specific edition differences can probably be based on popular crates.
Rust-GCC should support the idea of per-crate (*) editions from the start, but initially it can use 2021 behavior everywhere.
(*) Or rather per-span editions, to account for macros.
After that you try to compile some real code, encounter something like libc (?) using 2015 edition in its dependencies, and implement enough 2015-specific behavior to make it compile. Repeat for top N crates.
That might be sufficient to determine the priority order that parts of the editions need to be implemented in, but it definitely isn't enough to claim edition support. Anything other than 100% support of the given requirements of the edition means that it doesn't support the edition.
The absolute LAST thing I want to see for rust is some kind of weird fragmentation, where you have to know what compiler you're using to decide if you can use some language feature (or figure out why something is being miscompiled). I have enough headaches with that already trying to deal with feature flag differences between clang and gcc (You do not want to get me started on this rant, I have to deal with a large codebase that requires different versions of clang and gcc to compile correctly and it is Not Fun).
The last thing we want to do is fragment the ecosystem. This is why we started our cargo-gccrs project which strictly mimics the flags being passed to rustc, this is why we set up our testing project to run on various parts of the rustc testsuite and have rustc check our testsuite for correctness and this is why we're posting this message.
If the community believes we cannot claim edition support without supporting all editions, then we certainly won't make that claim!
The poll seems to heavily favor supporting the 2015 edition, which is what we will strive to achieve.
We really do not want to make anyone's life harder haha. The solution you mentionned about erroring out on unsupported editions was already mentionned by @bjorn3 and is an ongoing issue:
Forbid passing --frust-edition=2015 in bootstrapped compiler until all name resolution rules are respected
Which we can probably achieve with a deftly placed #ifdef in our compiler driver, allowing people hacking on the compiler's first stage to not face it but end-users to deal with it, forcing them to be aware of our limitations in that regard.
Finally, we are still very far from creating any release of the compiler - special care will be taken in order to not make anything harder than it needs to be for the community, and we will certainly not be rushing said release.
This suggests to me that rust needs to come up with an unambiguous specification of what rust is. I know that there have been ongoing efforts over the years, but if the Rust Foundation and everyone involved with rust really wants others to be able to create their own compilers like gccrs, then having an unambiguous definition is going to really need to become a top priority. Otherwise we're going to have the weirdness that is the various C/C++ compilers all of which implemented 'the standard', but each of which have corner cases that weren't covered in sufficient detail by the specification, and which then result in slightly different output for the same code. Which brings up the following:
Rust is a language, but it has to be compiled by tools to produce final binaries. Should rust also define the flags that compilers must accept? Are there other things that need to be considered as well? I'm thinking about this because things are not the same as when gcc was first created, when projects were often small enough that you didn't really need a build system, version control, etc., etc., etc. So maybe flags need to be considered as well, so that the compiler can be integrated as a component in a larger system, rather than the main thing we interface with.
Glad you guys already have it in place! Like I said earlier, for me personally that would be sufficient.
Glad you guys already have it in place! Like I said earlier, for me personally that would be sufficient.