pre-RFC: a new symbol mangling scheme

[fitzgen]

It encodes information about generic parameters in a reversible way.

This will be HUGE for tooling and profilers! (eg for Twiggy’s monos sub-command)

[michaelwoerister]

This will be HUGE for tooling and profilers! (eg for Twiggy’s monos sub-command )

Yes, that’s one of the main motivations for changing the current scheme. I’m glad it’s appreciated :)

[cuviper]

The new scheme is based on the name mangling scheme from the Itanium C++ ABI .

Maybe this is a non-goal, but I kind of expected that this would let tools like c++filt work on Rust names as-is. This doesn’t seem to work with any of your examples though, even changing the _R prefix to _Z.

[michaelwoerister]

Why isn’t it the mangled name of core::ptr::drop_in_place::<type> ?

Because ptr::drop_in_place wasn’t thing yet the last time I dealt with drop-glue :) You’re right of course.

Likewise, the other shims could use the name of the function that they’re shimming, it’s not like there’s a “real” version that they can collide with

I’ll look into it.

[Centril]

To what extent is the goal here to stabilize a symbol mangling scheme and thus make it a part of “the spec”? Is the goal here just to replace the current unstable scheme with another, but better, scheme?

I’m wary of giving any stable guarantees that all Rust compilers must adhere to and which tools can expect will always hold. If this is just improving the unstable scheme then I have no objections.

[michaelwoerister]

Maybe this is a non-goal, but I kind of expected that this would let tools like c++filt work on Rust names as-is.

I think the main problem here is the two namespaces Rust has but C++ doesn’t (or at least not in this form). For manglings that only describe things in the type namespace (which won’t occur in practice), c++filt should work. E.g. _RN12std_a1b2c3d43foo3FooIN12std_a1b2c3d43bar3BarIiEEEE should demangle to std_a1b2c3d4::foo::Foo<std_a1b2c3d4::bar::Bar<int>> (modulo the _R).

Other problems would be the different set of basic types and things like the instantiating-crate suffix. The scheme described above could probably be brought closer to pure Itanium mangling in some places but actual compatibility is not a goal, I’d say.

[michaelwoerister]

To what extent is the goal here to stabilize a symbol mangling scheme and thus make it a part of “the spec”?

I wouldn’t want to commit to this yet. Maybe after a year of testing, or so. But this RFC would not stabilize the scheme as something to be prescribed for all implementations of Rust.

[nicoburns]

Would it make sense to standardise a version number somewhere, such that the mangling scheme could evolve, but tools could get an accurate demangling from only the binary (assuming they are aware of all versions)?

[Centril]

I wouldn’t want to commit to this yet. Maybe after a year of testing, or so.

So to clarify, a year of testing is unlikely to be enough here I think. My concern about committing to anything is that it makes it difficult to evolve the language with things such as const generics, higher kinded types, type-generic bounds (for<T: Debug>) and various unknown unknowns. That said, @nicoburns’s notion of versioning is interesting – what are your thoughts on that?

But this RFC would not stabilize the scheme as something to be prescribed for all implementations of Rust.

Great; so long as this is the case the language team need not be involved and T-compiler can deal with this alone (which improves the chances of adoption).

[gbutler]

I’m wary of giving any stable guarantees that all Rust compilers must adhere to and which tools can expect will always hold

Out of curiosity, could you elaborate a little on why stabilizing an ABI is something that Rust is so reluctant to do? Could there not be ABI “Editions” kind of like Rust Language Editions where backwards incompatible changes to ABI are introduced over time?

EDIT: Never mind. I see that you already have a reply to someone else along these lines. Also, someone else mentioned “Versioning” as well, so, nothing new here. Please move along.

[ubsan]

In order to support this without running into conflicts, symbol names for monomorphizations must include the id of the crate they are instantiated for. This scheme does this by appending an $<crate-id> suffix to the symbol.

Why? This is what inline linkage is for.

Rust allows unicode identifiers but our character set is restricted to ASCII alphanumerics, _ , and $ . In order to transcode the former to the latter, we use the same approach as Swift, which is: encode all identifiers via Punycode , a standardized and efficient encoding that keeps encoded strings in a rather human-readable format.

This is silly. We should just use utf-8 in our symbols, like the C++ compilers do (https://gcc.godbolt.org/z/WJXQsG (note that MSVC’s output is broken due to compiler explorer, not MSVC))

[gbutler]

We should just use utf-8 in our symbols,

How about use Puny-Code on back-ends/platforms that don’t support utf-8 identifiers and use utf-8 on those that do as part of the spec?

[ubsan]

If that’s necessary, but if LLVM is our backend shouldn’t they take care of it? On any system with an OS, it shouldn’t be a big deal, because C++ has already trod this road.

[gbutler]

but if LLVM is our backend

That shouldn’t be assumed by the spec should it?

[ubsan]

I’d argue that the spec should say that the mapping between name mangled identifiers and the identifiers used for linkers is implementation defined, since we shouldn’t really say anything about implementation of linking. In fact, I’d rather we define the name mangling scheme outside of the specification, since it’s not really important to the language, but only to specific implementations. We can define a specific name mangling scheme used by rustc, which other implementations can be compatible with.

[vadimcn]

It depends on compiler internals and its results cannot be replicated by another compiler implementation or external tool

I am not convinced that repeatability is worth fighting for, or even desirable. Consider this: a C++ compiler needs to generate symbol names after seeing just the header file. If it doesn’t generate the same symbol name as compilation that created the corresponding object file, linking will fail.
This is not a problem for Rust, which can save symbol names in crate metadata. As long as two implementations are compatible on crate metadata format level, they can generate symbol names however they like.
Am I missing something?

[Soni]

would this be any good towards allowing generic statics?

static<T> EVENT_BUS_TO_HANDLERS: EventBTH<T> = EventBTH { v: Vec::EMPTY };

(where Vec::EMPTY is a generic const as opposed to a generic static, so it’s irrelevant here. this would probably use a lazy_static in practice.)

[gnzlbg]

Big +1.

I’ll give you a +2 if the mangling library of rustc would be implemented in the nursery and would come with a cross-validated demangling library that extracts all information from a symbol and comes with tools to build the dictionaries for compression/decompression.

What I learned writing cargo asm/cargo llvm-ir is that everybody that needs demangling either writes a Rust demangler by hand, or tapes their own solution on top of rustcdemangle because that library isn’t enough for their purposes.

It would be really nice to have a solid library for this.

---

@Centril

To what extent is the goal here to stabilize a symbol mangling scheme and thus make it a part of “the spec”?

I would say “to no extend”. I suggest wording this RFC as a better name mangling scheme for rustc only (whether this evolves in a de-facto standard or not time will tell). Putting name mangling in the Rust spec would require a lot of work (*) and deliver very little value without a full ABI.

(*) we would need a mangling scheme that would work on all past, present, and future platforms that Rust targets. If we put a name mangling scheme in the spec, and a future platform does not support it, then we won’t be able to target this platform with Rust. Name mangling is something that screams “implementation defined”, as in, the spec might mention name mangling, and might mention guarantees about name mangling (e.g. the type of mapping), but nothing specific about how it works on each platforms.

---

@gbutler

I think it would be better to not derail this discussion with a discussion about a stable Rust ABI. If you are interested in this, it might be worth it to discuss that in parallel in a different issue. Name mangling would be just a small piece of that discussion.

[michaelwoerister]

Would it make sense to standardise a version number somewhere, such that the mangling scheme could evolve

Yes, I thought about that and I even planned to do it. It would be easy to add a version number just after the _R prefix, like in _R2N...E. _R without a number would mean the first version (as is done for numbering of substitutions). I’d want to make sure that the grammar allows for this.

[gbutler]

I think it would be better to not derail this discussion with a discussion about a stable Rust AB

Definitely agree. Apologies for bringing up something so off-topic.