Pre-RFC: Add language support for global constructor functions

language design
1

Hi! I've written up a WIP RFC for adding global constructors, otherwise known as ctors or static initializers.

Crates like typetag currently use these constructors, but only support our tier-1 platforms because they depend on rust-ctor. By adding compiler support, we could expand this to all platforms.

If anyone could give feedback, either to the wording or the ideas, it would be greatly appreciated!

I've tried to fill in as much detail as I can, but it's fairly unrefined. I plan on refining this a bit then submitting an RFC pull request, if all goes well.

EDIT: This is the second version of the pre-RFC. I edited this on April 29th following replies - it does not incorporate all feedback yet, but at least some of it. There should be a little pencil icon on this post (top right I think?) which allows seeing the older versions.

Summary

Adds an attribute #[unsafe_global_constructor] marking a function to be run as part of program initialization, before main is called.

Global constructors will be run after the standard library has been initialized, but in an otherwise unspecified order. It is unsafe to declare a global constructor, and direct use will be discouraged in favour of wrapping libraries.

Motivation

There are three main motivations for this. The first is to enable collecting items from various crates into one place, and enabling higher-level crates like typetag crate to function on all platforms.

There is currently no cross-platform way to coordinate between crates which do not directly reference each other. The current solution is the inventory crate, the backing typetag. It supports first-tier platforms using rust-ctor.

For other kinds of initialization, we might be able to use a crate like lazy_static. But this won't work for cross-crate coordination because the crate using the data has no idea the crates providing the data exist. The key idea behind inventory is that crate a can define a data store, then crates b and c depending on a can add to that store.

Then an end-user using both a and b can just know that a fully works for b's data without having to explicitly initialize it. The biggest example if this is, again, typetag. Crate a can define a serializable trait, then end users can deserialize data directly into a trait object. The underlying concrete struct is defined in b or c, but a::Trait::deserialize can still deserialize

The currently existing mechanism for creating global constructors is rust-ctor. It works very similarly to this RFC, but with the disadvantages of only supporting Linux, Windows and Mac and running constructors before std is initialized. With compiler support, we could extend this to support all platforms.

One particular motivating factor is the want for typetag support on the wasm32-unknown-unknown platform. See rustwasm/wasm-bindgen#1216, mmastrac/rust-ctor#14 and dtolnay/typetag#8.

The second motivation is allowing initializing FFI libraries without synchronization, either with std::sync::Once or lazy_static.

This is a less strong motivation, as we can currently

Guide-level explanation

Most rust code runs "after main". Functions running were usually all called at some point called by another function, and those functions were called by others, etc., until we reach the main function which started it all.

This makes sense for most cases, but in some cases, having separate entry points which can initialize global structures before main can be useful. Global constructors fill this niche.

Consider this code:

use std::sync::atomic::{AtomicU8, Ordering};

static my_atomic_var: AtomicU8 = AtomicU8::new(31);

#[unsafe_global_constructor]
fn run_this_first() {
    my_atomic_var.store(32, Ordering::SeqCst);
}

fn main() {
    println!("Hello, world! My variable is {}", my_atomic_var.load(Ordering::SeqCst));
}

This is an example of declaring a global constructor. run_this_first will be called during program initialization, separately from main. When multiple global constructors exist, it's unspecified which runs first.

At compile time, we initialize my_atomic_var to 31. Also at compile time, the list of global constructors is created, and includes our function run_this_first.

When our program runs, first, the global constructors are loaded, and run in some order. Among these is run_this_first, and my_atomic_var is set to 32.

Finally, we enter main, and observe the variable as 32.

Great, right? It is, but we have to be careful.

Global constructors run before main, and Global constructors don't just run before main, they also run before other initialization the standard library performs. They should never use std::io, and should never panic. To recognize this unsafety, all unsafe_global_constructor functions must be declared as unsafe.

Finally, note that creating global constructors should be avoided whenever possible. Any nontrivial computation can slow down program startup, and upmost care must be taken to ensure all code is sound.

Most rust code, outside of a few support libraries, is expected to be completely free of global constructors. If this kind of initialization is needed, it's recommended to use a higher level library like inventory. Other crates might use this without ever realizing it, when they use crates like typetag.

As using global_constructor directly is discouraged, I don't anticipate this feature being taught to new rust programmers.

Constructing things at compile time and running code at runtime is Explain the proposal as if it was already included in the language and you were teaching it to another Rust programmer. That generally means:

  • Introducing new named concepts.
  • Explaining the feature largely in terms of examples.
  • Explaining how Rust programmers should think about the feature, and how it should impact the way they use Rust. It should explain the impact as concretely as possible.
  • If applicable, provide sample error messages, deprecation warnings, or migration guidance.
  • If applicable, describe the differences between teaching this to existing Rust programmers and new Rust programmers.

For implementation-oriented RFCs (e.g. for compiler internals), this section should focus on how compiler contributors should think about the change, and give examples of its concrete impact. For policy RFCs, this section should provide an example-driven introduction to the policy, and explain its impact in concrete terms.

Reference-level explanation

Any otherwise-plain unsafe rust function may be marked with the #[unsafe_global_constructor] attribute. When marked, it will be added to a list internal to the compiler, and included as a global constructor to be run before main on program launch.

For example,

#[unsafe_global_constructor]
fn my_constructor() {}

Internally, this will add the function to LLVM's @global_ctors list.

Global constructor functions will be allowed in all crates, and in all modules (including inside other functions). Privacy of the global constructor function will not effect it being a global constructor.

Global constructor functions must:

  • take zero arguments
  • be monomoprhic
  • return ()

To mark a function as #[unsafe_global_ctor] without satisfying these requirements is an error

All global constructor functions in all included crates will be called, but the order is explicitly unspecified. There are no guarantees, even for functions declared directly adjacent in the same module.

Global constructor functions may be under a #[cfg] flag, and this will behave as expected. The following constructor will be called if the code is compiled with the construct_things feature, and won't if it isn't.

#[cfg(feature = "construct_things")]
#[unsafe_global_constructor]
fn my_constructor() {}

Global constructor functions must not be under #[target_feature]. Global constructor functions are always called, and thus it wouldn't make sense to have code which can't run on some of the target machines. For example, both of the following will result in compile-time errors:

#[target_feature(enable = "sse3")]
#[unsafe_global_constructor]
fn bad_ctor() {
}

#[target_feature(enable = "sse3")]
mod my_sse3_module {
    #[unsafe_global_constructor]
    fn second_bad_ctor() {
    }
}

Drawbacks

This introduces life-before-main into rust, something we explicitly avoided in the early days. See this quote from the old website's FAQ (inlined for posterity):

Does Rust allow non-constant-expression values for globals?

No. Globals cannot have a non-constant-expression constructor and cannot have a destructor at all. Static constructors are undesirable because portably ensuring a static initialization order is difficult. Life before main is often considered a misfeature, so Rust does not allow it.

See the C++ FQA about the “static initialization order fiasco”, and Eric Lippert’s blog for the challenges in C#, which also has this feature.

This brings up two problems which are still relevant today:

  • Like C++'s static initializers, global constructors will have an unspecified run order. As we can't initialize (only change) statics in global constructors, the danger is somewhat mitigated, but not entirely. Two global constructors could still rely on run order, and introduce subtle bugs.

Rationale and alternatives

  • Simply don't implement this.

    As mentioned in the drawbacks section, this is a dangerous addition.

    If we don't implement this, we could then further shame using rust-ctor, or let it be and simply not "grace" the feature with compiler support.

    On the other hand, I believe the advantages do outweigh the disadvantages. We currently don't have a good way to implement cross-crate initialization (like inventory) without global constructors. If we explicitly discourage direct use and ensure all users are aware of the unsafety, we should be able to minimize the danger.

  • Somehow define the order in which global constructors are called.

    For example, in C++, it's guaranteed that static initializers declared in the same file execute in the same order that they are declared in. It could be prudent to define partial order, or a complete order, to the way in which rust's global constructors run.

    This could be useful, but it could also allow people to start depending on fickle orders. The order being unspecified leaves it all up in the air, and ensures that users know they can't depend on anything they don't synchronize themselves.

    Guaranteeing order of global constructors between crates might be reasonable, but I'd be very wary of having any sort of implicit ordering within the same crate.

    If, for example, we were to order as encountered in a parse tree, this can all become more hairy. It'd be fairly easy to have one module implicitly depend on another module's global constructor running first. But then, what if rustfmt reorders the mod declarations? What if in refactoring, one module is renamed, making it sort differently and putting it above the other? These seemingly entirely innocent changes could break code depending on this order.

    Even worse, though, the breaking wouldn't be discovered until runtime. Global constructors have a real potential to give us unpredictable runtime errors, and I think keeping the order fully undefined should help us avoid that.

  • Implement something closer to the inventory crate, allowing separate crates to coordinate data without running code on app startup.

    If this becomes a viable alternative, it would be able to solve one of the motivation factors for this RFC without many of the disadvantages of global constructors, such as initialization order and runtime cost on application startup.

    As of right now, this is looking more appealing.

    There are two disadvantages I know of:

    • This may require a more complicated implementation, whereas LLVM already supports code generation for C++ static initializers. [TODO: research differences?]
    • This would not help FFI initialization, another use case for global constructors.

    See the [distributed slice] crate for an example implementation of something like this for some platforms.

And, we have some smaller changes which could be made:

  • Rename the attribute.

    C++ calls this kind of thing a "static initializer", and the existing rust-ctor crate simply uses the #[ctor] attribute.

    We could rename #[unsafe_global_constructor] to #[unsafe_global_ctor], #[unsafe_ctor], #[unsafe_initializer], #[unsafe_static_initializer], or another combination.

    A different alternative would be to emphasize the registering nature of this, and go with something like #[register_main_hook].

    I chose #[unsafe_global_constructor] as it's reasonably descriptive, references being program-wide, and doesn't imply it's necessarily initializing a particular value. It likely is initializing something, but this differentiates the feature from C++ static initializers, which always initialize a single static which was previously undefined (TODO: citation needed).

  • Apply #[unsafe_global_constructor] to fn() typed statics, in addition to or instead of to functions.

    This strictly increases the possible uses. I've not included it for the sake of being minimalistic, no other reason.

Prior art

This is heavily inspired by the rust-ctor crate, which implements this feature in "user-land" for Linux, Windows and Mac.

One main basis for this is "static initializers" in C++. A number of blogs describe using this feature, and why not to:

C++ static initializers allow a program to initialize static variables to some value, possibly calling functions which will be run at runtime. Problems occur when one static variable depends on calling methods on another: the initialization order is unspecified, so programs written this way crash 50% of the time.

This could definitely be a problem in Rust as well, but it's mitigated by one key difference: #[unsafe_global_constructor] will never initialize a static variable.

The constructors proposed can change value of statics, but all statics must still have some sane and initialized default.

A second, smaller difference will be heavily discouraging use of this feature outside of specific abstraction libraries. If end users never explicitly use global constructors, then

TODO: read through & add Object Pascal (Pre-RFC: Add language support for global constructor functions - #26)

TODO: read through & add Ada (Pre-RFC: Add language support for global constructor functions - #15 by mjw)

Unresolved questions

  • Is it feasible to allow panicking within global constructors?

    I personally don't know enough about landing pads and such to know if we could do this. (or even if we could say something like "all panics in global constructors are guaranteed segfaults", which would be better than leaving this up in the air).

  • Should static methods be allowed as global constructors? For example,

    impl MyStruct {
    #[unsafe_global_constructor]
    fn my_gctor() { ... }
}

    There doesn't seem to be much downside besides "it looks odd", and the upside is allowing something that we don't really have any reason to deny.

  • Should extern functions be allowed as global constructors? For example,

    #[unsafe_global_constructor]
extern "C" fn my_gctor() { ... }

    I'm unsure of what our interactions with extern "C" will be, or what someone might expect from this. Unless we can say for sure that this is fine, it seems like disallowing this would be a good conservative position.

  • What platforms is it feasible to support?

    I make the assumption that because C++ has static initializers, LLVM will support @global_ctors on all of its supported platforms. This could be a naive assumption.

  • How should this interact with #[no_main]?

    This is an unresolved question purely due to lack of research on my part. Since this is a pre-RFC, hope it's alright to leave this in here until I actually do that research?

Future possibilities

  • #[unsafe_global_constructor] on fn()-typed statics could be added at a later date.
  • The order in which global constructors are run could be partially or fully defined at a later date.
3 Likes
2

I think panicking can be allowed in ctors iff they’re monomorphized to panic=abort. We very much do not want to make panic=UB in ctors, but also it doesn’t make sense to recover from a ctor panic.

As far as std goes, I think we should use a whitelist of “ctor-safe” functions, and potentially a lint to enforce it.

1 Like
3

Some suggestions as the author of the ctor crate (who would honestly love to see the need for my crate disappear!):

Having #[ctor] methods run before stdlib initializes is more of a bug than a feature. The only reason it doesn’t run after-stdlib-init-but-before-main is because there is currently no hook for this. If there was a way to run in this “Goldilocks” zone of initialization, it would be far more valuable. Ditto for destructors - run after main, but before stdlib teardown.

Global constructor order is a tough problem to solve. If module A uses FFI to initialize a curses library, but imports module that does some dylib magic in its global constructor required for that curses library to initialize, how do we ensure the correct ordering? LLVM/ELF/etc generally offer a priority system for this, but it very quickly breaks down.

Could we design a system where Rust constructors could guarantee that they run after modules they import? This could be very useful for typetag and inventory-type crates.

Notes on prior art:

Go has an interesting per-package init() design that runs initializers in a predetermined order (afaict alphabetically). This ensures that init order is reliable and consistent across invocations, assuming that a new package isn’t added. Packages have no control over initialization order with their sub-packages, which means they must be careful not to rely on things initializing correctly in the sub-package for their own init.

Java’s static constructors run once-and-only-once, but lazily and “depth first” - ie the children are initialized before the parents. There are a whole boatload of gotchas with this approach, mainly around locking, recovery from failure, etc.

Current uses of the ctor crate:

The three uses I’ve seen so far (which seem to provide us 90% of the value of this feature) are:

  1. Initializing FFI libraries before methods are called
  2. “Collecting” items from other modules. This could be classes implementing a certain trait, data/configuration, or registering hooks into other modules (ie: a logging system).
  3. Initializing things that aren’t const before code runs. This overlaps a lot with lazy_static.
6 Likes
4

Well, I’m not a fan of global constructors, but the existence of rust-ctor illustrates the danger of taking useful functionality from C/C++ and refusing to implement it. In a language as flexible as Rust, someone will implement it anyway, but without the benefit of the compiler knowing what’s going on, often resulting in brokenness.

FWIW, it should be possible to implement inventory without using global constructors. rust-ctor works by placing statics in a specially-named section to invoke OS-specific functionality that automatically treats them as pointers to initialization routines. But you could also put statics in an arbitrarily-named section and use OS-specific functionality to get the beginning and end of that section at runtime:

That way you can have a function iterate over all the statics in the program that are declared in that section, with no need to run code before main.

In theory, it should also be possible for rustc to implement a version of this “put me in a list” operation itself, based on the metadata embedded in rlibs and Rust dylibs, rather than having to rely on OS-specific constructs.

Edit: The above won’t work with dylibs, though, since you’ll only see the symbols defined within your own dylib. A metadata-based native Rust version wouldn’t have that limitation.

8 Likes
5

I wonder if global constructors could be made explicit (and if not why not). That is:

use std::sync::atomic::{AtomicU8, Ordering};

static my_atomic_var: AtomicU8 = AtomicU8::new(31);

#[global_constructor]
fn run_this_first() {
    my_atomic_var.store(32, Ordering::SeqCst);
}

fn main() {
    run_this_first(); // without this line it does not compile
    println!("Hello, world! My variable is {}", my_atomic_var.load(Ordering::SeqCst));
}

This doesn’t solve the problem of constructors depending on each other, although we can easily enforce some DAG ordering if needed. As far as this just adding boilerplate, I’m not sure that’s a bad thing given the other option is just not knowing about initializers. We could always add a std::blah::run_all_initializers() to tone that down.

5 Likes
6

Your current proposal doesn’t properly motivate why lazy_static is not enough. In fact, it doesn’t mention lazy_static at all.

8 Likes
7

I'm in general not very excited about this. It seems to me to encourage global singletons and action at a far off distance. There are however more serious problems...

Please mention that a #[global_constructor] must be unsafe.

And what about cranelift, or other backends? Perhaps the JVM if we stretch things a bit?

  • be monomorphic

This seems backwards. An unsafe fn introduces a proof obligation; it does not discharge one. You are effectively using #[global_constructor] here as the proof obligation discharger instead of unsafe { ... }. However, there's no dischargement of that the composition of all #[global_constructor]s are in fact sound.

The proposal strikes me as a security problem.

Many people don't review their dependencies because there's an implicit, and perhaps naive, trust that people don't do nefarious things. Even people who do review crates can have a bad day.

While it is possible to inject evil code in any function in crate D and have crate A simply execute that function, it is far easier and more reliable to simply add a #[global_constructor] in crate D and it will now be magically called.

Therefore, I think at minimum the crate that defines fn main() { ... } or whatever entry point there is, must explicitly opt into invoking the global constructors in some fashion. One solution to that is to have a #[lang_item = "start_hook"] unsafe fn core::start::run_all_hooks(); function which will simply call all the functions that are #[global_constructor]s. You then use this function in fn main() { ... } and explicitly use unsafe { ... } to discharge all proof obligations. I believe this should also side-step the questions around standard library initialization, extern functions, and panics. This also doesn't rely on LLVM which is in my view a plus.

I think it's safe to say that this is not a realistic option. I think this should be supported either on all (literally) platforms, or no platforms.

I think that's a good idea; when I hear "constructor", I expect it to construct something. I associate this term with -> Self functions in Rust. Something like #[register_main_hook] seems appropriate. Initializer also seems better than constructor.

Why yes, why no?

The amount of things that are const will steadily increase over time. This should become more and more a peripheral problem.

7 Likes
8

I think this RFC can replace a part of lazy static and improve performance. Is this right?

1 Like
9

Like C++'s static initializers, global constructors will have an unspecified run order. As we can’t initialize (only change) statics in global constructors, the danger is somewhat mitigated, but not entirely.

I think this is the major point that makes this proposal a reasonable direction, over the fiasco of C++ static initializers. I think the RFC would benefit from a detailed discussion that contrasts what is being proposed vs what exists in C++. Show some seemingly innocuous C++ code in which the initializer of one static observes a different static in an uninitialized state (breaking the type system), such as the following which segfaults on my machine:

#include <string>

extern std::string S;

char R = S[3];
std::string S = "0123456789";

int main() {}

Then show how there is no "equivalent" Rust code under your proposal i.e. the proposal does not break the type system; any static initializer whose code refers to a value S will always observe a legal value of the right type according to the type of S. Now, that may be Option<Something> which is None or MaybeUninit<Something> which is not initialized, but in all cases the value observed by a static initializer is accurately described by the thing's type.

I think adding this discussion would help get people on the same page regarding the proposal, especially readers who have only heard that static initializers are a disaster without having seen enough of the details to understand why this proposal is not the same thing.

10 Likes
10

I would like to see in the RFC a (very serious and fleshed out) alternative of introducing instead a more restricted foundation that still allows for implementing something similarly powerful to the API of inventory without introducing any life-before-main. Fundamentally the thing needed for inventory is “have a way of knowing about things without depending on them,” which is a very different requirement from “have a way to run things before main.”

See linkme::distributed_slice for something in the direction that I have in mind.

I would encourage you to consider seriously whether this approach might be better than the one in the RFC, as I think it might be, and pivot the RFC accordingly.

17 Likes
11

If you're injecting arbitrary evil code, you can just use the method rust-ctor uses to register global constructors... or link in some C code that does the same... or put evil code in build.rs... or stick a call into every function in the crate... or just take 5 minutes to figure out which functions in the crate are actually used. I really don't think #[global_constructor] would make a meaningful difference.

3 Likes
12

For the record, this is the same approach that I mentioned in my earlier post. @dtolnay is the author of both linkme and inventory, which is older; I guess you changed your mind about the best approach?

For cranelift or other native backends, as demonstrated by rust-ctor, it's just a matter of putting a function pointer in the right section. JVM seems pretty far afield, but it does support global initializers.

1 Like
13

What would be the behaviour when building a binary that does not call these global constructor functions if one of your dependencies specifies one? (e.g. via #![no_main] if it becomes part of the std-lib setup, or via a custom linker script which ignores the specified section).

14

The RFC doesn’t propose adding anything to the Rust-specific main shim, but using existing OS functionality to register global constructors. They’re usually called by either the dynamic linker or (for static binaries or embedded stuff) crt0. Thus, #![no_main] wouldn’t affect anything. A custom linker script that drops the section would presumably cause the constructors to not run, but… there’s no shortage of other ways to screw up your binary by putting the wrong thing in a linker script. :slight_smile:

Edit: I meant main shim, not start shim. I was a little confused. In my defense, this is what a typical Rust startup looks like:

  1. The native symbol called start or _start (from crt0) calls
  2. the native symbol called main; Rust provides the “main shim”, which calls
  3. either std's, lang_start, marked [lang = "start"], or a user fn marked #[start], conventionally named start; the former calls
  4. fn main()

Yes, it goes start -> main -> start -> main. Perhaps that helps explain my confusion. ;p

2 Likes
15

For prior art, looking at Ada should be worthwhile, particularly for the run-order problem.

There’s an decent overview here of both the language model and some Gnu extensions:

https://docs.adacore.com/gnat_ugn-docs/html/gnat_ugn/gnat_ugn/elaboration_order_handling_in_gnat.html

2 Likes
16

...but that doesn't work on all platforms, which is why we are discussing this?

...assuming you have a C compiler available (which can be prevented)

I don't buy the general argument that "two wrongs make a right"... I think build.rs is a security problem, but I don't see why another risky avenue should be added.

None of those functions may actually be called always and at opportune moments.

2 Likes
17

Can we enumerate specific use-cases of what people would use it for, and see whether we can add more specific solutions to these problems?

Running any number of arbitrary pieces of code generalizes the problem to the maximum, with associated risks and costs, and maybe we don’t need to pay for that much generality?

If I understand correctly, the typetag doesn’t actually want to run code, it wants to learn about all implementors of a specific trait to be able to handle them all in a coherent way:

// generated (conceptually)
#[derive(Serialize, Deserialize)]
enum WebEvent {
    PageLoad(PageLoad),
    Click(Click),
    /* ... */
}

I’ve also ran into similar situation where I’d want to have a factory pattern, or some plug-in system, in which any type from anywhere in the program can participate. So that seems useful, but could be solved with something else, like an automagically-generated enum of all these types.

13 Likes
18

The problem of this mechanism is that without run order guarantees code in global constructors will trigger undefined behavior if it calls any module that requires its own global constructors to run first.

This means that effectively global constructors cannot use any code from any other crate, since that crate may be changed to rely on global constructors without an ABI break, which is pretty limiting.

So I think it’s essential to provide a deterministic run order: a crate’s constructors must run before the ones in any crate that depends on it, and within a single crate they must run in the order they are found by a simple parsing that goes inside submodule.

While this guarantees no inter-crate UB, the developer must be careful to not use anything in the current crate that requires a global constructor to be run.

EDIT: it seems like that with this guarantee (and possibly some stdlib changes), unsafe is not required, although in most cases global constructors will be declared unsafe since they will modify static mut data.

19

I’m not excited about adding native support for this to the language. However, I do think it should be possible…just not quite this natively. Rust should have enough support for emitting data into arbitrary sections of the compiled binary that you can easily put function pointers into the .ctors section, and then I think the remainder should stay a library rather than a language feature. (There are many other uses of accumulating data in separate binary sections, and we need to support those regardless; if there isn’t enough native support to do that, then I think we should add that.)

Separate from that, I wonder how many uses of this could be handled through having feature-flags on that library to allow just calling ctors_init() from the start of main() so that it can’t run before main. Register callbacks, call them intentionally on startup. That would avoid the “life before main” problem.

5 Likes
20

I don't think you've appropriately defined a security model in your posts. As @Comex points out, not having compiler support for library constructors is not going to save you, at all. "Not having this makes it harder and less reliable" is not a good response, and arguably luls reviewers into a false sense of security. You already have arbitrary link section control, so you already need to audit this.

A lot of locking primitives don't work on embedded platforms; intrinsics like likely and #[cold] cannot be expected to do much on platforms without speculative execution. Supporting every low-level feature on every platform is not a reasonable expectation.

Prior art in dynamic linker design calls these ctors and dtors. In C++, they also have nothing to do with class ctors and dtors. I think that deviating from this nomenclature is confusing, since I expect this feature will not be used heavily, and users will expect a name similar to gcc and clang's.

I don't like dynamic linking, but I think it's extremely necessary. For example, I really wish I could use something like gflags in Rust, especially a version compatible with shared libraries (which const evaluation might not be...).

4 Likes