Idea: global static variables extendable at compile-time

Following this discussion I thought I’ll describe a functionality which I think can be quite useful for some cases and AFAIK can not be implemented today using other means. In short I would like to propose an idea of static variables which can be extended by other crates. It can look like this:

// crate `foo`

// extendable static should be public and have type `&[T]`
#[extendable]
pub static ERROR_MSGS: &'static [(u32, &'static str)] = &[
    (ERROR_CODE_1, ERROR_MSG_1),
    (ERROR_CODE_2, ERROR_MSG_2),
];
// crate `bar` which depends on crate `foo`
use foo::ERROR_MSGS;

// if extended static has type `&[T]`, then this const should have type `T`
#[extend(ERROR_MSGS)]
const CUSTOM_ERROR: (u32, &'static str) = (0x123456, "custom err message");

Now if our project contains both foo and bar, then foo::ERROR_MSGS will contain 3 elements. In other words compiler will have to collect all #[extend(..)] statements and construct extendable static only after all dependencies got compiled. (yes, I am aware that it does not fit the current build process) Now we can use linear search to convert error codes to human readable error messages.

Because crates are compiled in parallel, it’s probably will be better to define inclusion order for extension elements to make order deterministic, one option is to use crate names to determine inclusion order.

What do you think about such functionality? Does associated complexity outweigh potential merits? Do you see other uses for it outside of error codes?

UPD: linkme project by @dtolnay does very similar thing using linker shenanigans.

7 Likes

This is basically linkme::distributed_slice, as brought up in the Reddit thread.

I am pretty optimistic about the general approach of exposing distributed elements as a &'static [T]. I think this solves most of the remaining reasons that people want global constructors that are not already expressible in other ways in Rust. In particular this would supersede inventory as it would be sufficient for implementing all the things currently based on it: typetag, gflags, pyo3 etc.

I would like Rust to support this pattern in some form; syntax tbd. Want to write an RFC? Once this exists I think we can continue to not support global constructors / static initialization which are powerful but unsafe and undesirable for many reasons.

20 Likes

I am not sure if I will be able to describe this proposal properly (especially regarding how this functionality should be handled by compiler), I will try to write a draft a bit later and will link it here.

@newpavlov Do you still plan on writing this proposal?

Asking because I’m interested in pursuing this as well, but I’m also a little short on time.

If there’s still a need for an RFC in around a month or so, I could probably get things together and write something up, or work together if you’d want to collaborate. But if you’re still working on it and have time earlier than that, I’d be happy to just support your version and move forward with it.

Overall I just want to see this feature move forward, and do whatever is necessary to see that happen.

3 Likes

I would like to help with this one. Any progress since last post?

1 Like

I haven't done anything with this since my last post!

If you're interested in going further with this, I think both seeing how well linkme works in practice (the inventory crate currently uses ctor, but it should be able to use linkme instead, hopefully?), and going ahead writing a (pre) RFC for including something like linkme::distributed_slice in std are good next steps.

I previously wrote Pre-RFC: Add language support for global constructor functions, and a lot of the background information and motivation could probably be copied into a Pre-RFC for adding this.

I'm interested in working on this as well, but I've done very little since that last thread finished (with the conclusion that this is a safer and more minimal way to go). If you're willing to carry this further, that would be awesome! I probably won't initiate or write much, but if you've got any questions I can try and help.


Edit: After writing this, I've looked into the current state of things a bit more. Looks like linkme is intended as an alternative to inventory, not as a new backend for it.

This is a tracking issue for switching typetag to use linkme over inventory, then: https://github.com/dtolnay/typetag/issues/15

And a partial implementation: https://github.com/dtolnay/typetag/pull/16

I think the best ways to help this forward now would either be to contribute to that (and thus add further proof that linkme is a good enough interface to add to rustc), or to work on a new Pre-RFC for adding the feature.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.