[pre-RFC] lazy-static move to std

IIUC const fns can’t use heap allocation and will not be able to in short/mid term.

1 Like

One of the most common uses I have for lazy static is for regexes. Is const fn capable of supplanting that use case? (I genuinely don’t know. I’ve heard from some folks that it should be theoretically possible.)

Overall though, I do think lazy static belongs in std.

2 Likes

Miri can already do heap allocation. So getting them into const fn is mostly a matter for writing and accepting a bunch of RFCs, the implementation is already there.

Oh, and we should get const fn stabilized at some point…

Which kind of language features does that need? For example, everything HashMap does should work fine in const fn (though some of it is unconst).

1 Like

It would probably make sense to defer inclusion of lazy_static in std until it’s clearer where the limits of const fns are in practice.

6 Likes

Regardless of decisions about moving to std vs. waiting on const fn, whether it’s OK to leave the no_std implementation as a polyfill on crates.io, etc, it would be great if some changes that are currently pending a crater run could get a chance to merge, release, and bake for a little while before lazy_static goes into core/std.

@RalfJung while you’re asking about needed language features, I would also add that recently I have been writing a bunch of code that depends on calling opaque FFI functions (either directly or indirectly) from inside lazy_statics. I have not been tracking the RFCs and implementation of const_fn, but I feel somewhat safe in assuming that the feature will not be including the ability to call unsafe extern "C" functions. It’s not clear to me that this use case (plus others which are also more niche than “const_fn isn’t stable yet”) would have lazy_static as an important enough crate to move into std (and hopefully core at that point, I think leaving that feature split between std/crates.io for std/no_std would be :frowning:).

And how const fn() -> Vec<u8> should work? Will this return vector initialized on start of the program or will somehow be stored in data segment of the binary?

Yeah I don’t see us even trying to support extern "C" in FFI any time soon. Probably never. Non-determinism would break coherence unless we take some super expensive counter-measures.

So, this seems like a good motivation for lazy_static in libstd.

The final output value will probably still have restrictions, @oli-obk knows the status there better. What I have been talking about (and I should have been clearer) is the data that’s used during the computation. So you could e.g. build up a HashMap and fill it with all sorts of stuff and in the end compute some number (or some other Copy data), that would work fine.

However, given that you can only ever get a shared reference to a static, actually I see no problem with just putting the buffer part of the Vec into static memory. You can’t move out of this or push to it or otherwise cause reallocation, after all.

I mostly was talking about heap allocation (and I as well should’ve been more clear) in the context of return types.

I think we should be very careful with this approach, as code can assume that the underlying memory is handled by allocator. But maybe returning &'static Vec<u8> will be fine, I don’t know.

As an initial step we can allow

const fn foo() -> &'static [u8] {
    let mut v = Vec::new();
    v.push(42);
    v.leak()
}

and the leak method is the one from http://codyps.com/docs/leak/x86_64-unknown-linux-gnu/stable/leak/trait.Leak.html

And as a further step we can then consider a form of CtfeAllocator which allows us to return a Vec<u8, CtfeAllocator>. Although that is dangerous due to mutating methods and thus possibly impossible to safely

No, that is not correct. For example, this is a fully safe function:

fn fake_a_box<'a>(x: &'a &T) -> &'a Box<T> {
  unsafe { mem::transmute(x) }
}

The matching thing for Vec is also legal. See http://www.frankmcsherry.org/serialization/2015/05/04/unsafe-at-any-speed.html for more on this :wink:

1 Like

I’ve removed the point about generated documentation and added

const fn definitions are compile time definitions that can compute statics. If these functions could perform heap allocations lazily, there would be no need for lazy_static! in std.

Well, not really no need, as @anp mentioned above. static initialized via FFI calls would still need lazy_static.

However, using a CTFE-initialized static is certainly preferable where possible (no run-time initialization checks needed).

The need to move lazy_static to std is based on wide adoption and doing something important to many developers. I’m presuming lazy_static crate would be the best place for FFI statics if the wide use case went away. I do not mean to say that lazy_static itself would go away.

This RFC even quotes http://rust-lang.github.io/rfcs/1242-rust-lang-crates.html, which makes it very clear that std is an optional, hard-to-reach goal for a crate.

You’re skipping a stepshere – I’d petition for it to be moved into rust-lang first.

There’s very little motivation given here as to why lazy_static should be uplifted. The motivation given is very generic, stemming from What should go into the standard library? – it applies to many crates, not just lazy_static. Furthermore, that’s a discussion from 2015, and http://rust-lang.github.io/rfcs/1242-rust-lang-crates.html is the result of that, it’s been superseded by the RFC which basically explicitly considers std inclusion to be not a major goal.

These days the things that go in std are usually things that have to, like futures (for async support) and simd (since they need intrinsics)


I actually feel that petitioning for a lang-level addition will work better, here. Crates are assumed to be something you can easily import (not always true, but usually true), so uplifting them isn’t considered necessary or even a good thing (because now you’ve frozen the API). However, language additions can’t be done in external crates – only sometimes simulated as macros, so this may be work.

8 Likes

These days the things that go in std are usually things that have to, like futures (for async support) and simd (since they need intrinsics)

@Manishearth That’s a fair starting point. In my opinion:

  • std is a beneficial home for lazy_static! because, as a quasi language feature, it’s had to track several changes to the language that would be easier to manage if it were under the privileged environment that std offers. Its API has barely changed, but the implementation has several times.
  • lazy_static! is a worthwhile addition to std because, as a quasi language feature, it’s a self-contained cross-cutting concern that has applicability as broad as std itself.

However, I do think we could explore the rust-lang alternative more too.

Pardon the dumb question, but if lazy_static! goes into std, how do I use it in #![no_std] contexts? Because embedded code certainly uses a lot of lazy_static!

I suppose you really mean into core, but please say so.

I think that it’s safe to assume when someone says “this needs to go into std!” and “this” doesn’t involve Box or std::sync shenanigans, they means “this needs to go into libcore!”

shrug

3 Likes

It’s also a question of whether it needs to allocate memory. If it doesn’t, then it probably should go in libcore. If it does, then it won’t work in all [no_std] environments, so probably does need to be only in std.

Yeah, hence "Box shenanigans" (though we can split hairs and talk about Unique instead… =P)

1 Like