A random number generator for libstd

[drXor]

Given that fast CSPRNGs exist, do we have any good reason not to pick a CSPRNG?

They have a point. My opinion: std is not the place for anything with cryptographic guarantees; those should live in libraries curated by domain experts.

That said, I don’t thing RNG belongs in std.

[CAD97]

To get at the root of the issue, why is rand not sufficient, that you want random number generation in std?

Rust takes a very minimal-std approach (or at least tries to); batteries are available for free at crates.io should you need them, rather than packed in with the stdlib. There’s a general consensus that std/core should consist of a) things that can’t be implemented elsewhere* and b) things that 99% of Rust users can benefit from (directly or indirectly). This means core abstractions that the language itself interfaces with and widely applicable ways of interacting with those abstractions.

A library living outside of std is liberating in that it doesn’t have to abide by Rust’s versioning, and you can explicitly list what version of it you need. rand has only one goal: generate random numbers correctly and usefully. std has to worry about so much more.

---

I agree that an RNG source should default to a CSPRNG if unspecified, for the same reason that std's HashMap defaults to SipHasher instead of something more performant: secure by default. If you need better specific behavior, you know what to ask for.

I also agree that std shouldn’t have to worry about providing chryptographic guarantees, and leave that to domain specialists. Another point towards having rand outside std and as the de facto “standard batteries” for those who need them.

---

*well ok, core and std are less special than they could be due to the ability to ignore them and be them with #[lang] attributes, but the average user shouldn’t need to handle reimplementing the tautologies that are <u32 as Add> and <Box as Deref> and other such lang items.

[ahmedcharles]

My position is that std should be an interface to the compiler, i.e. it should deal with lang-items, intrinsics, anything that can’t be implemented in stable Rust. I’d much prefer if it didn’t include OS integration, though and ironically, I think it should provide FFI to the relevant C types, which it doesn’t today (note, C types, like c_int, not libc (which should be the C standard library but isn’t… but I won’t rant about that here)). I suppose you could say that Rust has shown me that a minimalist standard library is possible and perhaps desired.

[JeffBurdges]

Afaik, there is no “convenience” or “increased productivity” achieved by merely avoiding the rand crate, which provides all the desired functionality without excessive complexity.

Instead, we should adding links from std's documentation to select high quality crates like rand, or even rustup installing rand's documentation.

We could ask if rand itself has some core that belongs in std, but really orthogonality wins over the “convenience of not googling” here.

As an aside, you could avoid using rand by abusing RandomState. Ironically though, one could easily imagine HashMap being removed from std after an external crate does probabilistic data structures as well as rand does to prngs.

As a further aside, I do think most functionality in std belongs there: Iterators may only interact with the core language through for loops, but they go beyond this in organizing the language. I suppose generators will drag Iterators and Futures deeper into std, but previously.

[hjr3]

The std lib does provide ffi types. Example: https://doc.rust-lang.org/std/os/raw/type.c_int.html

[ciprian.craciun]

The module should not:

• […]
• have a cryptographically secure generator

Please do not do this!!!

(I know it is a re-hash of what others have said in earlier replies, but I think this is such an important point, that I feel then need to repeat it…)

Do not introduce a non-cryptographical random number generator as the default in the standard library, especially for a modern language that tries to be “safe” (at least in some regards)… Today any modern OS has a secure random source, and there are enough fast cryptographic secure random number generators out there, that there is no good reason to use them.

[scottmcm]

Instead, we should adding links from std 's documentation to select high quality crates like rand , or even rustup installing rand 's documentation.

This scares me politically. I don’t want to encourage people sending PRs to link to their crates from std docs, no matter how good that crate is.

[kornel]

Perhaps adding of dependencies could be made easier, more automated instead, so there wouldn’t be a desire to have something in std just because it’s convenient, rather than because it’s essential for the language or interoperability?

For example, Node.js is experimenting with including modules on the fly. The Rust equivalent would be to have:

use rand::random;

Automatically add the rand create as a dependency.

[Nemo157]

That seems like it’s almost possible now, a custom lint that extends the existing error[E0432]: unresolved import `rand` error and checks if the crate exists on crates.io, then emits a cargo fixable suggestion to add it to the dependencies (hmm, can cargo fix touch the Cargo.toml, or is it limited to the source files currently?).

[Finn]

Ok, your safety arguments are pretty convincing. Often people are somewhat feather-headed, they could fall into a trap.

Having a cryptographically secure generator as default, finding a canonical API starts to become problematic. Despite that, std would become bloated rather than minimalistic. Furthermore, having half of the functionality in std and the other half in rand is quite cluttered.

Then it seems better to leave RNGs out of std.

[mrec]

the RNG needs to accept an (optional) seed to reproduce results later

I agree that’s a requirement for something, but I think it’s also a reason why this shouldn’t be in std. You’re advocating a CSPRNG. If a weakness is found in a CSPRNG, it’s no longer CS, at which point the algorithm needs to be fixed or replaced, at which point you’ve lost reproducibility from a given seed.

[zackw]

If a weakness is found in a CSPRNG, it’s no longer CS, at which point the algorithm needs to be fixed or replaced, at which point you’ve lost reproducibility from a given seed.

People who need reproducibility need to fix the algorithm anyway, and that in itself strikes me a inappropriate for std given the existence of rand and the general desire to keep the stdlib small. A stdlib RNG should be like rand::rngs::ThreadRng, IMO: based on a cryptographic generator, but without any guarantee of which cryptographic generator, and not offering any way to explicitly set the seed, thus directing people who need reproducibility to something else.

[dhardy]

As maintainer of the Rand crate, I’ve seen most of these arguments before — though the question about integration with std remains an interesting one.

We specifically made the rand_core crate fairly minimal. OsRng adds an external source of randomness (entropy) to avoid determinism. thread_rng is a fast auto-seeded cryptographic generator. Whether you need any more functionality really depends what you want to do with your randomness.

Personally I don’t think all this should be included in std itself. The amount of code is non-trivial (OsRng is implemented for many different platforms, we have a backup entropy source and are discussing options for entropy in no_std crates, thread_rng has periodic auto-reseeding with negligible overhead, …).

The caveat is that HashMap requires random keys hence some of this is included in std anyway — as duplicate code. Possibly the best solution to this is for std to depend on rand at some point in the future (though this is a circular dependency) or for HashMap to be moved out of std. Or maybe the required code should be in std, and exposed, and rand should use that instead of duplicating functionality — but this puts quite a bit more code in std and requires more maintenance work in std; still, it is the most straight-forward to implement.

Note that the Rand lib does contain quite a lot of code, though we’re in the process of moving PRNGs out to sub-crates and may also move most of the distribution code outside of Rand. Still, if you only want random data and don’t want to simulate a dice-roll or sample from a normal distribution or shuffle a vector etc. then there is quite a lot of unused code in the lib.

[Tom-Phinney]

Moving functionality to std just makes it less likely that IoT authors, who generally use no_std (due to their devices’ small memory footprints), will employ adequate sources of randomness. In the long run, requiring std for decent randomness is likely to increase the number of malware-laden captive IoT devices on the Internet, rather than improve the situation.

[dhardy]

This is a good point, but not an unsolvable one. 90% of Rand code is no_std compatible; the issues are a few things using Vec and OsRng. So most of the required code could be put in the core lib, and then lang items or similar could be used to specify the entropy source (in std or in a no_std crate).

[ahmedcharles]

TIL… Thanks!

[dcarosone]

Lots of platforms (but by no means all) have CPU intrinsics for random number generation. There are important caveats and variations on these, but they are still useful, especially in embedded case where other entropy sources are problematic. In projects where more comprehensive CSPRNG’s are available, they could/should still at least be used as a mixin entropy source.

I’ve used the rand crate in no_std and it works fine, for the clearly-documented subset of features.

What interests me here is the intersection of the two: so far as I can tell, there isn’t a clear way to get rand to use such an instruction where available – other than perhaps implicitly via OsRng if that happens to use it, and that’s not applicable not in the embedded case.

Is there something that could be put in core to access CPU randomness sources, and used by rand (and probably HashMap too) for benefit where available? It might not be something to recommend for direct consumption by those looking for quick shortcuts, given some of the caveats, but the current alternative seems to be requiring very specific platform knowledge in user code.

(Apologies if there is already such a thing that I missed, it was a while since I went looking).

[dhardy]

There’s currently a PR open on the Rand repo about adding hooks for entropy collection on no_std platforms. I won’t link to it because it’s a mess of details (which are entirely solvable) and it’s easy to find. In my opinion the most important part is the big picture.

Currently in Rand:

• rand::random produces a random value of many possible types, via conversion code on top of thread_rng
• rand::thread_rng provides a stream of random integers/bytes via a psuedo-random generator over an entropy source
• rand::rngs::EntropyRng abstracts over external entropy sources, converting to high-quality random data (but is potentially slow)

The goal of Rand is to provide all the above. The goal of rand_core is to expose a standard API for RNGs.

I bring all this up, because it exposes different layers of a random number system which could be introduced into std / core. As mentioned, std already has a copy of thread_rng in order to serve RandomState. Making this no_std compatible requires a little more work (which is the object of the PR mentioned above) but is feasible.

But it begs the question: is the goal of core to be minimal or merely to be portable? And how much functionality should std include? Should std depend on rand or is that the wrong way around?

(Unfortunately making std depend on rand without a cyclic dependency is hard because rand::rngs::OsRng requires file system IO on some platforms.)

[scottjmaddox]

I think it’s important to note that it’s significantly more difficult to build, test, and contribute to core and std than to an outside crate. This alone is a strong argument against bringing anything into core/std that doesn’t need to be in there.

I agree with the others who have suggested curated crate recommendations and cargo-fix tie-ins to easily add dependencies to Cargo.toml, in order to further reduce the friction of depending on outside crates.

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.