ReadBuf as part of Rust edition 2021

[Qwaz]

Passing an uninitialized buffer to generic Read implementation is one of the most common soundness bugs in the Rust ecosystem (e.g., RUSTSEC-2021-0015, RUSTSEC-2021-0029, gfx#3567). Unfortunately, initializing the buffer often leads to performance degradation, and there is currently no way to use generic Read without initialization cost in stable Rust.

RFC 2930 provides a solution to this issue. It adds ReadBuf type and related APIs to Read trait, which allows safe management of uninitialized buffer. This RFC has been merged, but there is no implementation available yet.

What I would like to suggest is to prioritize, implement, and advertise RFC 2930 as part of edition 2021. Technically ReadBuf is not a breaking change, but as noted in the RFC, ReadBuf introduces nontrivial amount of complexity to one of the standard library's core traits. Thus, advertising it as a change introduced in a new edition will help people learn and adopt new APIs, because people are more willing to accept larger changes at the edition boundary.

Additionally, raising the awareness of ReadBuf would help resolve longstanding misconception about uninitialized buffers in Rust. Although incorrect use of an uninitialized buffer can lead to observable memory safety errors (#1, #2), the impact of using an uninitialized buffer is often overlooked. I've seen several soundness bugs that follow the similar pattern of uninitialized read() (e.g., RUSTSEC-2020-0123, glium#1907, claxon#26). When implemented, ReadBuf could be a solid example that explains an uninitialized buffer cannot be passed to an API that is not expecting it, which I believe helps prevent the occurrence of similar soundness bugs in the future.

[mbrubeck]

Welcome back to the Internals forum, @Qwaz!

I just want to point out, in case anyone isn't already familiar, that @Qwaz and the other members of the Rust group at SSLab are responsible for identifying nearly half of all the vulnerabilities reported to rustsec.org in recent months. They probably know more than anyone else about which Rust features lead to the most security bugs in the wild.

[jhpratt]

I'll take what @mbrubeck said at face value, and say that if you believe this is worth prioritizing, it probably should be. There's a fair amount of time until Rust 2021 gets released (October IIRC), so something like this certainly has time to land at the same time.

[CAD97]

Not to throw a damper on the accepted RFC, but I'm a little worried about possible confusion between ReadBuf (a partially uninitialized buffer to read into) and BufRead (a read that uses reader managed buffers rather than caller managed buffers). The latter is potentially going to be important for async, so having these share such similar names is... unfortunate.

I do support prioritization of this functionality, though!

[Aloso]

To quote RFC 3085:

The expectation is that the 2021 Edition will not be marketed as an "all encompassing event" in the same way as the 2018 Edition.

The reason for this is that many people were confused why features were advertised as part of the 2018 edition, even though they were available in edition 2015 as well.

I think it's better to clearly distinguish between an edition and the Rust version in which the edition was introduced. A marketing push does more harm than good if it is dishonest about the things it advertises. I believe this is the main reason why RFC 2052 ("Edition 2021 and beyond") was rejected.

[Soni]

(why not just call it a Buffer or UninitBuffer?)

[Qwaz]

One goal of edition 2018 was wrapping significant updates and advertising it to users who are not paying close attention to Rust. That is exactly what I thought is needed for ReadBuf update, and why I suggested to include ReadBuf in edition 2021.

Thanks for pointing out that the Rust core team is inclined towards taking a different approach this time. The "Rallying point" editions section of the RFC says:

There is a certain segment of potential Rust users that may be interested in Rust but not interested enough to follow along with every release and track what has changed. For those users, a blog post that lays out all the exciting things that have happened since Rust 2018 could be enough to persuade them to give Rust a try. We can address this by releasing a retrospective that goes over the last few years of progress. We don't have to tie this retrospective to the edition, however, and as such it is not described in this RFC.

What I probably want is then to include ReadBuf as part of this retrospective post so that it gets necessary attention.

[josh]

I would love to see ReadBuf stabilized, and then mentioned in several places.

Tracking Issue for RFC 2930 (read-buf)

[djc]

I'm pretty excited about the improvements that ReadBuf can bring, but I'm also still wondering if can slightly generalize is to close another safety gap.

My observation is that Vec<T> is analogous to a ReadBuf in that it consists of a filled part (len) and an uninitialized part (capacity). However, if we wrap the Vec's backing byte array in a ReadBuf, we will still need to unsafely set the len for Vec. Would it be worth it to give the ReadBuf type some kind of type parameter that will allow it to update the backing store when the ReadBuf itself is dropped?

Or, going even further, should we extend ReadBuf to support types of data other than u8, especially as we see the work on safe transmutes come to fruition?

[Johannesd3]

I was also thinking about it, but it's not really analogous.

• ReadBuf: Filled -> Initialized -> Uninitialized
• Vec: Filled = Initialized -> Uninitialized (-> Unallocated)

I think connecting those two will just make it unnecessarily complicated.

On the other hand, I like the idea of a generic ReadBuf. For example, audio samples are often stored as 16- or 32-bit numbers.

[josh]

I don't think we should make ReadBuf typed. We could always introduce a generic version in the future if someone proposes and implements that, and we could make ReadBuf a type alias for it. But that shouldn't prevent us from stabilizing the current version, which has an established use case and implementation.