I was recently thinking about how several soundness violations have been found during the run-up to the 1.0 release, both in the language and in the standard library. How many more will turn up after? How could we have more confidence that there are unlikely to be further holes lurking?
One way to do so, of course, would be to prove soundness, as e.g. the Patina effort has begun to do. But doing this for the whole language and all of the unsafe code in std is likely to be a considerably large and lengthy undertaking (especially as both continue to grow and evolve).
The violations found so far have tended to turn up in a seemingly haphazard and accidental way: in at least one case (but I think there were more), example code mentioned in an RFC discussion thread which surprisingly compiled turned out to be due to a soundness hole in the language.
One way that we could gain confidence would be if we knew that people were constantly and actively looking for holes, and not finding any. One potential way to accomplish that would be by posting bounties for any holes that people do manage to find, either in the language itself or in unsafe code in std. This would be not-dissimilar to the bounties which some other projects post for finding security holes in their software.
(Perhaps the bounty could be in some approximate proportion to the seriousness of the hole, e.g. larger for language than for library issues, larger in proportion to the amount of time the functionality has been stable, …?)
Anyone else think something like this might be worthwhile?
I’ve thought about idly this from time to time; I think it’s a pretty great idea. @nikomatsakis@brson@wycats: seems like the kind of thing you all might have some thoughts/opinions on.
I don’t have much experience in this area but we’ve certainly talked about it from to time. It seems like a good idea. (We’d have to budget for it, obviously.) I’d be curious to see a survey of relevant precedents (I can think of Knuth of course ) to get some idea how this is frequently done.
I think those rules need to be carefully designed. This exact rule would most likely cause people to sit on the holes they find until the bounty reaches an "acceptable" amount even though game-theory says to cash out immediately.
I would presumably be sufficient to make stable == stable_but_in_beta for rewards, to prevent bad things from landing (assuming we want any kind of tiers). People might stay mum about unstable functionality, but presumably that’s “fine” as far as our ultimate goals are concerned (keep bugs out of stable).
It’s perhaps worth noting that all the soundness violations we’ve found that I’m aware of have been quite dubious to exploit. Generally they involve things that reasonable code would Almost Surely not do (lots of mem::forget stuff), and even then require special conditions to actually exploit in a reliable way. That is to say, they require on highly specific code structures, rather than bad data.
Two exceptions stand out, I think.
The one where some messed up bounds meant HashMap was making 'statics meant a program’s design could silently go off the rails pretty fast, creating massively exploitable code.
The 32-bit Vec-capacity-overflows-isize exploit (which persists to this day!) is theoretically doable with anything that takes an input, but is obviously delicate as you need to be able to successfully coerce the target code to request >2 GB contiguously on a 32-bit system.
Regardless, bug bounty programs have… weird… economic and social implications. Definitely a tread-lightly scenario.
Yes I was kind of thinking: What motivates current and future rust contributors best?
What would motivate me would probably be social rewards (recognition) and participation. And what would excite me personally right now would be to have a clear plan for soundness violations and something that ensures a quick process for assessment, mitigation and fix. Rust is an exciting language, the goal of safety is quite ambitious. Simply making that work ever-present would (hopefully) bring an air of success to the community.
I have a few thoughts on this as I work on an Infosec Team and have to deal with bug bounties as part of my day-to-day.
First, I think this is a great idea, as Rust’s “soundness”/safety has security-critical implications.
I’d note that generally with any sort of bug bounty program, the organizers of the bug bounty have total and final say as to whether participants are paid. I hope you wouldn’t be dissuaded from enacting a bounty program like this for fear that you’d have to pay out over inconsequential issues. That’s part of the rigmarole of having a process like this.
There’s definitely administrative overhead in terms of evaluating claims, as by opening up such a program you will get irrelevant spam submissions you’ll have to deal with. Companies like HackerOne and BugCrowd can help triage this spam for you, so hopefully people reviewing bugs only have to deal with high-signal issues.
Maybe start with just giving more public recognition (prominent mention in release announcements and a page with the history of all such bugs and people who found them), rather than awarding money.
I think many projects do this. Just have a special page for them (e.g. Stack Overflow) and give 'em a special mention in the next announcement. Next up is the tshirt + mention, after that there are cash prizes. Money can be problematic though as @Gankra mentioned.
Mozilla already has a bug bounty program that could be useful for inspiration.
Tarsnap’s bounty program is also well-known. One thing they do is award small bounties for non-security bugs, to encourage people to start looking.
Projects have got into trouble before for cheaping out on prizes. Remember that an exploit for widely used software can be sold to organized crime or authoritarian governments (but I repeat myself, eh?) for six figures USD. Granted, it’s not clear what the market will make of a compiler soundness hole, because it doesn’t a priori imply an exploitable hole in any actual program. But they sure could be useful for introducing bugdoors
Wouln’t it be more constructive to have bounties for proving unsafe code correct? Make a list of the unsafe parts of the standard library, sort them by importance and size, and assign some bounties to them. On the other hand, that would divide work on rust in a paid and a non-paid part, which might demotivate people to contribute for free.
) to get some idea how this is frequently done.