What do you guys think of rewarding developers for issues they resolve? This requires some more thought, and of course funding, but it is probably possible to do and it feels very much like the right direction to me.
I would be vary
3 Likes
Interesting story. Developers can break things on purpose to get paid for fixing them. To make this impossible we need to not pay developers for fixing bugs. We want to pay developers for developing working features, so that should be what earns a reward. A client should submit a set of tests which need to be passed to unlock a reward. This means of course that the software will only be as good as the tests are. But if we had some way of verifying that the submitted code meets the requirements, we could unlock the reward and could keep bugs from being built in. Someone could still make a request to fix a specific bug, but that requires this fix to be funded, so that is no way to make easy money either. As long as the initial tests are good enough to test for potential bugs, the developers will have a hard time hiding bugs to profit from later on.
2 Likes
In general, I think this is the logical next step after Rust in software development. We now have reliable code, thanks to the Rust language. Next thing we need is expressive, complete testing, to make feature development accessible and effective. People should be able to go from "I want this program to do x when I do y" to a complete automated testing suite which can be used as a target for developers. Then these feature requests can be funded and the reward goes to the developers.
I've moved this discussion to its own post, as it's a tangent off the foundation discussion.
1 Like
I haven't ever seen this successfully work before. That doesn't mean it can't work, but it's an unproven model.
4 Likes
The nash-io/openlimits project gives crypto currency rewards for completing issues. Maintainers tag issues based on their size to determine their reward. I have worked on this project before and it felt like the perfect work model for open source.
The reason it has not proven itself yet may be because it is just now becoming viable with crypto currencies and thriving open source community.
One wouldn't even have to imply malice for this to go terribly wrong.
Even if developers don't purposefully break stuff just to fix it later, incentivizing a higher number of PRs could result in people wanting to push and merge half-baked solutions, doing it for the money rather than for the sake of a proper fix. This would in turn result in a generally lower-quality codebase.
6 Likes
Maybe there could be a bounty on solving specific, carefully selected, issues? Perhaps ones that have been languishing for a long time. Or for some much needed spring cleaning.
2 Likes
Code review and testing is still possible, I don't see any issue here. The solutions will only be as good as the requirements are, so all that is needed for this to work are good tests, which guarantee a degree of quality.
I feel like doing this broadly would have negative consequences, especially around the edges -- What things qualify? Why only PRs, not other work? Which repos? Etc.
That said, paying people to work on specific things is of course totally fine. If, say, a company wants to sponsor someone to get a new target working, that doesn't have nearly the same kinds of problems. All the details of defining success and such can be worked out as normal for a contract.
7 Likes
Sure, but it's more difficult if there's more noise.
Paying piece rates for anything means that the people doing the work can't treat it as a stable source of income. I think it would be better to pay people to work on Rust for some predictable number of hours a week.
Paying for pull requests also prioritizes "scratch this specific itch" coding over everything else; developers often find that more enjoyable than most any other sort of development, and (not coincidentally) I have the impression the project is not starved for that kind of people-power. I think it would be better to pay for the more tedious and thankless parts of keeping the project going -- debugging, testing, hammering out change proposals, that sort of thing.
22 Likes
This is a new dynamic with the reviewer when that process gets in the way of the author being paid, especially when the reviewer's honest opinion might be to reject the PR altogether.
15 Likes
I think this is an interesting thought exercise but one that is also rife with footguns if not done correctly.
On one hand paying people for work done is always a good thing. Paying people for contributions could be a way to open up contribution to a wider group of people that may not be able to afford to do something that would take that amount of time for free.
On the other hand, like others have pointed out, adding money into the equation increases the potential for abuse of the system as well. Increasing the amount of low quality PR's and thusly increasing the load on maintainers and others doing code review which I think would end up as a net negative.
There could be some sort of middle ground here maybe one that's not even really related to PR's themselves. What if we paid for things that are traditionally less desirable to work on in open source? Things like improving documentation and tutorials, doing operational work for services run by the Rust team (i.e. crates.io). An arrangement like that could work while mitigating the pitfalls possible from extra PR noise motivated by money.
3 Likes
Purposefully writing bad docs and improving on it several times to get more money
Just to make it more clear what I mean, let me give some examples. I imagine the community making requests for changes, and then funding those requests. These requests define how the reward will be unlocked, either by a predefined automatic test suite or by a set of people verifying the solutions manually:
Task: "Add documentation to Server type" // just a title
Reward: 5 tokens [Fund this task] // current reward and a button to add funding
Requirements: // subtasks
- Explain methods with examples (3 tokens) // reward is distributed over subtasks
- add links to related types (1 token)
- ...
Verifiers:
- John Doe [link]
- Test: "All methods provide examples"
These requests could be made by anyone, and could be forked and changed by the community to improve them. They should be thought of as goals. Anyone can add funds to a task at their own accord. If they are unconvinced by the current state of the task, because the verifiers are incomplete or untrusted, they can fork the task, make changes and fund that version. Tasks can also depend on each other as subtasks, which can be funded and completed individually.
There are a lot of advantages aswell as a lot of problems with a system like this. Advantages I can think of:
- Incentives to contribute to projects: money
- Incentives to formalize goals publicly: developers will work to receive rewards
- More involvement of non-developers
Problems so far:
- Developers can deliver minimal solutions to provoke more tasks and funding
- Reviewing many solutions requires more work
- solutions may be stolen and developers may be cheated for their reward
I am not sure how to circumvent all of these problems now. For the 1. problem I would be fine with requiring the feature request to have very specific quality requirements, like automated tests. However this means that maintaining a request will require even more effort, which adds to the 2nd problem.
And the 3rd problem seems very difficult to solve. Feature requests could simply use a solution without rewarding the developer, because they can steal it when reviewing it. I think this could be prevented by exactly tracking which account submitted which content to the platform and which content is merged into projects, this basically boils down to checking for plagiarism, and people could even accuse maintainers for not rewarding their solutions, but this will always be a risk to deal with.
I still think it is an idea worth following and over time I think it can be possible to flatten out mitigations to these problems.
Hi, my feeling is people
- are not opposed to such mechanisms in principle
- are opposed to Rust Foundation/Rust core team(s) managing them
This seems fair because if
- a 3rd party sets this up and screws up - no big deal
- Rust Foundation sets this up and screws up - this may alienate key people and kill Rust
If you can work on setting this mechanism up outside of "core rust" - power to you 
There's nothing stopping anybody from offering bounties for Rust features right now.
5 Likes
I recently watched this video about reward systems backfiring. Might be interesting to some people.
4 Likes
I've seen this used: https://issuehunt.io/ It allows adding a "bounty" to issues to fix.
I'm not sure how helpful this is. Is it supposed to be profitable enough to make a living? In that case, it looks kinda like "gig economy". Software development is a pretty well-paid profession in general, so the bounties would need to be significant to be competitive.
1 Like