Setting our vision for the 2017 cycle

[steveklabnik]

@brson

It's going to be hardest for the docs and community teams, but I'm glad you've already got ideas here that serve them, and I hope they will chime in and help identify what they think is important.

So, I haven't spoken with the docs team about all of this yet, given that it was just posted, but here's my own personal thought:

For the docs team, I would hope that we can manage to switch from reactive to proactive.

What do I mean by this? Well, ever since I came on board, I've been in 'reactive mode.' "Here's the language and standard library, get to work." Consider it like a consumer/producer problem:

The "producer" are people working on things: the language, libraries, etc. Once they've produced their code, it goes into the buffer. Then, the consumer is writing the docs: something gets popped off the queue, worked on, and documented.

The problem is, with many more producers than consumers, the buffer fills up. This leads to bugs like Ensure accepted RFCs are all in the manual and elsewhere · Issue #20137 · rust-lang/rust · GitHub, and you can see what happened there.

So, what to do? Well, now that it's not just me, I'm hoping that we as a group can finish off the backlog in the buffer. Once we do, we can stop fighting fires and start taking a more pro-active stance on docs. Some of that is tied into all the other goals, for example. So like, let's take one of them: FFI. We could have a whole little mini-book on FFI and Rust, and how to make it good. But I don't have time for that; the standard library still has functions with no docs. This would be the "proactive" mode: we're focusing on what docs the ecosystem needs to have, rather than just catching up on a huge backlog.

The next level is getting rid of the producer/consumer issue entirely: @chriskrycho opened RFC: Require documentation for all new features. by chriskrycho · Pull Request #1636 · rust-lang/rfcs · GitHub , which basically says "nothing can ever go into the buffer". In order to manage that, we as a team will have to help others help us, by getting more people to write docs, even if they don't usually write them. We want to keep high standards, yet not let that block actually shipping.

---

Now, for some replies:

---

@mgattozzi

things like #[no_std] don't work

Could you elaborate on what you mean here? What doesn't work about it?

@cbreeden

And if so, do you have ideas for how to approach such a problem? I would be interested in possibly helping with such a project if there are some good ideas.

I think this post by @wycats is a big part of it:

@wycats

I've been thinking that for advanced programmers, we really need a few different guides that take advantage of what you already know. For example, a Ruby programmer is going to come to Rust with quite a different perspective than a Java programmer, who comes to Rust with a different perspective than C++.

@leonardo

It's not easy and not short.

One argument passed for the hasher isn't short enough? What would be your ideal syntax here?

[mgattozzi]

I think this post covers a lot of the problems one runs into when turning off std. I would consider making an OS as bare metal as you could get and something you would run into when working on embdeded development systems which have no OS. One shouldn’t have to use things like ralloc just to get it to work. These are the kinds of problems I meant with workarounds. Like you can use no_std and there are solutions but they’re not baked into Rust without finagling. http://os.phil-opp.com/set-up-rust.html

[eternaleye]

@djc

One example: the Windows docs for std are currently not available online, because rustdoc will only build the docs for the current platform (or really just the currently applicable set of conditional compilation conditions). How are Windows/OS X users supposed to learn std in this situation? Another: how to document module items whose definition depends on conditional compilation stuff? Do I have to copy docs for each platform?

One thing I wonder, actually - how feasible would it be to (in essence) do "symbolic execution" of #[cfg] directives? Specifically, have rustdoc "branch" on encountering a #[cfg] directive, generate documentation under both, and generate fragments for each such section? That could then allow selecting from a menu of #[cfg] options when reading the docs.

In terms of phrasing this as a vision statement, "Users of any platform should be able to easily read the documentation relevant to a crate as they would actually use it."

[leonardo]

@steveklabnik

One argument passed for the hasher isn't short enough? What would be your ideal syntax here?

Do you have a complete correct up-to-date usage example somewhere on the Rust site or on the internet? Just showing the first a step and asking "to use the desired hasher instead" isn't enough, in my opinion. In the page I'd like to see a complete example of fast hashing of a simple struct (like a struct { a: char, b: f64 }). Once the example is complete, I think the resulting code is not short and not simple.

[dc_coder_84]

I am not a Rust programmer. Only knowing some C++. Not sure if my idea makes any sense…

I have heard regarding the question: “Can Rust do OOP?”. The answer seems to be yes and no. There is this classic book “Design Patterns” from the gang of four. My idea is a guide that says when you want to implement a certain design pattern (from this book or other newer design patterns) what is the best way in Rust to implement it? When it is not possible to implement it then what should be done in this case to achieve the same good code quality or the same goals of a design pattern? The guide should at least cover the design patterns in the book mentioned. The more the better.

[jethrogb]

@mgattozzi

I think this post covers a lot of the problems one runs into when turning off std. I would consider making an OS as bare metal as you could get and something you would run into when working on embdeded development systems which have no OS. One shouldn't have to use things like ralloc just to get it to work. These are the kinds of problems I meant with workarounds. Like you can use no_std and there are solutions but they're not baked into Rust without finagling.Redirect

I assume you're specifically referring to the "fixing linker errors" section? That seems to me to be a totally reasonable thing when you're working bare-metal. In my experience, regardless of whether you're working in C, assembly or Rust, you always need some boilerplate code and linker scripts when doing OS dev.

[cuviper]

@aturon

[Other possibilities, or potential non-goals]
Adding compiler/platform targets

I would love to see firmer support for the existing non-x86 architectures. I feel a bit guilty that I'm only building i686 and x86_64 for Fedora so far, but these are the only Tier 1 targets. Perhaps I should bite the Tier 2 bullet for aarch64 and armv7, but the rest don't have bootstrap binaries for rustc and cargo, especially ppc64 and ppc64le I'd hope for next.

[mgattozzi]

I can see that. I guess it would just be nice to have it just work depending on the target platform without having to do the extra bit of linking. I.E. if compiling for arduino using no std rustc knows what to do without having to do that extra work. Making it more accessible to people who just want to code on a system without having to figure out how to configure it properly could go a long way towards adoption of Rust in that ecosystem.

[kornel]

In terms of compiler errors, currently linker errors are the weakest. It’s just “we ran this unreadable long command and something didn’t work”.

I know it’s probably hard to fix, and it’s not rustc, but the rest of the toolchain, but still — these errors happen, and it’s often hard to figure out why.

[jethrogb]

@kornel

In terms of compiler errors, currently linker errors are the weakest. It's just "we ran this unreadable long command and something didn't work".

I know it's probably hard to fix, and it's not rustc, but the rest of the toolchain, but still — these errors happen, and it's often hard to figure out why.

I don't know if the linker output is unambiguously parseable or if there is some way to output something machine-readable, but it would be very helpful if rustc would tell you which specific crates have unresolved symbols.

[arthurprs]

But that’s implementing a new hasher, not using a different hasher. I can argue that people with enough know-how to implement a hash function themselves are completely capable of implementing rust hasher interface.

[eternaleye]

@jethrogb

I don't know if the linker output is unambiguously parseable

Regrettably, no

[quote="jethrogb, post:31, topic:3958"]or if there is some way to output something machine-readable, but it would be very helpful if rustc would tell you which specific crates have unresolved symbols. [/quote]

Probably the most promising approach would be using LLD from the LLVM project as a library, but that's going to be blocked on:

1. LLD maturing significantly
2. LLD growing a C API, rather than just C++
3. Rust bindings for that C API
4. LLD supporting all of Rust's platforms

[comex]

I brought this up in a different thread, but I think ‘insecure but fast hashing’ is a red herring in most cases. Instead, the default HashMap should use an adaptive scheme that starts with an insecure hash and switches to a secure one if an attack is detected (based on load factor and maximum distance of entries from their ideal location). That should provide the best of both worlds except if code size needs to be minimized, which is not the reason most people seem to want to use insecure hashers.

[comex]

@eternaleye

Regrettably, no

Why not?

[anp]

I see many specific, granular issues/complaints/problems here. But I see very few attempts to translate those into high-level goals that could actually be set, planned, and executed over the course of an entire year.

Might it be more productive for this conversation if each specific problem or issue is “translated” into the high-level goal it’s related to?

An attempt to summarize high-level items I see in the thread so far that aren’t in @aturon’s original post:

• Rust needs better answers (either guides, outreach, or new features) for teams & programmers coming from OOP – this seems very related to “Rust should have a lower learning curve” to me, but could be distinct if it covers type system features for inheritance?
• Rust’s ownership system needs to aid in modelling more complex systems (i.e. parent/child communication)
• Rust’s tooling needs to be first-class to aid adoption (beyond IDE support, i.e. rustdoc)
• Rust’s community and processes need better visibility to “casual” participants and newcomers (e.g. RFC visibility and dashboards, also casual is a bad word for this, but I can’t think of a better one here)
• Rust needs a better story for embedded development (re: no_std)
• Rust’s support for compile-time code generation should be improved and stabilized, especially for those uses that are already encouraging many users to stay on nightly (custom_derive in particular)
• Rust needs better testing and doc generation across conditional compilation directives
• Rust needs better publicly available information and guides about improving performance for common programs (i.e. hashing), or it needs to refocus its efforts on providing fast-by-default tools in std and common crates
• Rust needs to define and execute better processes for improving documentation and guides across the ecosystem, but especially for core projects
• Rust needs cleaner integration with the system toolchains it depends on

I have some of my own thoughts, too, so I’ll be back :).

[eternaleye]

@comex

Instead, the default HashMap should use an adaptive scheme that starts with an insecure hash and switches to a secure one if an attack is detected (based on load factor and maximum distance of entries from their ideal location).

Hm, maybe something that (during resize) measures both the maximum load encountered while resizing in both the new and old backing vector, and upgrades if it does not decrease sufficiently? That way, you avoid having any added code in the hot paths of insert and lookup.

@comex

@eternaleye

Regrettably, no

Why not?

Well, for one, "which linker?"

Invoking the linker isn't especially bad, as there are basically only two options in terms of "how to invoke it" (ld vs. link.exe), and which one to use is clear from the target. However, if you're going to parse the output, you're now going to get tripped up by different output from ld.bfd vs. ld.gold vs. different versions of same vs...

[brettcannon]

@aturon

Rust's should provide a more seamless FFI story

An essential adoption vector for Rust is to use it for building components in a larger system. For this use-case to really sing, we need a seamless "FFI" experience. The meaning of "FFI" here is potentially quite broad—from making it trivial to bind unsafe C APIs (#include somelib.h for Rust), to C++ integration, to embedding in languages like Ruby, Python or JS, to compiling to wasm. We've been making progress on all of these fronts, but to make an even bigger impact, we could consider choosing a small number of such scenarios to really focus on as a community this year.

I can confidently say that if you improved the FFI story so that it was easy to use with Python then you would garner a lot of support and users from the Python community. For the past two years I have constantly heard rumblings from people at PyCon about how they would give Rust a look if they could write Python extensions in it (and I'm guilty of wanting this myself). And if you managed to make it work with something like Cython or CFFI so you could also work with PyPy then Rust would get even more mileage out of its Python story.

[eternaleye]

@brettcannon

And if you managed to make it work with something like Cython

Cython example: GitHub - synapticarbors/rust-cython-test: Rudimentary example of calling rust from cython

The header (which the example hand-writes) can be autogenerated using rusty-cheddar

@brettcannon

or CFFI

CFFI example: Calling Rust from Python/PyPy using CFFI (C Foreign Function Interface) · GitHub

[comex]

I suspect that in practice there aren’t that many different variants of error message that commonly used versions of commonly used linkers output in common cases, and if the format isn’t recognized you can always fall back on showing the user the original spew. It’s definitely a hacky approach, but that’s different from unworkable

[burntsushi]

Another example: https://github.com/jbaiter/python-rust-fst

(It even has CI setup and everything to build wheels.)