Setting our vision for the 2017 cycle

[arthurprs]

But that’s implementing a new hasher, not using a different hasher. I can argue that people with enough know-how to implement a hash function themselves are completely capable of implementing rust hasher interface.

[eternaleye]

@jethrogb

I don't know if the linker output is unambiguously parseable

Regrettably, no

[quote="jethrogb, post:31, topic:3958"]or if there is some way to output something machine-readable, but it would be very helpful if rustc would tell you which specific crates have unresolved symbols. [/quote]

Probably the most promising approach would be using LLD from the LLVM project as a library, but that's going to be blocked on:

1. LLD maturing significantly
2. LLD growing a C API, rather than just C++
3. Rust bindings for that C API
4. LLD supporting all of Rust's platforms

[comex]

I brought this up in a different thread, but I think ‘insecure but fast hashing’ is a red herring in most cases. Instead, the default HashMap should use an adaptive scheme that starts with an insecure hash and switches to a secure one if an attack is detected (based on load factor and maximum distance of entries from their ideal location). That should provide the best of both worlds except if code size needs to be minimized, which is not the reason most people seem to want to use insecure hashers.

[comex]

Regrettably, no

Why not?

[anp]

I see many specific, granular issues/complaints/problems here. But I see very few attempts to translate those into high-level goals that could actually be set, planned, and executed over the course of an entire year.

Might it be more productive for this conversation if each specific problem or issue is “translated” into the high-level goal it’s related to?

An attempt to summarize high-level items I see in the thread so far that aren’t in @aturon’s original post:

• Rust needs better answers (either guides, outreach, or new features) for teams & programmers coming from OOP – this seems very related to “Rust should have a lower learning curve” to me, but could be distinct if it covers type system features for inheritance?
• Rust’s ownership system needs to aid in modelling more complex systems (i.e. parent/child communication)
• Rust’s tooling needs to be first-class to aid adoption (beyond IDE support, i.e. rustdoc)
• Rust’s community and processes need better visibility to “casual” participants and newcomers (e.g. RFC visibility and dashboards, also casual is a bad word for this, but I can’t think of a better one here)
• Rust needs a better story for embedded development (re: no_std)
• Rust’s support for compile-time code generation should be improved and stabilized, especially for those uses that are already encouraging many users to stay on nightly (custom_derive in particular)
• Rust needs better testing and doc generation across conditional compilation directives
• Rust needs better publicly available information and guides about improving performance for common programs (i.e. hashing), or it needs to refocus its efforts on providing fast-by-default tools in std and common crates
• Rust needs to define and execute better processes for improving documentation and guides across the ecosystem, but especially for core projects
• Rust needs cleaner integration with the system toolchains it depends on

I have some of my own thoughts, too, so I’ll be back :).

[eternaleye]

Instead, the default HashMap should use an adaptive scheme that starts with an insecure hash and switches to a secure one if an attack is detected (based on load factor and maximum distance of entries from their ideal location).

Hm, maybe something that (during resize) measures both the maximum load encountered while resizing in both the new and old backing vector, and upgrades if it does not decrease sufficiently? That way, you avoid having any added code in the hot paths of insert and lookup.

Regrettably, no

Why not?

Well, for one, “which linker?”

Invoking the linker isn’t especially bad, as there are basically only two options in terms of “how to invoke it” (ld vs. link.exe), and which one to use is clear from the target. However, if you’re going to parse the output, you’re now going to get tripped up by different output from ld.bfd vs. ld.gold vs. different versions of same vs…

[brettcannon]

Rust’s should provide a more seamless FFI story

An essential adoption vector for Rust is to use it for building components in a larger system. For this use-case to really sing, we need a seamless “FFI” experience. The meaning of “FFI” here is potentially quite broad—from making it trivial to bind unsafe C APIs (#include somelib.h for Rust), to C++ integration, to embedding in languages like Ruby, Python or JS, to compiling to wasm. We’ve been making progress on all of these fronts, but to make an even bigger impact, we could consider choosing a small number of such scenarios to really focus on as a community this year.

I can confidently say that if you improved the FFI story so that it was easy to use with Python then you would garner a lot of support and users from the Python community. For the past two years I have constantly heard rumblings from people at PyCon about how they would give Rust a look if they could write Python extensions in it (and I’m guilty of wanting this myself). And if you managed to make it work with something like Cython or CFFI so you could also work with PyPy then Rust would get even more mileage out of its Python story.

[eternaleye]

And if you managed to make it work with something like Cython

Cython example: https://github.com/synapticarbors/rust-cython-test

The header (which the example hand-writes) can be autogenerated using rusty-cheddar

or CFFI

CFFI example: https://gist.github.com/seanjensengrey/f5d73bbdf22cfa1ad463

[comex]

I suspect that in practice there aren’t that many different variants of error message that commonly used versions of commonly used linkers output in common cases, and if the format isn’t recognized you can always fall back on showing the user the original spew. It’s definitely a hacky approach, but that’s different from unworkable

[burntsushi]

Another example: https://github.com/jbaiter/python-rust-fst

(It even has CI setup and everything to build wheels.)

[brettcannon]

Very nice! It would be neat to try and work in struct/enum support somehow so that there was corresponding niceties on the Rust side as well (e.g. exposing the Python code object in such a way that methods that exist in Python are also exposed in Rust).

[nikomatsakis]

This is great. It’d be nice if you linked to the original comments for context too

EDIT: I know, I know, “give 'em an inch, they want a mile”

[Eh2406]

This is a grate https://github.com/dgrunwald/rust-cpython project! But it has no windows support https://github.com/dgrunwald/rust-cpython/issues/10 becust rustc dose not “dllimport on Windows/MSVC”

Rust <–> Python can use some love!

[zepto]

Wondering if a runtime for embedded systems cropped up as a suggestion? Seems to be a great use case for rust given the memory safety, etc. For sure I’d like to sketch for arduino/genuino in rust.

[anp]

No problem. I’ll update it in a few hours and try to keep up with new comments.

[aturon]

If the current order of vision statement implies prioritization

I should’ve been explicit about this. The order does roughly reflect the sense one gets from the survey results, but it’s rough at best. I’m not sure whether a ranking of the final vision statements is a good idea, vs trying to scope them so that it’s feasible to get all of their goals done in a year.

[gnzlbg]

uncompromising performance

I cannot use Rust in production because its abstraction capabilities are not powerful enough to tackle the problems that arise in my domain.

My application domain is, ironically, high performance computing (numerical simulations). The main piece missing is type level values, but there are also others.

I need type level values to abstract algorithms and data structures over the spatial dimensions of whatever problem I am trying to solve, such that I only have to implement them once. Quad/Octrees, BVHs, structured/unstructured meshes of simplices or hexahedras generalize to arbitrary spatial dimensions. Most numerical methods for PDEs (finite differences/volumes/elements, DG, Lattice Boltzmann) do so as well.

Type level values also would allow better fundamental libraries for, e.g., linear algebra.

But with linear algebra we come to the second piece missing: better support for EDSLs. EDSLs make libraries easy to use, but they are extremely hard to implement in Rust (harder than in C++ and not as powerful). Anything that improves Rust EDSL story would help us implement the libraries that we need. From the top of my head, procedural macros, (way) better CTFE, variadics, and maybe HKTs would help.

There are also some issues with FFI integration. HPC code relies heavily on MPI, whose standard define some macros, that might have different types depending on the implementation used. This makes writing a rust wrapper around MPI a bit of a hassle because those macros have to be dealt with at the build system level, instead of within Rust itself.

Finally, most HPC environments don’t have direct access to the internet. This means no rustup.rs, no cargo, no nothing. It would be great if there were ways to use these tools from remote systems, e.g., via ssh tunnels, or if there was an easy way to prepare the environment for a crate in some system, and ship it easily to another one (where everything gets recompiled).

So while I would love to use Rust in production for high performance work, the language currently doesn’t allows us to abstract at the level that we need to do our job efficiently. IDE integration, better compile times, lower learning curve… all of these are great, but even if all of these are fixed we would still be unable to use Rust in production.

For this reason I would like to see in the vision for 2017 how Rust plans to expand to those system level/low-level programming fields that it currently struggles at. My field cannot be the only one, so it would be nice to hear of others as well.

EDIT: I rephrased the post because its 1:30am here and the points I tried to make were even unclearer than they are now. EDIT 2: I think it is actually sad, that Rust concurrency is so good for its age, and that we have so many massively parallel problems (as well as access to so many cores), but cannot use it

[josh]

I’m glad to see FFI included in these goals, and in particular the mention of seamless #include support. In particular, that gives me hope for detecting library version issues at compile time rather than via runtime crashes. Right now, libfoo-sys crates declare the various C symbols they use, with no way of knowing if those declarations match those of the version of the library the crate compiles against.

[sciyoshi]

Like @Eh2406 said - the rust-cpython bindings are actually really solid. I’m working on a blog post ATM about how to write extension modules for Python in Rust. Still, I agree that the story there can be improved, especially if the work on custom garbage collectors makes progress and the interop with Python’s refcounting can be worked out. I’ll need to take a look at how that can work with Cython too.

[aturon]

the rust-cpython bindings are actually really solid.

This reminds me: one of our perpetual problems is just making good stuff that exists more visible. We’ve added a lot of links within the web site, but it ends up being totally overwhelming.

How can we make it easier for people to discover things like Rust’s Python story?