Rust 2018 All Hands!

[aturon]

Today is the beginning of the Rust 2018 All Hands in Berlin – a weeklong gathering of around 50 Rustaceans representing 15 teams and working groups!

An overview of the event can be found on this document, and we’ll be posting to this thread with updates throughout the week.

To start with, you can see the team goals for the week, or take a gander at the tracking spreadsheet for Rust 2018.

More soon!

[aturon]

I’ve posted my slides from the kickoff talk here. Hopefully we’ll get Niko’s slides up soon as well. I also believe both talks were recorded.

[mark-i-m]

Err… um… the wifi password is public knowledge… [cough] [cough] [shuffle]

[aturon]

Not the password, just the SSID

[aturon]

Monday recap

We spent the first half of the day with kickoff talks (my slides are above; Niko also gave a talk), introductions, and more. In the second half we had our first round of meetings. Here’s a summary of what got done:

Compiler clients

• Agreed on kinds of info that different clients need and the layers of stability involved
• Overall interface to rustc from external tools
• Should enable composable plugins (e.g. combining clippy with another static analysis)
• Builds on incremental query arch

WG: CLI

• Reviewed survey results
• Discussed overall WG strategy:
  • Focus on improving existing crates and making them discoverable
  • Based on the survey, this primarily means documentation + testing
  • Work backwards from the “CLI Guide” we plan to deliver for Rust 2018
  • Picked a few focus areas: input/output libs (color, logging, etc), testing, getting started guide for CLI tools

Community

• Mozilla Activate page — ready to ship tomorrow

Macros 2.0

• Selected a subset of “macros 2.0” to stabilize in Rust 2018
  • Called “macros 1.2” — custom attributes applied to items and proc-macro-!-macros in item position
  • No hygiene (coming later)
• Follow-up work for API details on proc macros
• Stabilize import of macros through the module system

Verification

• Bringing together testing and verification work — what’s needed here, how can we coordinate?
• How can we get a foothold for verification

[jimmycuadra]

I wish I was there, if only to be a fly on the wall. Sounds like a lot of fun!

This might be the wrong place to ask, but…

Some folks might have seen my half-kidding tweet earlier, but I actually would like to know what the plans are for being able to return an impl Trait type from a trait method (RFC 2071). Is the plan for that to be included as part of the 2018 era? I’m hearing conflicting information about it being postponed into the distant future which is disappointing and somewhat surprising to me, as the main project I’d want impl Trait for needs it in traits!

[aturon]

impl Trait in traits is very high priority, but it’s not feasible to ship in the main 2018 edition release (targeted for September). We will certainly see it implemented in 2018, and possibly stabilized this year as well.

[aturon]

Here’s a link to @killercup’s excellent presentation on the CLI WG this morning.

[aturon]

Tuesday recap

@killercup’s CLI presentation

Infra

Rust All Hands 2018 - Infra Team: Meeting-notes

• Talked about the bors queue today
• Came up with a 3 part plan for making sure that CI is ready to support the 2018 edition work
  • Do more testing on PRs prior to r+ (right now we do just one builder, we want to test on a few more platforms)
  • A couple ideas for addressing the bors timeout issue
  • Want to improve rollups so that they don’t invert priorities

WG: CLI

Notes

• Crates.io: Distribution Friendly Badge
  • Lint Cargo.toml for dependency declarations that are friendly to distribution
  • Check primary as well as transitive dependencies
• Extensible Cargo Install RFC

Unsafe code guidelines

• Reviewed various models and their pros (none) and cons (many)
• Came to a seemingly workable “access-based” model suitable for experimentation
• Blew a few minds

WG: embedded

• Embedded Rust on stable
  • Three main blockers
    • compiler-builtins - two possible solutions - https://github.com/rust-lang/rust/issues/49380
    • ~Xargo~ rust-std component - OK from Alex - https://github.com/rust-lang/rust/issues/49382
    • panic-fmt - compiler team will discuss what’s the best solution
  • P-med unstable features
    • asm! - leaning towards core::asm::$arch module with intrinsics - https://github.com/rust-lang-nursery/embedded-wg/issues/63
    • const-fn - oli-obk will move to FCP
  • P-low unstable features
    • #[used] move to FCP (?)
  • CI Testing in rust-lang/rust
    • OK from Alex. Will come up exact proposal about what the exact tests to add
• Ship llvm-binutils with the toolchain
  • OK from Alex. Will open issue to discuss details
• embedded Rust book
  • Initial audience has been defined (knows Rust and knows embedded)
  • Came up with high level outline

Libs

• Stabilizing SIMD — we hope to stabilize in 1.27
• Custom global allocators (not collections)
  • Have a plan for stabilization, mostly by punting the collection questions to a separate crate
• Talked through an assortment of unstable API issues

Compiler

• Continued consumer discussion, exploring long-running compiler process details:
  • how to handle arena
  • how to improve cost of serialization/deserialization
  • what query API looks like in more detail
• Explored end-to-end query compilation:
  • Good news: Found a workable plan
  • Bad news: Seems like a lot of work, includes a number of “monolithic” passes that wouldn’t be very incrementalizable (at first)
  • Conclusion: Seems like work we have to do, but is it the work we should do now?
    • Have to investigate benchmarks in more detail

Tools

• 2018 edition planning
  • Want the tools to keep working as we land changes for Rust 2018 elsewhere
  • rustfix is key for the edition release; we have a plan for making it happen
• clippy lint audit planning
  • want a 1.0 for Clippy this year
  • before that: need to audit the set of lints so that the defaults represent a consensus design
  • developed a plan for the audit
    • Categories: {correctness, performance, style, pedantry, restriction, nursery}
    • For each: deny/warn/allow by default
    • Possibly pull lints from rustc to clippy (and vice versa)
    • Probably going to result in an RFC

Verification

• agreed that there should be a common (but not exhaustive) specification language
  • and it should be a subset of Rust
• RustBelt goals
  • making semantic model more faithful to the language
  • expand memory model
• Figure out what tools need in order to work
  • CFG
  • Types of all variables, function arguments, and returns + type definitions
  • Trait impl and inherent impl information for types
  • lifetime information
  • MIR
• TODO: is there any ‘low hanging fruit’ that we can look at for Servo?
• Discussed possibility of a verification domain team, possibly combined with testing? Logistics/details/actually happening tbd

[varkor]

• Blew a few minds

Intriguing and yet cruelly unelucidated.

[rayascott]

As long as it’s not motherboards.

[aturon]

Wednesday recap

Cargo: rustup and xargo integration

• High level story of what rustup integration will mean
  • TLDR: Cargo’s UI covers the 80% cases, rustup remains a separate tool
  • Not planned for 2018 edition
• Remove “target” notion from toolchains
• Bring xargo wholesale into Cargo
  • Also unsure about 2018 edition

Community (RustBridge)

• Work on RustBridge resources (in-a-box project) got done
  • Venue and Logistics Guide
  • Budget Guide
  • Coaching Tips
  • Brief Intro to Rust + exercism Teaching Modules
  • Outreach Template Letters to Universities and CoWorking Spaces

Docs

• figured out which resources needed to be updated and at what priority
  • also who is going to be responsible for tracking progress
• produced a list of resources that may be added to the bookshelf
  • ex, rustc guide, various WG docs that are being produced, etc
  • talked with CLI WG about what they’re doing in this regard
• divide up resources: Learning Rust, Using Rust, Mastering Rust
• “This week in Rust docs” is going to move to a new docs team blog https://rust-docs.github.io/blog/

Unsafe code guidelines

• discussed the finer points of how a union ought to be represented
  • Questions centered around possible optimizations, and what that means for representation invariants
  • Approached rough consensus on the choices here
• discussed cramertj’s proposed chain to Pin
  • Possible invariant to add to the Pin type
  • Still some open questions

Verification

• Formalized as an official WG (announcement forthcoming)
• What do we need to do to enable Rust verification work to proceed?
  • What might a spec language look like?
  • What might annotations look like?
  • How do we think about compilation and testing with these kinds of annotations?
• What’s compelling about a Rust verification story (vs e.g. ocaml)?
• Plan:
  • collect a corpus of example verifications we’d like to perform
  • develop APIs for the data that verifiers need

Infra

Rust All Hands 2018 - What can Infra do for you?

• Representatives (or out-of-band feedback) from embedded, compiler, tools and infra teams, along with individual feedback.
• List of ideas
  • Many hopes and dreams - some are already possible, some are Coming Soon
  • Bunch of low hanging fruit
  • A few cross-cutting concerns of interest across the board
  • Infra team to try and prioritise in the context of the bors plans etc
• Some ideas on advertising infra and attracting people

Web site

Sketched domain WG pitch pages and provided feedback

WG: embedded

• cross team meeting
  • clarified what the tier system means for embedded targets
  • decided on a solution for the compiler-builtins / core problem
  • talked about disabling i128, etc for AVR to workaround codegen bugs in LLVM and land the backend
  • Talked to the infra team about what kind of CI tests we need for embedded Rust
• lengthy discussion with oli-obk about interrupts and cooperative multitasking using generators

WG: wasm

• Talked about what we want to ship for Rust 2018
  • What goes in the book?
• How to enable production users of rust-wasm on npm
• Talked a lot about JS callbacks and Rust functions Lang
• triage’d Epoch 2018 plans, deferred a few things, settled on some release targets we … think … are realistic

“WG”: Codegen

• Goals: Working on code size improvements and code perf improvements (runtime perf)
• perf.rlo-esque site for tracking runtime perf
• Triage existing perf bugs
• Collect low hanging fruit
  
  https://twitter.com/tsion/status/978929677212569600

[aturon]

Thursday recap

• Talk about WG-wasm
• Talk about WG-net

Compiler

• Looked at compile times for the Stylo crate
  • Best way forward: generate less code for LLVM to build
    • MIR-only rlibs
    • parallelization in rustc

libsyntax2

• Design constraints for a new version of libsyntax
• No particular timeline here, but we do want to think about impact on proc macro stabilization

Cargo

• Custom registries: basically ready to stabilize
• Profiles: v2 design is clear
  • simplify down to Dev and Release for the 2018 edition
  • move out things like rpath, which are more workstation-relevant
• Public deps
  • Major change of plans here:
    • Improve rustc errors on incompatible types from different versions of the same crate (e.g. serde(1.0)::Serialize
    • Allow crate authors to label deps as private, which enables a lint
    • Change resolution heuristics to favor fewer duplicates where possible
• Build system integration
  • Analyzed RLS needs
  • Let’s add a minimal notion of build plans, even if they just provide rustc invocations with command-line arguments. We can iterate later about how to provide higher-level metadata rather than just rustc command lines.
  • Let’s add a minimal implementation of metabuild, and we can iterate later on how to implement the individual build crates in the ecosystem (e.g. parsing Cargo.toml or finding native libraries), as well as on better interfaces to those build crates.

WG: wasm

• How to reduce code size bloat around panics
  • Worked through a bunch of wild ideas
  • Ended up working through a simple case
    • Started at 44k, just to panic
    • Now: 350 bytes
• wasm-bindgen now works with commonjs (i.e. in node)

Unsafe code guidelines

• Made a plan to draft a reference about the questions around unsafe code, and what is known and unknown about their answers
  • thread::abort
  • Places where we’re not sure yet — you shouldn’t do them, but others might
  • Will help organize future investigations as well
• Statics, consts, and Sync
  • Came up with a bug fix that may be needed for soundness
• Drilled down into a potential model for aliasing
• Then Ralf and Julian holed themselves away for hours and hopefully Thought Deep Thoughts
  • Focused mainly on valgrind integration

WG: net

Five things we plan to do:

1. Hasty: small blocking web server, intended for learning when you’re new to Rust
2. Tyger: stable API over hyper providing the interfaces we need; want to reach 1.0 for the 2018 edition
3. Build middleware based on Tower’s interfaces
4. Improve debugging for futures based on a combination of backtraces (for async/await) and explicit bread crumbs
5. Documentation

• Async Programming in Rust book
• Hasty guide
• Middleware construction guide

Tools (custom test frameworks)

• we have an eRFC open for a while: https://github.com/rust-lang/rfcs/pull/2318
  • nearly in FCP
  • worked through and reached consensus on a lot of the details
• discussed procedural macro design for writing frameworks
  • whole crate proc macros
  • helper crates to get people started
• Cargo integration is basically settled
• Common test output crate will begin as “what libtest does” as a crate, get own RFC later on
• see Rust 2018 All Hands — Custom Test Frameworks for details

Docs

• Publicized doxidize: https://github.com/steveklabnik/doxidize
  • TL;DR: like https://docusaurus.io/ but for Rust. Markdown > doc comments
  • got good feedback, we will explore various parameters/possibilities in the future
• Talked about “intermediate” docs
  • got some good ideas from people: https://github.com/rust-docs/team/issues/8

HolyJit

• 99 bottles of beer working in the JIT-ed brainfuck code.
• eddyb is enthusiastic! (Mission complete)

[mark-i-m]

How does doxidize fit into the rustdoc rewrite? Is it a replacement or an extra tool?

[ErichDonGubler]

Looks like it’s meant to fit the “intermediate documentation” use case – here is a dogfooded documentation site for doxidize. Wait…AND it has functionality that rustdoc has? I’m not sure even AFTER looking. It seems like it wants to subsume rustdoc and more…

[jethrogb]

Remove “target” notion from toolchains

Can you elaborate on this? Maybe @japaric?

[seanmonstar]

Tyger: stable API over hyper providing the interfaces we need; want to reach 1.0 for the 2018 edition

o_O Oh really? What’s this?

[crlf0710]

withoutboats/tyger

[aturon]

o_O Oh really? What’s this?

I believe we had a chat about this a few months ago with you and Carl – the basic idea is that hyper, over time, has been increasingly targeting the lowest-level cases and providing the greatest flexibility. The networking WG is interested in a higher level crate that pulls together a few best practices to provide a simple, ergonomic async http story (building in TLS, configuration, and a few other concerns). Hyper would be taken as a private dependency, actually providing the implementation, while provided API surface would be narrower and something we could hope to reach 1.0 on relatively quickly.

[Eh2406]

Public deps

• Change resolution heuristics to favor fewer duplicates where possible

I am willing to help with the resolver part of this, when we have a public description for the gole and there is work on the lint part.