Moving bits of rustc into crates

[nikomatsakis]

I considered that, but it’s already so annoying to deal with two orgs, I don’t really want to deal with three…

[nrc]

Yeah, there is already a mirror of libsyntax on crates.io (rustc_ap_syntax) which is used by rustfmt and Racer. It would be cool if we could make that the source of truth rather than a mirror. Making libsyntax stable is a bit challenging, but if it only had to happen once it should be OK. Separating out libsyntax tests would be another challenge. Perhaps we could start this with some of the deps of libsyntax?

[CAD97]

Since “stable” out-of-tree libsyntax was mentioned, see also @matklad’s libsyntax2 experimental project (RFC, repo, LALRPOP)

[michaelwoerister]

Sounds like a good idea for code that doesn’t change much anymore. However, fixing bugs in external code requires two pull requests and two reviews. There’s also the chance of reviewers not being as strict for crates outside of rustc, so I’d proceed with care.

[nikomatsakis]

However, fixing bugs in external code requires two pull requests and two reviews.

Well, it depends. Often just one, which lands without bors, and then a simple bump in the minimum version of Cargo.toml. But the larger point definitely stands: coordinating updates is more work.

[nikomatsakis]

Anyway tbh my primary immediate concern is the FxHashMap stuff, which chalk uses. I see a few options here:

• Make chalk use fxhash but leave compiler untouched
• Make compiler use and re-export fxhash
• Create our own fxhash-like crate (rustc-hash?) in rust-lang-nursery and have compiler + chalk use that

I’m leaning towards the third option right now.

[MajorBreakfast]

rustc-hash ?

A bit more creativity please “chalk” and “polonius” are good names. I’m sure you can come up with something interesting: “flashhash”, “ultrahash” or something else that does or does not contain the word “hash” ^^’

[gbutler]

or something else that does os does not contain the word “hash”

TripleRMap (Rust Ricky Resin)?

[est31]

I think factoring out generic bits and pieces of the code and moving it into separate crates is a great idea.

The great drawback was already mentioned which is atomic changes across the entire codebase. I think that components can be refactored though to not rely on a multitude of rustc’s internals and be generic.

[HadrienG]

Looks like I’ve reserved Hashish as a software project name early enough

[michaelwoerister]

I think I’ve seen already existing crates.io dependencies for smallvec and arrayvec when compiling rustc. At the same time we have versions of both in rustc_data_structures. Those seem like good candidates to look into.

[mark-i-m]

A bit more creativity please “chalk” and “polonius” are good names. I’m sure you can come up with something interesting: “flashhash”, “ultrahash” or something else that does os does not contain the word “hash” ^^’

ferrishash

I think that components can be refactored though to not rely on a multitude of rustc’s internals and be generic.

My understanding was that the first pass would be over more generic data structures, so there wouldn’t need to be as much disentangling…

[leodasvacas]

The rustc-hash crate now exists and rustc will now use it.

[nikomatsakis]

As an aside: I definitely agree with creative crate names, but I couldn’t come up with one that felt right in the allotted time. =) Also, on the scale of crates, this crate … doesn’t seem like one that has to have a cool name. =)

[MajorBreakfast]

@nikomatsakis I think that “rustc-hash” isn’t a good name choice because the crate isn’t inherently linked to the compiler. This makes the “rustc-” part is a bit misleading. Consider naming it “ferrishash” like @mark-i-m suggested. The name is available and it sounds great!

[gbutler]

rustc-hash sounds better. It is the hash implementation used by the rustc compiler. That doesn’t mean it is only used by the compiler or can only used by the compiler.

[MajorBreakfast]

I don’t agree. A shared prefix means to me that the two crates have a strong relation. For example “actix” and “actix-web”. Besides, “creative names” are encouraged in the crates.io package policies (see the section about why crates.io has no namespacing).

[gbutler]

A shared prefix means to me that the two crates have a strong relation

Yes, I would agree, and I would think that the crate being created in rust-lang specifically for the purpose of the Rust compiler no longer needing to have that create internally, suggests just such a strong relationship. Of course, just an opinion.

[MajorBreakfast]

It’s a fast non-cryptographic hash map and suitable for any project. Hence no strong relationship to rustc except that rustc uses it.

Very similar: The “url” crate from servo is also general purpose. It’s not called “servo-url” for that reason.

[gbutler]

Yeah, that argument makes sense. That being said, I think for fundamental stuff, like this, having a name that functionally describes as opposed to being “memorable” (or cute) is valuable. If I’m looking for a blessed “HashMap” implementation, it would be very easy to get too “Cute” with the name. But, I definitely see where you are coming from.

For example of what I mean by “Too Cute”:

• HashBrown
• Hashish
• TripleRMap (see above for the joke)
• HashTag
• PoundIt

What would be “good” names (IMHO) would be something like:

• rlang-hash
• rl-hash
• rl-std-hash

etc.