Moving bits of rustc into crates


#21

Looks like I’ve reserved Hashish as a software project name early enough :wink:


#22

I think I’ve seen already existing crates.io dependencies for smallvec and arrayvec when compiling rustc. At the same time we have versions of both in rustc_data_structures. Those seem like good candidates to look into.


#23

A bit more creativity please :slight_smile: “chalk” and “polonius” are good names. I’m sure you can come up with something interesting: “flashhash”, “ultrahash” or something else that does os does not contain the word “hash” ^^’

ferrishash

My understanding was that the first pass would be over more generic data structures, so there wouldn’t need to be as much disentangling…


#24

The rustc-hash crate now exists and rustc will now use it.


#25

As an aside: I definitely agree with creative crate names, but I couldn’t come up with one that felt right in the allotted time. =) Also, on the scale of crates, this crate … doesn’t seem like one that has to have a cool name. =)


#26

@nikomatsakis I think that “rustc-hash” isn’t a good name choice because the crate isn’t inherently linked to the compiler. This makes the “rustc-” part is a bit misleading. Consider naming it “ferrishash” like @mark-i-m suggested. The name is available and it sounds great!


#27

rustc-hash sounds better. It is the hash implementation used by the rustc compiler. That doesn’t mean it is only used by the compiler or can only used by the compiler.


#28

I don’t agree. A shared prefix means to me that the two crates have a strong relation. For example “actix” and “actix-web”. Besides, “creative names” are encouraged in the crates.io package policies (see the section about why crates.io has no namespacing).


#29

Yes, I would agree, and I would think that the crate being created in rust-lang specifically for the purpose of the Rust compiler no longer needing to have that create internally, suggests just such a strong relationship. Of course, just an opinion.


#30

It’s a fast non-cryptographic hash map and suitable for any project. Hence no strong relationship to rustc except that rustc uses it.

Very similar: The “url” crate from servo is also general purpose. It’s not called “servo-url” for that reason.


#31

Yeah, that argument makes sense. That being said, I think for fundamental stuff, like this, having a name that functionally describes as opposed to being “memorable” (or cute) is valuable. If I’m looking for a blessed “HashMap” implementation, it would be very easy to get too “Cute” with the name. But, I definitely see where you are coming from.

For example of what I mean by “Too Cute”:

  • HashBrown
  • Hashish
  • TripleRMap (see above for the joke)
  • HashTag
  • PoundIt

What would be “good” names (IMHO) would be something like:

  • rlang-hash
  • rl-hash
  • rl-std-hash

etc.


#32

“quick-hash” is also still available (“quick_hash”, “quickhash” as well)

BTW in in the JavaScript community package names like “node-…” and “…-js” are a bit frowned upon. “rlang-hash” isn’t a good name. It just indicates that it’s written in Rust - just like every other crate.


#33

Hm, one problem with a lot of the data structures seems to be that they must support serialization via rustc’s own libserialize. That makes it hard to move them out-of-tree. Does anybody have a good solution for this?


#34

Could libserialize act as a serde backend?


#35

One of the problems with something like “serde” is that due to coherency rules, only the crate that creates a new trait may impl that trait for an arbitrary crate and only a crate that declares a struct may impl an arbitrary trait for that struct. 3rd party crates may not impl traits they did not define themselves for structs they did not themselves define.

I wonder if there could ever be a notion of “Friend” or “Blessed” impl crate to work around this restriction in a way that maintains soundness? What if a crate could declare “friend” crates and those “friend” crates would be permitted to provide impls for arbitrary 3rd party traits?


#36

I’m suggesting that those data structures can implement serde for their own types, and libserialize can implement its data format for serde. There’s no coherency problem with that.


#37

I don’t think we can use serde in the compiler yet because of bootstrapping restrictions (right, @eddyb?).


#38

We can’t use serde for e.g. target specs’ JSON because of bootstrap and proc_macro (I need to finish PR #49219 for that).

We can’t use serde for the rest of the compiler because of an impedance mismatch of serde's generic-over-(de)serializer philosophy, and rustc's use of contexts with lifetimes for arena-allocated resources.
That’s where issue #36588 comes in, which is also blocked on solving the bootstrap proc_macro problem.


#39

Would this mean rustc won’t be buildable without an internet connection? What happens if crates.io goes down for some reason?


#40

The rustc build system as a whole already uses external crates, and these are bundled in the source tarballs under src/vendor/.