Torrent crates

Soni · 2018-10-16T01:16:02.103Z

Crates should be available over bittorrent. Cargo should have an optional daemon that seeds them. Crates with torrent dependencies should be publishable on crates.io, which should also make an effort to seed them. Such crates should only be visible on crates.io after crates.io has retrieved the relevant torrents, thus there’ll be a small delay in publishing, but that’s okay since these crates never go down - even when crates.io is down.

kornel · 2018-10-16T01:52:56.674Z

Cargo supports custom registries. You could implement a torrent-backed registry on localhost and configure that in Cargo.

Soni · 2018-10-16T02:01:38.564Z

Torrents are immutable and censorship-resistant, making them a suitable alternative to conventional registries.

Your suggestion doesn’t solve the problem of publishing torrent-dependent crates on crates.io.

earthengine · 2018-10-16T02:04:54.762Z

I am with this idea.

kornel · 2018-10-16T02:14:57.446Z

Indeed, but crates.io is itself a centralized service and the implementation doesn’t guarantee immutability (it doesn’t allow it as a matter of policy, but technically the index could be mutated), so if you wanted these properties, you’d have to publish elsewhere anyway.

kornel · 2018-10-16T02:16:33.364Z

Another integration method could be via git torrent transport. Cargo (but not crates.io) supports git dependencies, and for git deps you can also add rev=hash, so a Cargo git dep using git-torrent-transport and a hash should be pretty immutable.

Soni · 2018-10-16T02:25:25.088Z

gittorrent is bad, any project using gittorrent should be avoided at all costs. any “decentralized” project that relies on a distributed centralized system (bitcoin) is inherently bad.

(unless you like climate catastrophe, ofc.)

(yes, distributed systems can be centralized. e.g. cloudflare is distributed, but I wouldn’t call it decentralized.)

(Just because anyone can run a node, doesn’t mean it’s decentralized. If you, running a node, don’t have any power over it, then it’s not decentralized, it’s just distributed. Since bitcoin is based on an algorithm, and changing that algorithm creates incompatible implementations, then bitcoin is inherently centralized - the network is owned by the algorithm, rather than being owned by a person.)

Nemo157 · 2018-10-16T11:59:59.353Z

I was quite keen on trying to get crate distribution via IPFS working, until I started implementing IPFS and became disenchanted with a lot of the technical choices in it.

I still like the idea of having crates distributed via a global content addressable store, but I don’t think it would be good to build this directly on raw torrents. You probably want something that has a mutable naming scheme on top like IPNS to support short references for the Cargo.toml, then the hash can be entered into the Cargo.lock so that even if the name is changed you still can find the correct package. You could build this on top of torrents yourself, but it would be good to have it part of an existing system.

Soni · 2018-10-16T12:04:46.738Z

Nope, I have no plans to support mutable content.

You might be interested in dat tho.

Nemo157 · 2018-10-16T12:08:20.424Z

Soni:

Nope, I have no plans to support mutable content.

Do you want to manually check for new versions of crates continuously? To be able to automate updating you need some reference to a mutable list of versions. It also makes the user interface much nicer to be able to specify a nice name instead of a hash in the Cargo.toml

serde_json = { ipns = "serde.rs/json" }

Soni · 2018-10-16T12:12:07.261Z

well, dat has .well-known support, so you could just use a domain name.

whatever = { foo = “dat://example.org” }

I still like torrents. they guarantee immutability, and thus reproducible builds.

Nemo157 · 2018-10-16T12:28:32.256Z

Yep, IPNS is built off well-known DNS TXT records as the basis for lookup as well. I think pretty much all global naming schemes will end up being rooted in DNS since it’s so widely available.

By resolving the mutable references into actual content-addressed-hashes for the Cargo.lock you maintain reproducible builds (with guaranteed immutability), while having a way to automatically find out about new versions of crates and supporting a nicer UI. Torrents would work well for the content-addressed-hash part, it’s just the user-friendly mutable addressing on top that they don’t target.

system · 2019-03-25T08:30:59.729Z

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.