Call for help: Cargo "airplane" mode

1

One of the most frequent requests we get around Cargo is the ability to run “offline”. This usually comes up in two cases:

  • As part of a larger build system.
  • Because you’re on a plane.

For the former, we’ve developed a bunch of tools (like cargo vendor) that let you control the distribution of artifacts and ensure your build is done with local artifacts.

For airplanes, though, we’re not as complete as we could be: if you make non-trivial changes to a Cargo.toml file, Cargo will refuse to re-resolve the dependency graph and produce a new lockfile, Cargo needs to hit the network.

We’d like to change that, by allowing Cargo to use the artifacts that happen to be around on the machine to attempt to re-resolve graphs and so on. And @alexcrichton has put in the work to write up detailed mentoring instructions for implementing exactly this. If this is a problem you care about, this is a chance for you to scratch that itch!

You can find the instructions here – please let us know if you’re interested in taking it on!

49 Likes
2

This is a feature I’ve been looking forward to.

I just want to expand here that “airplane” is not the only possible reason for no internet connection.

Here are a few examples:

  • Limited data plan, the user wants to do some development offline, without using up data.
  • In area with bad/no internet connection. Yes, there are still areas like this, especially in the less developed countries.
  • No internet connection for a set period because ISP decided to do some intrusive maintenance, or there is some other problem on the ISP’s side. (Yes, this does happen from time to time where I live).
19 Likes
3

… or because you are working on things that will never be allowed to be connected to the internet.

17 Likes
4

… Or as hinted in the first post, you’re building a Debian package so you have to avoid network access.

10 Likes
5

So is it meant to work a little bit like how it does in Go?

6

Really useful for CI systems as well

4 Likes
7

So is it meant to work a little bit like how it does in Go?

Can you clarify what you mean? We're not all go programmers here.

8

Is there a reason we need airplane mode at all? Perhaps only hit the network if we absolutely need to?

9

My impression is that this is a toggle for the situations where the sensible thing to do is to sync with crates.io, but you don't have an Internet connection at the moment.

e.g.

When I start a new project, I probably should be using the latest patch version of my dependencies. But maybe I'm toiling away on a long bus trip and tethering is just going to wear down my phone battery a bit more than I can reasonably spare.

2 Likes
10

Hey there everyone,

I am new to Rust and just joined this forum right now, however, an Offline (call it airplane or something else) mode where it caches dependencies is helpful in many cases, it provides faster builds, less data consumptions and overall increases the happiness of the developers using a Packaging tool in many ways. As is evident from the success of Yarn and Offline mode in NPM the build times after Yarn introduced the concept and NPM adopted it afterwards the build times greatly reduced and people are overall much happier. This was a much needed feature in the Packaging space for JavaScript and people greatly appreciated how quick it made the things, especially happy were the Australians and people in countries where they don’t have the best internet connection. I’d love to see this feature land in Cargo and look forward to trying rust. :heart:

2 Likes
11

Go simply has a path to where packages are saved, if it has been downloaded already, there’s no need for it to be downloaded again.

1 Like
12

Cargo has this already.

$ ls ~/.cargo/registry/src/github.com-1ecc6299db9ec823/
advapi32-sys-0.1.2          env_logger-0.4.3         liner-0.4.2               pretty-0.3.2                   smallvec-0.4.4
advapi32-sys-0.2.0          error-chain-0.10.0       linked-hash-map-0.3.0     proc-macro-hack-0.3.3          socket2-0.2.1
aho-corasick-0.5.3          error-chain-0.11.0       linked-hash-map-0.4.2     proc-macro-hack-impl-0.3.3     stable_deref_trait-1.0.0
aho-corasick-0.6.2          error-chain-0.11.0-rc.2  linxal-0.6.0              proc-macro2-0.1.3              strsim-0.5.2
aho-corasick-0.6.3          fern-0.4.3               log-0.3.0                 procedural-masquerade-0.1.2    strsim-0.6.0
ansi_term-0.9.0             filetime-0.1.10          log-0.3.8                 procedurals-0.2.3              syn-0.8.7
ansi_term-0.10.2            filetime-0.1.12          lzma-sys-0.1.7            prometheus-0.2.8               syn-0.10.8
antidote-1.0.0              fixedbitset-0.1.8        magenta-0.1.1             protobuf-1.4.1                 syn-0.11.11
app_dirs-1.1.1              flate2-0.2.19            magenta-sys-0.1.1         psapi-sys-0.1.0                synom-0.11.3
array-macro-0.1.1           fnv-1.0.5                matches-0.1.6             pulldown-cmark-0.0.11          syntex-0.58.1
array-macro-internal-0.1.1  foreign-types-0.2.0      matrixmultiply-0.1.13     pulldown-cmark-0.0.14          syntex_errors-0.58.1
ascii-0.7.1                 frunk_core-0.0.17        matrixmultiply-0.1.14     quasi-0.32.0                   syntex_pos-0.58.1
ascii-canvas-1.0.0          fs2-0.4.2                mdbook-0.0.22             quasi_codegen-0.32.0           syntex_syntax-0.27.0
aster-0.41.0                futures-0.1.16           memchr-0.1.11             quick-error-0.2.2              syntex_syntax-0.58.1
atty-0.1.2                  futures-cpupool-0.1.6    memchr-1.0.1              quick-error-1.2.0              tabwriter-1.0.3
atty-0.2.2                  gcc-0.3.43               memchr-1.0.2              quick-error-1.2.1              tar-0.4.13
atty-0.2.3                  gcc-0.3.51               mime-0.2.6                quote-0.2.3                    tempdir-0.3.5
.
.
.

But there's two parts to crates.io: Obviously there's the crates, but there's also the index, which to my understanding is used for dependency resolution.

Cargo keeps a local copy of the index as well, but refuses to use it when offline. (this is for reasons entirely unbeknowest to me; probably worth a gander at the issue in the first post) My guess is that it really wants to make sure that it is up to date.

P.S. as somebody who codes on the bus, I can attest that Cargo does use things from .cargo/registry/src when offline... as long as you don't touch your Cargo.toml. :V

7 Likes
13

It’s worth noting that Cargo will be completely offline if you’re working on an existing project. Only when adding new dependencies, or when creating a new project, will it require a refresh.

This is still quite annoying for developing offline, but I feel like it’s not a bad policy. Cargo doesn’t (currently) want you to start a new project or a new dependency with old packages.

1 Like
14

I think there is a good reason for offline mode: privacity. Personal and global data is sold nowadays and well paid. In fact is the main income of many wealthy companies, so is a main concern. I’ve success on porting slackware to arm hand float, and rust made its debut on 15.0 (now alpha). When I reach the rust package I noticed the need of a live connection, This is not the Slackware way. When I eat a cow, I preferred in dead, I don´t eat cows alive, nor develop whis a logger looking at me. So i will wait for an viable version of rust to include in my port. No offline: no port.

15

Well, rust doesn’t need an active connection to build your code. The only reason you need to be connected to the Internet is when you build a project for the first time with Cargo or when you add a new dependency, and the network is needed just to download the dependencies and the list of available crates from the registry.

16

Only when adding new dependencies, or when creating a new project, will it require a refresh.

It's also not something you can really get around, unless as a developer you want to locally mirror crates.io (but if you can do that, what was the point of using airplane mode again? :slight_smile: ).

17

I agree with all of this.

I would like to add that I would like cargo to support using non-up-to-date versions of dependencies when it cannot refresh its package index, if an older compatible version is available in the local cache.

For example, if I am on an airplane, and I want to add a new library to my project. Say, I add this to my Cargo.toml:

[dependencies]
some_new_crate = "0.2"

Assuming that I have already used some_new_crate for some other project (or it was a dependency of something i installed with cargo install), it should already be available in the local cache in ~/.cargo.

Assuming that the locally-cached version is compatible (say, I have 0.2.1 in the cache), I would like cargo to be able to use that, even if there might be a newer version (say, 0.2.5) on the internet. If cargo cannot refresh its index or download the latest version, it should fall back to whatever is available in the local cache.

I’d much rather have a slightly out-of-date library in my Cargo.lock (because that’s what I happened to have available in the offline cache at the time I added the dependency and Cargo had no internet connection) and be able to proceed to work on my project, than to be blocked and not be able to do any work at all, because Cargo insists on connecting to the internet to check what the newest version is.

Right now I cannot do this. It is a major blocker. There are many common libraries (such as log, rand, failure, …) that I use in many of my projects and are in my ~/.cargo cache. But I cannot use them. If I realize I need one of them for a project that hasn’t needed it yet, I am stuck. Starting a new project from scratch is impossible without an internet connection. This is awful.

EDIT: I might try some of the scripts and workarounds provided in the reddit thread linked to in the github issue from the OP.

4 Likes
18

@SheepKey have you tried -Z offline? I think it will do what you mention (it will use the best-available version from the cache). It only works with nightly, though.

2 Likes
19

FWIW I was recently in a remote area for about 10 days without internet access, and found myself running into this when attempting to create a new toy crate. I pretty quickly reinvented the wheel and hand-rolled my own Cargo.lock file by splicing together a bunch of other lockfiles that I had sitting around. Not great UX, but it got me by.

3 Likes
20

What exactly does Cargo use the network connection for when creating a new project? Can this be avoided by using explicitly versioned dependencies? What about when there are no dependencies at all?