External Dependencies in crates and cross-platform development

tools and infrastructure cargo
1

Case Study:

My current development environment is VSCode on Windows, using rust-analyzer for code analysis, running tests and debug builds on Windows, and running release builds on WSL.

I want to begin using blessed crate https://diesel.rs with a MySQL backend. The 'Getting Started' page has reasonable instructions for Linux, given that I mainly need to install libmysqlclient on top of other dependencies usually present for compiling programs (after installing mysql and setting it up), and then I can include diesel in my Cargo.toml file. I can isolate the dependency using a feature, but I don't want to exclude the feature on Windows targets, because rust-analyzer builds for Windows to see if I have compilation errors.

I have not yet gotten it to work on Windows at all. Installation is more complicated: the dependency mysqlclient-sys has a build script dependent on environment variables, which were not set by the MySQL installer on Windows, and it isn't clear from the script's error message which are not set correctly. Diesel's instructions recommend a bundled mysqlclient-sys but this fails in building openssl-sys because perl is not installed in the Windows environment. I had to search for answers to these problems because the documentation didn't address it, and some threads were years old and out of date. And finally, getting diesel_cli to build via cargo install in PowerShell does not imply that building will succeed in VSCode, as that's the current status of my Windows setup.

Discussion:

I really like the crate ecosystem and ease-of-use in adding dependencies to my project. This is really the first time I can think of that a crate has a dependency on something I haven't already installed, and it's been a uniquely frustrating experience that I can't spend my time focusing on writing code and testing out the new dependency.

Is there Could there be a way to allow this dependency management to occur within the cargo files, such as:

  • specifying config values in an environment file, similar to how dotenvy reads from a .env file? (and having cargo build request them if they're missing)
  • specifying and checking requirements that cannot be installed by cargo or build scripts, such as perl

Ideally I also don't have to provide in-depth installation instructions for diesel stuff beyond MySQL, just as diesel shouldn't have to for mysqlclient-sys, and just as mysqlclient-sys shouldn't have to for openssl-sys. (MySQL table setup can easily be scripted.)

My project has no reason to be platform-specific, so it would feel bad to ditch Windows as an allowed target platform. But on the other hand, having a streamlined build process is such a core feature of cargo that it also feels bad to have these issues trying to add a popular crate as a dependency for multiple popular platforms.

2

I don't know what's going on here, but you might have better luck asking in users.rust-lang.org. That forum is for providing people help with using Rust, this forum is for discussions about features of the Rust language and how we can improve it.

3

The individual issue has already been resolved. I'm using it as an example of a gap in my cargo experience in order to start a discussion around how to improve it.

Maybe I should reword "Is there a way" to "Could there be a way"

4

The system-deps crate might be heading a nicer direction for the whole ecosystem, and has been adopted by quite a few crates, but it seems there's still a long way to go.

5

I just would like to leave this as note here: While system-deps sounds like a great solution it has some important restrictions which make it hard to use it as general solution. It's based on pkg-config, which means it will work well on most linux based systems. Unfortunately that also means it won't work well in other setups like systems using windows or custom linux configurations or cross compiling setups. That makes it really hard to use it a solution if you need to support these other setups as well.

In general finding system dependencies is really messy and needs a lot of custom configurations so support widely different system configurations and requirements, especially on windows. As far as I'm aware there cannot be a cross platform way to declare this users just have "too much" control over where to put stuff and the whole system did grow organically over the last ~30 years or something like that.

1 Like
6

pkg-config also performs (previously performed?) the dubious action of patching how cmake performs certain loads/resolutions, which doesn't work with all dependencies....

7

I think solutions like that systemically underestimate complexity of the problem.

OSes with package management (like Linux distros) do a lot of work for you to clean and normalize all the weird libraries, their quirks, incompatibilities, bizarre requirements, etc. Then the problem looks like "just" picking the right library location and a few flags, so you may be fooled into thinking it can't be so hard.

But in every other environment that hasn't already done the work of building a whole OS worth of packages that work together – it's not a matter of "just" finding the libraries, it's a difficulty on the same scale of complexity as building an operating system distro, with all the ugly package-specific hairy mess that happens behind the scenes now being your problem, multiplied by every package by every target OS.

I think it's not an accident that Rust crates have bespoke build scripts:

  • if you support 1 package on O number of operating systems, like build.rs scripts to, this is 1xO problem size, which is manageable.

  • if you support 1 operating system with P packages, you have 1xP work to do, which is a lot, but a distro can do most of it for you, so for distro-centric system-deps/metabuild/pkg-config it feels like 1x1 problem.

  • if you try to make system-deps that works on more than just existing package managers, you suddenly have PxO sized problem, and no help. That's not realistic, which is why these tools are chronically bad on Windows and macOS, and/or try hard to delegate the hard work to the most Linux-distro-like thing they can get, like macOS Homebrew.

2 Likes
8

I have been out of the loop on Windows dev for many years, and never really done any dev work on MacOS X at all (classic MacOS I did dabble a tiny bit on). So this raises more questions than it answers:

  • What about nuget / homebrew? Not first party obviously, but I thought they were the standard solution for this on those platforms these days?
  • What do people who targets those platforms use then if they haven't settled on standard solutions? I know it was a mess on Windows back in the XP era, but surely they solved it since then?

Surely every single game dev or app developer can't be struggling with this, someone must have built a solution by now?

9

Like C build systems, this has been "solved" many times, over and over again, but everyone solved it differently, and the new problem is that everything is fragmented and incompatible with each other.

Apple has "solved" packaging only for Swift and Objective-C, and only for users of Xcode IDE, and only for development of Apple's app bundles (like Flatpak/Snap but different). Apple is so egocentric and full of NIH, they don't acknowledge existence of anything else. macOS has some POSIX compatibility[1], but Apple seems to have a deep disdain for it, and has their own proprietary replacements for all of it[2].

Apple's app bundles are a poor fit for command-line utilities, so for bringing non-Apple tools and libraries, macOS had 3rd-party solutions like Fink, MacPorts, and now Homebrew. Homebrew doesn't solve software packaging for macOS in a "native" way, but it's more of a port of non-Mac software to Mac. Almost like Cygwin or WSL1 on Windows, only a bit less alien than Wine Bundler on Linux.

Homebrew has solved dependency management for the packages that are in Homebrew's distribution, and only for packages that are using Homebrew.

Homebrew is a developer-centric 3rd party solution, and not part of macOS. Regular Apple users are unlikely to have it. To distribute an app to regular Mac users, you need to use Apple's preferred method of making self-contained app bundles, with their own nested framework bundles. Homebrew doesn't work like that, and even breaks this.

macOS doesn't have pkg-config, Homebrew does. Apple doesn't support .pc files. Apple has their own .tbd files, which are meant only for Apple's Xcode.

Unfortunately, build systems brought from the Unix world assume that pkg-config is normal. For macOS app bundles pkg-config has wrong defaults, and finds wrong versions of wrong libraries in wrong locations. pkg-config will link dynamically to Homebrew's non-standard libraries using absolute paths to its private directory. The app bundle won't record which dependencies it requires, because there's no such feature! Apple's app bundles don't support dependency management at all. You can only use what you shipped in the bundle, or what Apple shipped with the OS, nothing else. Homebrew isn't in the app bundle, and Cargo has no clue how to relocate dynamic libraries to inside app bundles.

The end result is that if you build with Homebrew, you get an app that works on your machine only. Even if someone else has Homebrew installed too, your app still won't work, unless the recipient manually installs all the Homebrew packages in the right versions (nobody does that).

When building Mac app bundles, you need to be very careful to make sure everything from Homebrew is only linked statically, while also ensure not to use Homebrew for anything that exists in macOS's libraries[3].

This Homebrew vs macOS problem is like building a binary on Debian, and then sending the bare executable to users of Arch (not as a package, just the elf binary, with no installation instructions). You just get a ton of linker errors for missing libraries or libraries having wrong versions.

So you have effectively something like apple-darwin-homebrew platform (like linux-gnu) and apple-darwin-actually-apple (like linux-musl), but Cargo is completely unaware of that, and Rust sys crates support this on case-by-case basis with custom env vars and Cargo features.

  1. they've got Unix® certification in 2003, and not cared about it since. They've left everything GNU to rot in 2007 ↩︎

  2. /tmp exists, but you're supposed to call NSTemporaryDirectory, which doesn't return char*, but NSString. Or ideally use FileManager.default.temporaryDirectory that returns a URL with a file:// path ↩︎

  3. I've had horrible crashes caused by linker mixing symbols from my Homebrew-built libraries and Apple's built-in system frameworks, and it's impossible to avoid linking Apple's system frameworks for anything non-trivial ↩︎

2 Likes
10

Nitpick: .tbd files are there to speed up linking. Instead of reading the actual library to extract this information, it is pulled out in a JSON document so that the linker can craft the actual runtime linker requirements. Not too dissimilar from .lib files on Windows for .dll runtime libraries (except that the Windows linker only understands .lib files).

Not dissimilar to using MinGW libraries on Windows really. Back in the day I had a toolchain setup to compile Windows binaries of the FFmpeg SDK for use on Windows. Nowadays MSVC supports enough C99 to compile FFmpeg, but this wasn't always the case.