Pre-RFC: Complete cdylib support in cargo

tools and infrastructure cargo
1

I’m trying to make easier for C projects to use the AV1 encoder I’m contributing to.

My ideal outcome is to make so random users could just do cargo install crav1e and have everything they need:

  • header files
  • static libraries
  • dynamic libraries
  • pkgconfig files

We are really far from there.

Installation is completely on your own since this rfc is still being worked on, but probably we’ll get something to use sooner or later :slight_smile:

Current status

Autogenerated headers

cbindgen works already quite well so producing a .h from the rust code is the easiest task by far. You add it in your build.rs and you are pretty much sorted.

Static libraries

So far the bare basics are covered with the crate type staticlib. A .a gets produced and if linked works, assuming you know which are the additional libraries you need to link beside the .a.

Shared libraries

There is a cdylib crate type, but the produced library is missing a soname and depending on the platform you might need to pass an rpath and more linker information that are currently missing.

Pkgconfig file

You need to provide a .pc file to correctly link the static library and so far producing it is pretty ugly:

touch src/lib.rs && cargo rustc -- --print native-static-libs 2>&1 | grep note | cut -f 2 -d ":"`

Is the simplest way I figured out, maybe there is a better way.

What to improve

Right now the most sore point is on cdylib and it is because you really want to give the linker additional information that cargo does not pass.

If you want to produce something you may either relink the staticlib and hope the code is already position-independent if needed.

Alternatively you may try your luck and use cargo rustc or touch the sources and pass the needed flags through the env var RUSTFLAGS or manually edit the produced library.

I already prepared a pr to have a mean to compute the correct linker flags in build.rs and pass it to rustc, but there are some concerns somebody will misuse the feature.

An alternative is to make cargo provide by itself at least the linkflags to generate the soversion on the supported platforms and restrict the ability to override this only for cdylibs.

On irc/discord one of the suggestions was to write down a pre-rfc before writing more code, so here I am.

1 Like
2

I believe the intention of https://github.com/rust-lang/rfcs/pull/2376 is to largely solve most of the issues here, but it may not cover everything just yet!

3

It does not cover passing soname and rpath at linking time, so solves about 1/2 of my problems (and that’s why I started working on the other half).

4

Makes sense to me!

For the link-args changes I think that specifying them in build scripts is the way to go, but as I mentioned on irc/discord I think we’ll want to be a little more strict than allowing it in every build script (but we can still add to Cargo!)

5

Please don't install anything system-wide (like headers or pkg-config files) via Cargo!

Cargo is unable to clean up anything other than executables on uninstall, so it means that cargo uninstall crav1e won't uninstall the extra files. There are similar problems with upgrades: if you rename a file, the corpse of the previous version will be there, likely confusing linking and/or pkg-config.

The proposed RFC doesn't try to solve that:

However, we do not intend for cargo install to become a full-featured package management mechanism; rather, we expect cargo install to work analogously to make install .

To me make install is a horror that breaks operating systems, so please don't do that!

  • For macOS, Homebrew is happy to package Rust crates. brew install crav1e will even work for more users and integrate much better with the rest of the system.

  • For Debian/Ubuntu, there's cargo-deb which supports packaging of additional files (with upgrades, uninstall, etc).

2 Likes
6

both homebrew and deb/ebuild/rpm use make install or equivalent under the hood.

The packaging phase just install everything on a destination directory and then the manager track the files and puts them in the right place :slight_smile:

cargo would need to know about destdir, prefix, libdir and incdir to be useful to the package manager.

(If you try to make an homebrew recipe for crav1e in the current state you’d see firsthand why I’m focusing in being able to pass link arguments from build.rs)

7

Should I update the patch to work only on cdylib and cdylib+staticlib and error out if the type does not match?

8

The critical difference is that there's a "hood" above that which tracks these files and can upgrade and uninstall them. If I run brew remove package the headers are deleted and it disappears from pkg-config, as expected.

1 Like
9

As packager I need to have the ability to install, I really suggest you to try and see how cumbersome is to write a brew recipe for crav1e as it is.

10

I notice that Meson advertises support for Rust now - would this fit your use case?

11

I think that’d help! I don’t know if that’s all we need though, (hence why we’re largely postponing Cargo feature work now until after 2018)

1 Like
12

No, since meson does not support anything but running rustc on simple no-deps cases.

I discussed with some of the meson contributors and, there is friction in supporting cargo in any way…

13

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.