Meaning of #[link] kinds

[retep998]

@retep998

@alexcrichton

That's because this is incorrect. When a linker pulls in a static library it typically only pulls in the parts that are needed, not the entire library. This means that if you create a Rust dylib which links to a native static library it may not actually include all of the static library, which in turn can cause future linkage to fail.

A .lib can be a static library or a dynamic library. A .lib can either be bundled into the .rlib or passed to the linker independently. Here we have two boolean situations, so 2*2 = 4 possible combinations. All 4 combinations are legal in the Windows world.

1. Linking a .lib that provides dynamic symbols and not bundling it into the .rlib. dllimport is applied to externs, and the .lib is passed along to every linker invocation that uses that .rlib. If turned into a .dll, the symbols from the .lib are not re-exported. Currently corresponds to kind=dylib.
2. Linking a .lib that provides static symbols and not bundling it into the .rlib. dllimport is not applied to externs, and the .lib is only passed to the first immediate linker invocation. If turned into a .dll, the symbols from the .lib are re-exported with dllexport.
3. Linking a .lib that provides dynamic symbols and bundling it into the .rlib. dllimport is applied to externs, but the .lib is not passed to the linker. If turned into a .dll, the symbols from the .lib are re-exported with dllexport but still pointing at the original .dll, not the Rust .dll.
4. Linking a .lib that provides static symbols and bundling it into the .rlib. dllimport is not applied to the externs, and the .lib is not passed to the linker. If turned into a .dll, the symbols from the .lib are re-exported with dllexport. Currently corresponds to kind=static.

What I would really like is a kind that corresponds to #2.

[arielb1]

Why are symbols reexported in case 3? Aren’t they from the native dll?

[retep998]

@arielb1

Why are symbols reexported in case 3? Aren't they from the native dll?

That's a good question. On one hand it is entirely valid for a DLL to re-export symbols provided by another DLL (this is actually used quite a bit on windows with the system DLLs, such as kernel32.dll re-exporting many functions from kernelbase.dll). On the other hand it is also valid for the import library for the DLL to export some symbols to the rust DLL and other symbols to the native DLL. The latter would be more efficient by avoiding a layer of indirection. Remember that since in case 3 the original import library for the native DLL is being bundled, that means it cannot be passed along to later linker invocations but rather the symbols have to be provided by the Rust DLL's import library. In both cases the Rust DLL's import library provides the symbols, it's just that in one case the exports redirect through the Rust DLL, and in the other case the exports point directly at the native DLL.

[arielb1]

@retep998

@arielb1

Why are symbols reexported in case 3? Aren't they from the native dll?

That's a good question. On one hand it is entirely valid for a DLL to re-export symbols provided by another DLL (this is actually used quite a bit on windows with the system DLLs, such as kernel32.dll re-exporting many functions from kernelbase.dll). On the other hand it is also valid for the import library for the DLL to export some symbols to the rust DLL and other symbols to the native DLL. The latter would be more efficient by avoiding a layer of indirection. Remember that since in case 3 the original import library for the native DLL is being bundled, that means it cannot be passed along to later linker invocations but rather the symbols have to be provided by the Rust DLL's import library. In both cases the Rust DLL's import library provides the symbols, it's just that in one case the exports redirect through the Rust DLL, and in the other case the exports point directly at the native DLL.

This is relevant when the .rlib is used as part of a Rust DLL, and the created DLL will include parts of the import library independently of whether it is bundled in the .rlib or not.

I mean, if you have, for example

#![crate_type="dylib"]
#![crate_name="rustdll"]

#[link("mydll",kind="dylib")]
extern "C" {
    pub fn MyDllEntryPoint();
}

Should the rustdll.dll contain a forwarder?

Anyway, should this behave distinctly from

#![crate_type="rlib"]
#![crate_name="util"]

#[cfg_attr(bundle,link("mydll",kind="dylib",bundle="true"))]
#[cfg_attr(not(bundle),link("mydll",kind="dylib"))]
extern "C" {
    pub fn MyDllEntryPoint();
}

#![crate_type="dylib"]
#![crate_name="rustdll"]

extern crate util;

pub use util::MyDllEntryPoint;

Possibly depending on bundling?

[retep998]

@arielb1

I mean, if you have, for example

#![crate_type="dylib"] #![crate_name="rustdll"]

#[link("mydll",kind="dylib")] extern "C" { pub fn MyDllEntryPoint(); }

Should the rustdll.dll contain a forwarder?

To better understand this whole situation we have to look at how libraries are passed along to the linker. When creating a binary (whether it is a .dll or a .exe), all .rlib dependencies of the binary are passed to the linker as well as the .lib import libraries for any .dll dependencies the binary has. In addition all the .rlib and .lib import libraries for all .rlib dependencies are passed in, recursively. Note that the dependencies of any .dll's are not linked in, unless it is a non-bundled native dylib which we can pass along.

The consequences of this is that when we are creating a Rust .dll, we have to make sure that all native dependencies other than non-bundled dylibs, both for the Rust .dll itself and all its .rlib dependencies, must be re-exported in the import library for that .dll somehow so downstream dependencies can link to them properly. For static dependencies, both bundled and non-bundled, this means the symbols have to be `dllexport'ed from the Rust .dll, and the import library should contain those symbols. For bundled dynamic dependencies, we don't need a forwarder in the Rust .dll, we just need to make sure the import library for the Rust .dll exports those symbols to the original native .dll they are from, since they are bundled and thus cannot be passed on to future linker invocations on their own.

[alexchandel]

It seems like we need to distinguish between 1) “bundling” a static library into a crate, 2) merely guaranteeing that a static library is linked only once in the final product, and 3) linking against a dynamic library in the platform’s method. (also, linking against Frameworks on OS X, but that’s just special case of dynamic linking)

Linking against a dynamic library would entail loading an .so for an ELF (e.g. Linux), or inserting LC_LOAD_DYLIB (& LC_DYSYMTAB & other) commands for a Mach-O (i.e. OS X). However, the analogous behavior for a PE (i.e. Windows) should be dynamically generating idata sections for DLLs. Linking against the .lib for a DLL is really a form of static linking and should be covered by case 1 or 2.

This also presumes that such dynamic generation isn’t possible on Linux or OSX. If it could theoretically be, then there are really 4 cases.

[retep998]

Linking against a .lib that provides exports is not the same as linking against a .lib that provides static symbols. How dllimport and dllexport is applied differ significantly in those two cases, especially when you start crossing boundaries between Rust libraries, even more so linking against a Rust dylib. We should avoid trying to overload the meaning of things and just keep things explicitly separated. Having different kinds for each of bundled static libraries, non-bundled static libraries, import libraries, and idata sections is the only way to ensure we can get linking actually correct instead of half-assing it.