Pre-RFC: Improved low-level support for applications that load plugins

[murarth]

It is common for dynamically linked libraries loaded as plugins to share functions and other globally-scoped symbols with the main program. For C programs (on Linux), this is accomplished by passing --export-dynamic to the linker when linking the main program.

When dlopen() brings the plugin library into the running process, any symbols present in the main program binary are made available to the plugin code, allowing plugins to call function and access static variables defined in the main program. As a result, it is unnecessary to link a library to the plugin that is also used by the main program, as these symbols will be overriden upon runtime linking.

At this time, it is not possible to accomplish either of these goals using the Rust compiler without the use of OS-dependent linker flags or unstable features which are likely to be removed.

Therefore, to improve support for this kind of design in Rust, I propose two new attributes:

• #![export_symbols], applied at the crate level, will cause symbols of the target crate to be exported to the dynamic symbol table.
  This attribute should be present in the bin-type crate for an application that loads plugins.
• #[plugin_link], applied to extern crate items, will import names into the enclosing scope, but skip linking the named crate into the host crate.
  This attribute should be present within plugin crates on any crate items which are expected to be present in the main program.
  Note that this is distinct from the #[no_link] attribute, which does not import non-macro names into the enclosing scope.

(Bike-shedding on attribute names is welcomed.)

[geofft]

I'm super interested in seeing good support for loadable modules, but we should make sure we get ABI-safety right, so I have a couple of questions about the overall approach.

When dlopen() brings the plugin library into the running process, any symbols present in the main program binary are made available to the plugin code, allowing plugins to call function and access static variables defined in the main program. As a result, it is unnecessary to link a library to the plugin that is also used by the main program, as these symbols will be overriden upon runtime linking.

I'm worried about the forwards-compatibility of this. For example, if my host program happens to use OpenSSL, and my plugin also wants to use OpenSSL, but the plugin interface doesn't have anything to do with SSL, the plugin should also explicitly link (declare a dynamic library dependency on) OpenSSL. Otherwise, if my host program decides to switch to BoringSSL or NSS or something, the plugin will stop working for no good reason. On the other hand, if OpenSSL is explicitly part of the plugin interface (e.g. the interface permits passing an open SSL connection into the plugin), then yes, the plugin should rely on the host program's loading of OpenSSL, but then the host program ought to explicitly re-export some or all of OpenSSL's symbols with something like pub use openssl.

#![export_symbols], applied at the crate level, will cause all symbols of the target crate to be exported to the dynamic symbol table.

Is there a reason pub can't just do this? Does pub at a top-level binary crate have any existing meaning? (Either way, it absolutely shouldn't do this for non-pub symbols. GNU ld's --export-dynamic supports --dynamic-list to do the equivalent privacy thing.)

Alternatively, instead of expecting to find global items in the binary crate itself, maybe the plugin crate and the binary crate should both depend on a plugin-API crate. Then the plugins' dependencies are just normal dynamic-library dependencies; libprogram depends libapi, libprogram depends libplugin depends libapi. This is no different from, say, libprogram and libplugin both depending libstd. It also allows linking the program and plugin against a relatively small shared ABI, so that you don't have to recompile the plugin if the program gets updated in a backwards-compatible way. Given how easy the Rust ecosystem makes having multiple crates, I'm not seeing the advantage of letting binary crates export symbols, but maybe I'm missing an intended use case.

#[plugin_link], applied to extern crate items, will import names into the enclosing scope, but skip linking the named crate into the host crate.

If you do the plugin-API-library-crate thing, then it's correct for the plugin to declare a dynamic-linkage dependency on the interface. (Even if you don't, it's still correct, but I'm less sure about whether dynamic linkers let you declare dependencies on binaries. But for those that do, that linkage dependency should be stated.)

This attribute should be present within plugin crates on any crate items which are expected to be present in the main program.

As above, this is part of the ABI, so if you intend to use a re-exported crate, you should just do e.g. use host_program::openssl; instead of #[plugin_link] extern crate openssl;. (Even on platforms without two-level namespacing, Rust's name mangling makes this work without symbol conflicts, since host_program is exporting these items under a different symbol name than it imports them.)

[murarth]

I'm worried about the forwards-compatibility of this. For example, if my host program happens to use OpenSSL, and my plugin also wants to use OpenSSL, but the plugin interface doesn't have anything to do with SSL, the plugin should also explicitly link (declare a dynamic library dependency on) OpenSSL. Otherwise, if my host program decides to switch to BoringSSL or NSS or something, the plugin will stop working for no good reason.

If a plugin is using an openssl crate, it will need to declare it extern crate, just like usual. Whether your project advises plugin creators to mark openssl with #[plugin_link] depends on whether you expect openssl to be linked into the main program in the future. If openssl may be replaced in the future, the host program could create an abstraction around openssl interfaces, which could be used without declaring openssl in a plugin crate.

On the other hand, if OpenSSL is explicitly part of the plugin interface (e.g. the interface permits passing an open SSL connection into the plugin), then yes, the plugin should rely on the host program's loading of OpenSSL, but then the host program ought to explicitly re-export some or all of OpenSSL's symbols with something like pub use openssl.

Re-exporting external crates is an option and perhaps the best option in this case. The re-export would live in a common library crate and a plugin could access it through that crate.

Is there a reason pub can't just do this?

A symbol table isn't created by default for bin-type crates. Without --export-dynamic, a dlopen'd plugin will not be able to access any symbols in the main program. Whether use of --dynamic-list is necessary is an implementation detail. I don't know whether or not it should be used here. However, implementation of these attributes does not modify reachability determinations used within the Rust compiler. To have said that "all symbols" would be exported was incorrect wording on my part. I'll edit the original post accordingly.

Alternatively, instead of expecting to find global items in the binary crate itself, maybe the plugin crate and the binary crate should both depend on a plugin-API crate. Then the plugins' dependencies are just normal dynamic-library dependencies

Honestly, I had not considered using dynamic libraries to solve this problem. However, while that would work, it requires passing -C prefer-dynamic to every crate linked into the project. If this is possible in a Cargo project, it's not clear to me how to do it. If it is possible, it would remain an option even if these attributes were included in rustc. As static linkage is rustc's default, the advantages of it must have been decided to be worthwhile. It would be desirable then, I think, to maintain the option of using static linkage for a program that loads plugins.

It also allows linking the program and plugin against a relatively small shared ABI, so that you don't have to recompile the plugin if the program gets updated in a backwards-compatible way.

I don't think this is currently the case. Every mangled symbol has a hash at the end that changes when recompiled. However, if this is changed in the future, I believe that the same would apply to a plugin system using these attributes. The host program, with common lib statically linked, would need to be recompiled, but the plugin would look for the same symbol names at runtime.

If you do the plugin-API-library-crate thing, then it's correct for the plugin to declare a dynamic-linkage dependency on the interface.

It's not necessary for a plugin to link to a library to resolve symbols that will be found in the main program. But while #[plugin_link] is not necessary to produce a functional plugin, it prevents unnecessary linkage of duplicated code, which would otherwise inflate plugin file size.

[geofft]

Honestly, I had not considered using dynamic libraries to solve this problem. However, while that would work, it requires passing -C prefer-dynamic to every crate linked into the project. If this is possible in a Cargo project, it's not clear to me how to do it. If it is possible, it would remain an option even if these attributes were included in rustc. As static linkage is rustc's default, the advantages of it must have been decided to be worthwhile. It would be desirable then, I think, to maintain the option of using static linkage for a program that loads plugins.

I don't think you need to do -C prefer-dynamic for every crate. You just need a way to mark this particular crate as dynamic. Its own dependencies would be static, other dependencies of the plugins would be static, etc.

That's a fairly self-contained change, and it seems roughly equivalent in scope to introducing #[plugin_link], which also will use a library for compilation but skip statically linking it. You could either do this with #[dynamic] extern crate foo_plugin; or by having some flag on the rlib that indicates that it should be linked dynamically. (I favor the latter because it's harder to get wrong, but either is fine.)

Another way to look at it: either way you're doing a dynamic link from the plugin to the exported plugin interface. It's just a matter of whether that interface is the binary itself or a separate .so. (And if you're badly worried about LTO, you can set up things so that at install time, the separate .so is just a facade over the main binary. glibc already basically does this with libdl, and I think Solaris libc actually does this with both libm and libdl.)

But the conceptual model is simper. You're no longer introducing the new concept of exporting things from binary crates, new attributes, etc. All you're doing is using the existing Rust mechanism of library crates with public items, and multiple crates being able to share a dynamic library crate. Optimizations are possible, but they must all fit the model.

And since the conceptual model is simpler and re-uses existing Rust features, there are fewer portability questions.

The host program, with common lib statically linked, would need to be recompiled, but the plugin would look for the same symbol names at runtime.

I am not sure you can rely on no ABI changes to some symbols if other parts of the same crate get touched. I think the only line you get there is the crate line (or extern "C", of course).

[retep998]

Just pointing out that I haven’t seen a single mention of Windows and how DLLs are handled there. That needs to be handled before this can properly move on to an RFC, in my opinion anyway.

[murarth]

@geofft

I don't think you need to do -C prefer-dynamic for every crate. You just need a way to mark this particular crate as dynamic. Its own dependencies would be static, other dependencies of the plugins would be static, etc.

Attempting to link to a crate with -C prefer-dynamic that was not itself built with the flag results in this:

$ rustc b.rs --crate-type=rlib,dylib
$ rustc a.rs -L . -C prefer-dynamic     # contains `extern crate b;`
error: cannot satisfy dependencies so `std` only shows up once
help: having upstream crates all available in one format will likely make this go away

The last two lines are repeated for every crate included in std. Adding -C prefer-dynamic to the b.rs line resolves the error. You're correct that dynamic linking is an option for a plugin design. It's just not so convenient right now. Perhaps a serious alternative to be investigated is the creation of a crate-level attribute to trigger -C prefer-dynamic behavior in all linked crates.

---

@retep998 pointed out in an IRC discussion on the topic that Windows does not permit statically linked symbols in a dynamic library to be overriden by symbols exported from a main program that is loading such library at runtime. I'm looking into setting up a Windows development machine to do some testing on this subject and see how it can be addressed.

Thank you both for your input.