A plugin system for business applications

Continuing the idea mentioned here Pre-RFC: Runtime reflection

We really need a plugin system to be a valid contender in the area of "business applications".

I've tried to accomplish it with Any and TypeId relentlessly, it's simply not possible without support from the compiler.

I am aware of the performance and safety implications, however:

  • this plugin system would have barely any effect on the performance because it would be used at one of the top levels of the architecture of the application using it
  • it would be opt-in

What I need to achieve what I can achieve with, say, java modules (module-info.java):

  • allow the users of my application to drop .so files in a directory - the plugins
  • i'm fine if the plugins cannot be loaded and have to be recompiled because of ABI issues
  • ability to inspect what traits are implemented by the plugin, and call them, allowing the plugin to hook itself into the lifecycle of my application; the plugin still has to implement a trait "Plugin", so that I can establish initial communication with it
  • the plugins can depend on each-other. TypeId is not enough here, as other plugins are in differend .so files; this is the dependency inversion mentioned in the other thread; yes, it must be ensured that the dependency graph doesn't have cycles; I'm fine with doing that at runtime and showing the user an error with where the cycle is

This would really make rust a good contender for business applications in which companies are willing to pour money and it would help the adoption of rust.

Not having this in rust is to me the biggest showstopper, because rust's type system is amazing.

Speaking of amazing, this is also the reason why a plugin system like described above cannot be hidden behind a C compatible ABI - it would mean giving up one of the main reasons why a company would choose rust in the first place.

If I have to give up traits and generics and all the good stuff, just to have plugins, then rust loses against, let's say java.

You might say "then rust is not the right language for your projects", well yes, currently it isn't.

The question could also be: why don't we want to make rust attractive for more companies who could pour in the money, thus driving rust's adoption forward?

Addendum

I really don't know what it would be best approach in terms of implementation. Probably not reflection, probably some rtti just for this use-case, I really don't know.

But I've been lurking around this community for some years already and I'm sure there are enough bright people who can figure it out, if it's set as a priority.

Also, the plugins don't have to be .so files, they can be in any other format, including a new binary format.

PS: I'm one of those who votes on stackoverflow for "rust most loved", but doesn't say he uses rust professionally.

WebAssembly works very well for plugins, and allows you to write the plugins in an arbitrary language. It also lets you sandbox the plugins so they only have access to what you give them access to.

1 Like

Wouldn't it have the same problems as a C ABI, that I mentioned in the first post, except it's a different ABI?

1 Like

Sorry but that really sounds dishonest, it's a hijacking of technical counter-arguments in the direction of "you aren't WILLING to make Rust more attractive to companies".

That is pretty much not the case. People here aren't thinking "hey, we must really stop this guy from making Rust appeal to companies!" In fact I think the sentiment is the opposite, and we generally are happy if some company chooses to use Rust for making something nice.

You should also define what you mean by "business" applications which is a very generic term and I don't see how "business" or "non-business" bears relevance to a technical proposal. Probably you should elaborate on some concrete use cases which are impossible to solve without a reflection mechanism.

5 Likes

There's this crate that does exactly what you want i think: https://docs.rs/abi_stable/0.8.3/abi_stable/

But i haven't tried it yet, and no idea what the experience would be.

1 Like

WASM Interface Types, which I believe would be the eventual way to define plugin interfaces, are much more capable than the C ABI. It still looks like it’s a long way away though.

1 Like