Associated Statics - a way to have high performance, (mostly) statically dispatched event busses


#1

I recently decided to look back on an event bus I wrote a long time ago. I had to do things like this back then:

/* some things omitted for simplicity */

pub trait Event {
  fn use_internal_data<F, T>(f: F) -> T where F: Fn(&InternalData<Self>) -> T;
  fn use_internal_data_mut<F, T>(f: F) -> T where F: Fn(&mut InternalData<Self>) -> T;
}

pub struct EventMetadata<T: Event> {
  handlers: Vec<EventHandlers<T>>,
}

impl<T: Event> EventMetadata<T> {
  pub fn new() -> EventMetadata<T> {
    EventMetadata { handlers: vec![] }
  }
  fn get_or_create_handlers(&mut self, bus: EventBus) -> &mut EventHandlers<T> {
    self.handlers.get(bus.id()) /* also some logic to grow the vec and stuff */
  }
}

(Yes, I now realize use_internal_data_mut can be exploited to set/replace the InternalData.)

The way this works is that you’d create a static EventMetadata (using lazy_static) for every impl you make, and pass it on as needed in places where it’s relevant. The goal is to provide (as close to) static dispatch (as possible) - with the EventBus code mostly just calling the EventMetadata functions, all usages of the Event basically boil down to getting the bus ID, and getting the vec entry for it. This is much faster than doing some unnecessary hash operations (which you would have to do if you were using, say, AnyMap).

However it would be nice to be able to do it like this instead:

pub trait Event {
  static INTERNAL: private::EventMetadata<Self> = /* something here, perhaps something with lazy initialization */;
}

mod private {
  pub struct EventMetadata<T: Event> {...}
}

And then the user doesn’t need to provide EventMetadata themselves (or be able to look at it), which can avoid quite a lot of confusion (as some might think EventMetadata lets you associate additional data to an event instance, while it’s metadata about the event type). This also simplifies the monomorphization case (as monomorphization can generate a static for every monomorph), so if you had a VecEvent<T>(Vec<T>) for example you wouldn’t need to impl a separate Event for every VecEvent<T> you wanna use. (or you might not be able to, if the VecEvent<T> is from another crate!)

The reason you can implement the Event trait even tho you can’t use eventbus::private::EventMetadata is because when using the default value the impl can be omitted. So you don’t ever reference the INTERNAL static in your trait impl, thus not having to reference its type either.


Pre-RFC: Associated statics
#2

For context, I’m trying to avoid things like this:

https://mozilla.logbot.info/rust/20180303#c14394317
https://mozilla.logbot.info/rust/20180303#c14395430
more importantly, this kind of confusion: https://mozilla.logbot.info/rust/20180303#c14395745 (cont https://mozilla.logbot.info/rust/20180303#c14395821 and https://mozilla.logbot.info/rust/20180303#c14395901 )


#3

It seems like we could support associated statics readily enough. (And, equivalently, generic statics and constants.)


#4

I’ve been told this doesn’t work with dynamic linker very well, but I’d really like this to be possible. it’d be very useful.


#5

I guess you meant here


#6

yep. maybe we should consider getting our own dynamic linker!


#7

Not that this helps, but it works pretty much everywhere except for Windows.


#8

is it possible to implement a workaround on windows?

a bootstrap dynamic linker bundled with the rust-based dll that then loads a rust dynamic linker to fix up the dynamic linker?


#9

I find it super-confusing, non-intuitive, and annoying, that associated items are not available for all kinds of types.

For example, one can add associated consts to both traits and structs, but one cannot add associated types to structs.

For consistency, associated statics should be usable for enums, structs, tuple structs, unions, etc. as well, not only traits, so that I can write:

struct A;

impl A {
    static FOO: X = Y;
}

I wish the whole associated items stuff would receive some consistency love before making it more complicated with GATs, associated statics, etc.


#10

This is due to technical challenges (awaiting lazy normalization) but it will happen eventually and it has been accepted by an RFC.

As for associated statics, if you can ensure that each static when monomorphized at a certain type has a single memory address, then I have no objections to adding it for consistency. But you will need to write an RFC to plug that consistency hole.


#11

I think I’ve figured something out here:

https://cybre.tech/SoniEx2/rust.eventbus/src/branch/master/src/lib.rs#L113-L158

this is painful but I think something like this could work? even on windows?

basically, as part of monomorphization, we generate something similar to this. except the compiler would be able to heavily optimize it:

  • the compiler can coalesce all occurrences of the same generic static into a single static ID: AtomicUsize/static ID_INIT: Once pair. this is way more efficient than these macros.
  • (for best results) these “ID” statics would be part of the monomorphized functions, not of the caller. (if not possible, might require small tweaks to ABI calling conventions)
  • calling it isn’t an ugly PITA, and you’d actually be able to see where the busses etc are being used. compare:
    let mut event = foo;
    register_hook!(&bus, 0, MyEvent, |event| foo);
    post_event!(&bus, MyEvent, &mut event);
    // vs
    let mut event = foo;
    bus.register_hook(0, |event: &mut MyEvent| foo); // type inference doesn't work here, heh
    bus.post_event(&mut event);
    
    the main thing here is the use of bus.whatever instead of macros. it’s almost like bus. acts as a sort of “indentation” as far as the eyes are concerned, so it becomes a lot easier to read. (not sure how to describe this any better)

note that the existing requirements of static variables (namely that the contents must be 'static, Send and Sync) fit quite well with the requirements of Any.


#12

The key issue with Windows is combining monomorphization with dylibs. If a generic symbol is monomorphized the same way in two dylibs, there is no way in the Windows linker model to deduplicate that when both dylibs are loaded in the same binary.


#13

What if instead of implementing a costly solution, we just provide a workaround?

Let the libs figure it out, but help them do it.

See my other thread about static generics (in functions), etc? That would help.


#14

@Soni I’ve read all of your other threads, but I didn’t see anything in them that makes a difference here. They seem to all be proposing different syntaxes for declaring generic statics (and some other stuff), but none of those threads even seemed to mention the fundamental problem that generic statics and dylibs are mutually exclusive, much less propose any sort of solution or workaround.

Do you have some sort of workaround in mind that you’ve been trying to propose but failing to articulate?


#15

You have a shared dylib that works as a registry for the other dylibs. The other dylibs touch this registry and manipulate their statics to point into this registry. You then have what I call a “dynamic linker hack” - you tell the OS dynamic linker to load a dynamic linker of sorts that’s used exclusively by Rust, and use that instead.


#16

I’m implementing a crate to add some kind of reflection to Rust, and associated statics would be very helpful. However I’m not clear about the semantics you propose here.

What if you implement the trait (containing a static field) for a type parameterized with lifetimes.

impl<'a> for A<'a> {
  static FOO : X = Y;
}

Does that mean that you would have one X instance for each possible lifetime ? This is how I would interpret the semantics… Unless you only consider the types after having the lifetimes removed ?


#17

Any static must have T: 'static. Associated/generic statics do so, too. Since any associated static can be desugared into a generic static, it requires the type to be 'static (for now, sadly).

This is assuming we don’t get any sort of HKTs. See also.