[Pre-RFC] `libstd` hooks

[ethindp]

Hi all, Currently, libstd provides abstractions for various functionality that anyone would expect in any modern programming language for each supported operating system. However, this is, I feel, majorly inadequate and inflexible because every time we want to add a new OS or environment we need to modify libstd. Therefore, I propose the following change which may require major modifications to libstd's internals, hence me posting about it here.

libstd hooks

Instead of using per-OS implementations of functions directly within libstd, I propose the idea of a hook: a highly specialized library or implementation of certain functions that libstd requires for a particular operation, such as threads, processes, FS access, etc. Here's how it would work: whenever libstd requires the use of a system/environment-specific interface, such as access to the filesystem, it will call one of its registered hooks to perform the operation. The hook may be within the main application itself or it may be in a dependency. Hooks are registered by using the std::hooks module and by calling std::hooks::register(). (I'm not sure on the prototype of this function yet.) If a hook is not registered for a particular operation, the function that is being attempted returns immediately with a new error, std::hooks::HookError. Hooks could also be unregistered, making that operation inoperative again, (possibly) allowing smooth transitions from one environment to another (though I don't know if this is even possible), or allowing a program to seamlessly change the operation of a function, by, for example, changing a system call to an improved system call that works better. These libstd hooks would make it possible for libstd to be completely independent of the underlying runtime environment and would allow anyone to publish new environment hooks for any operating system or execution environment that exists (or may exist) on any of the supported architectures by Rust/LLVM. This would additionally have the advantage of (possibly) eliminating no_std entirely. Instead, all one needs to do is write their own hooks for libstd and they can freely use libstd anywhere. This, I think, would greatly speed up the development of libstd, since you don't need to register hooks you don't need, and thereby automatically excluding functionality. The compiler, then, could (and would) most likely optimize away code that immediately returns with error conditions due to hooks not being present, or code that is never called.

This idea would also make it easier to add new functionality to libstd that depends on the external environment. You just add a new hook and off you go. You don't need to care about whether an OS or environment supports it because if someone wants it, they'll hook it and use it.

Requirement: divide libstd into multiple crates

In order for this to work, we may most likely need to take the existing OS implementations and split them off into a separate set of crates (or maybe a single crate if that is desirable) to eliminate the dependency on an existing OS.

Alternative solutions

We could stick with what we have right now if this just isn't feasible.

Drawbacks

• More overhead: those who want to hook into new environments and get the full standard library experience will need to write a lot of boilerplate code to enable all of the functionality, which will require them to thoroughly understand the execution environment they're running in. This transfers the burden of "understand the environment" from the library team and anyone who contributes new OS support implementations to libstd to those who actually need the functionality, which some may find undesirable.
• Difficulty may be hard: libstd is quite large, as it imports libcore and liballoc, and a lot of functionality may need to be removed or altered for this to work properly.

Advantages

• Modularity and portability: libstd will be completely generic across any operating system or runtime environment, regardless of how it works underneath. Contributors or maintainers of the library will no longer need to care about implementation details of how a given operating system handles file permissions or network connections, for example, because that will be abstracted away.
• Simplicity: this change may make it far easier for people to understand libstd. It may also lower the barrier of entry to those who want to add new things to the library in future.

Example environment

The UEFI firmware environment is an excellent example of how this hooks mechanism could be utilized. Currently, standard library implementations are found on crates.io and other external registries. The problem with these implementations is that they are effectively duplicates of original work by the library team. This does not follow the DRY principal, and causes major code duplication. It would be much better if libstd was generic in this manner because then everyone would be using the same standard library and would have all the guarantees that entails, including receiving any updates whenever rustc is updated. Currently, maintainers of these custom libstd implementations must either copy or re-implement the functionality that libstd provides, depending on the license of the crate in question.

As an example, a UEFI environment hook crate for this hypothetical libstd would need to just register the FS hook using either EFI_LOAD_FILE_PROTOCOL, EFI_LOAD_FILE2_PROTOCOL, or EFI_FILE_PROTOCOL, depending on what functionality they want. This could similarly be applied to other standard library functions:

• Networking: EFI_SIMPLE_NETWORK_PROTOCOL, EFI_MANAGED_NETWORK_PROTOCOL, EFI_TCP4_PROTOCOL, EFI_TCP6_PROTOCOL, etc
• Processes: EFI_BOOT_SERVICES.LoadImage()->EFI_BOOT_SERVICES.StartImage()
• Environment variables: NVRAM variables via GetVariable(), GetNextVariableName(), SetVariable(), QueryVariableInfo()
• Threads: Emulation via EFI_MP_SERVICES_PPI
• Etc

What do you guys think of this proposal? I'm not sure about the definitions yet, but this is just something I wanted to bounce off you guys and see if its even possible to begin with since it was bouncing around my head and wouldn't go away.

[CAD97]

@ethindp

The compiler, then, could (and would) most likely optimize away code that immediately returns with error conditions due to hooks not being present, or code that is never called.

Either the hooks are static/compile time and optimized out, or they're dynamic and runtime swapable. You only get one or the other.

@ethindp

This idea would also make it easier to add new functionality to libstd that depends on the external environment. You just add a new hook and off you go. You don't need to care about whether an OS or environment supports it

Sounds like a good way to get std functionality that's not grounded in or well suited to exposing OS functionality.

---

The functionality to be generic over the OS exists: it's std and its implementation. While I could see making std more easily extensible to new targets out-of-tree, the in-tree support is definitely staying in-tree.

[josh]

I don't think it's an issue to have libraries like these for various targets as crates in the ecosystem.

If we want to avoid duplication, we should ideally do so by merging support for a target into std. Or otherwise, we should consider factoring libraries out for other crates to reuse. But we should share via libraries, not hooks. Please see Linux kernel design patterns - part 3 [LWN.net] on the "midlayer mistake" for why.

[ethindp]

This doesn't really seem applicable in this discussion, but I'll bite anyway.

The core thesis of that article is that instead of there being a midlayer there should be a set of rich library routines for devices to call upon. But the problem with that thesis -- or more aptly the problem with the applicability of that thesis to this particular situation -- is that there is only one Linux kernel. That is, everyone uses the same kernel code everywhere. A distribution or organization/custom setup may have many different custom builds of the kernel, such as linux-ck, linux-xen, linux-hardened, etc., but nobody completely rewrites the VFS or block layers, for example, for a particular filesystem or block device. They all use the same code. This is counter to what exists in the Rust ecosystem today, where you have at least 2 different libstd implementations in existence, both of which have no relation to each other. The second one I know of is uefi_std, which is essentially a complete rewrite of the majority of libstd for UEFI pre-boot environments (along with additions to it). Linux (and Windows, MacOS, FreeBSD, ...) work vastly different from this. In one for or another they all use kernel modules or other forms of dynamically loadable kernel code at runtime. Nobody has to contact Microsoft or Apple for the kernel source code of their OS so they can rip out the FS layer and completely rewrite it from scratch to accommodate a new filesystem. They just write a kernel-mode driver or use something like dokan/fuse to add one instead. In the case of network devices (or virtual devices like qemu-nbd) they just write the nbd driver, which is in the form of a kernel module the majority of the time, and then they load it when needed and unload it afterwards (or keep it loaded depending on how the user feels at the time). So essentially what I'm getting at is that a kernel module, kernel extension, etc., is, essentially, a kernel hook. It might be a hook in the VFS layer, or a hook in the network layer, or it might even be a hook into core kernel code. But a module is still a hook. What I'm proposing here is something kind of like that. The Rust libstd would define a core interface/contract for particular functionality. Applications or libraries that want to use that functionality would either hook the functionality themselves or use an already published hooking crate. This would unify all the libstd implementations going forward -- everyone would be using the same libstd code everywhere. You wouldn't need to rewrite the fs module for a new way of working with the filesystem or a new operating system, nor would you need to modify the net module for changing what TCP/IP stack is used under the hood.

I know what you were essentially getting at -- that adding new subsystems should be done in-tree. The problem is that I'm not proposing that hooks be considered new subsystems. The hook system itself would be a new subsystem and would, theoretically, augment or obsolete the current way things are done (depending on whether existing code must remain, and given what you and @CAD97 have posted, it would just be an augmentation to how things are done now, which is fine), but new hooks would not be new subsystems. Eventually a new hook could become an officially supported libstd hook, but it also may not be. But in the end, nobody would need to use custom and possibly unmaintained duplicates and repetitions of libstd. The hook might be unmaintained but that's far easier to manage, I'd say, than pulling in added functionality from libstd into a duplicate libstd and then having to go through and do wholesale modification to that functionality (possibly removing big chunks of the code) for it to work in your environment.

[mathstuf]

Some questions I have about hooking systems (in general and some specific to things here):

• Can they stack?
• If stacking is allowed and I install A, B, then C hooks, how does one uninstall B? Can it be removed independently or does C need uninstalled, then B, and C reinstalled?
• If they cannot stack, how do I guarantee that fs hooks provided by crate A don't conflict with process hooks provided by crate B?
• How can I guarantee that all hooks are properly installed?
• Is hook installation a pre-main thing or an explicit call?
• If an explicit call, how late can it be deferred?
• Can hooks be updated after fork()?
• Are they per-thread or per-process? For example, can I (ab)use hooks to mock out filesystem access for my tests?

[ethindp]

Can they stack?

No. At least, not for this. It wouldn't make sense to stack hooks to libstd. In general, that depends on whatever your hooking. Some hooking architectures doo allow stacking.

If stacking is allowed and I install A, B, then C hooks, how does one uninstall B? Can it be removed independently or does C need uninstalled, then B, and C reinstalled?

I'll answer this generally. This depends, again, on what your hooking. Some systems will give you an identifier that you can store to access the hook or remove it independently from all others. Some systems require you to remove any hooks above the one you want to remove before you remove the one you actually want to remove (I know, that sounds confusing).

If they cannot stack, how do I guarantee that fs hooks provided by crate A don't conflict with process hooks provided by crate B?

wouldn't FS hooks provided by crate A be independent of process hooks -- or any other hook type for that matter -- provided by any other crate? The only time you'd have to worry would be if crate B provided FS hooks and registered them without you knowing. Then you'd need to realize that and force the hooks to be how you want them. But this could also be resolved by requiring that if you want to register a new hook you have to unregister the existing hook first, which could require knowing a special key for example.

Edit: The special key would be returned by the hook registration function. At least, that's what I was thinking.

How can I guarantee that all hooks are properly installed?

The right way, IMO, would be to verify that the register() method returns success each time, and to bail out if any hook registrations fail for any reason. If the function succeeds, you know that hook registration completed.

Is hook installation a pre-main thing or an explicit call?

I think it would be better for that to be explicitly called, if only because it would ensure that developers know exactly what is being registered and when, instead of some crate doing things behind their back. You could do it in pre-main but that doesn't seem like a good idea given that it would be easy to do something undesirable there that the developer wouldn't notice.

If an explicit call, how late can it be deferred?

It can only be deferred until you use functionality from libstd. If you bypass libstd and do your own thing via something like the syscall crate or the winapi crate, you can defer it as long as you like. But if you register only basic hooks for example -- like IO primitives for the println! macro to work properly -- then you can definitely defer registration until you actually need the functionality.

Can hooks be updated after fork()?

I can't imagine why not. From what I understand of fork(), that creates another process as a child process of the parent, and hooks would be per-thread, so you would start with a clean slate, if I understand how that works properly.

Are they per-thread or per-process? For example, can I (ab)use hooks to mock out filesystem access for my tests?

Per-thread would probably be best. Then you could definitely mock FS access in tests as well as being able to use FS access in different ways (e.g. networked FSes in one thread and local in another). If per-thread hooks aren't possible (e.g. on UEFI where code runs on different processors) then it would be per-process. But that dual model might be really difficult to achieve.

I know I didn't answer all your questions regarding hooking systems in general, but its difficult to do that when the way a hook works is extremely dependent on how its implemented.

I appreciate the feedback you guys have given thus far, and I hope we can keep refining this. Like I said in my OP I thought I might as well bounce this off of you guys since it wouldn't leave me alone.

[bjorn3]

@ethindp

wouldn't FS hooks provided by crate A be independent of process hooks -- or any other hook type for that matter -- provided by any other crate? The only time you'd have to worry would be if crate B provided FS hooks and registered them without you knowing. Then you'd need to realize that and force the hooks to be how you want them. But this could also be resolved by requiring that if you want to register a new hook you have to unregister the existing hook first, which could require knowing a special key for example.

The process hook may for example access /proc/ on some systems. If the fs hook hides /proc/, it isn't compatible with the process hook.

[bjorn3]

@ethindp

I can't imagine why not. From what I understand of fork() , that creates another process as a child process of the parent, and hooks would be per-thread, so you would start with a clean slate, if I understand how that works properly.

fork copies the current process including all memory, the stack of the current thread and the thread local storage. It doesn't copy other threads. After a fork many things aren't allowed because another thread that isn't copied over to the new process could currently be holding a lock which will thus never get unlocked.

[josh]

@ethindp First, I do think that UEFI support should be in the standard library, and I would love to see that happen. (And now that the Target Tier Policy is in place, we should start figuring out what it would take to raise the UEFI targets to a higher tier.)

But that's not the point of that article. I was making the point that even if we continue to have std replacements out-of-tree, and if we want them to be able to share code with std, we should do that by factoring out parts of std for those crates to share and call, rather than be called by.

The pattern normally used to avoid the "midlayer mistake" is that rather than calling a hook, you provide a library of all the implementation pieces, and then let a higher level library call or not call any of those pieces, and insert anything it needs to in between any calls. Much more flexible.

[kornel]

Global runtime hooks are scary (and for the record, I don't like the hooks that Rust already has).

It's an ability for any function anywhere to indirectly change behavior of other functions elsewhere. I can imagine people trying to do ugly hacks with it: you don't want another function to open a file? Patch File::open to block it!

There's no good way to implement it well for threaded programs. If it's global, then it costs in thread synchronization, and has risks of global mutable state. If it's per thread, then it will have overhead of thread-local storage, and have lots of gotchas when Rust code is used in thread pools, e.g. if you install a hook for your current thread and then run code that uses rayon or tokio mt runtime, that hook may or may not have effect depending on how these runtimes schedule the work.

I use Rust on platforms that are "natively" supported by libstd, so for me such hooks have zero appeal for their intended purpose, and would be just a source of concern about their potential misuse.

[ethindp]

@bjorn3 thanks for that, I didn't know it was that thorough. That would be a huge problem then. @kornel Makes sense, I was hoping that we might be able to come up with a way of mitigating that, but considering how difficult that would be, the only way to do so would be to not have them at all, or place severe limitations on them (e.g. libstd registers OS-specific hooks if on a non-bare metal target, and once registered they can't be changed). Of course, such a mitigation would effectively make the entire idea pointless. @josh Oh, I understand now, and that's actually a good idea. I would appreciate it if you could link me to more discussion on the right way to avoid the midlayer mistake if they're available. Such knowledge would come in handy for all systems that utilize layering functionality like Linux does. Also, I, too, would love to see UEFI support in libstd. The UEFI specification provides all the functionality we need (and then some), so with UEFI support could -- in theory -- come more modules, maybe. (I know that putting something in libstd is deliberately difficult, and I know why.) As an example, UEFI provides quite a large number of networking protocols for raw packets, wi-fi, bluetooth, VLANs, DHCP, TCP, and UDP, as well as security functionality, human interface infrastructure (HII) support for displaying GUIs, and so on. (Most of that could be put under std::os, though. The name os would be a bit misleading -- UEFI isn't an operating system -- but it seems pointless to add a fw module just for UEFI support.) Should I create a new topic to specifically discuss UEFI support or is it okay if this continues here? Its clear now that my original idea in the OP either won't work or would require a lot of work to get right. Though it might happen, just (not) in its original form. But that's definitely okay, and you guys have raised many points I hadn't considered.

[josh]

@ethindp

Oh, I understand now, and that's actually a good idea. I would appreciate it if you could link me to more discussion on the right way to avoid the midlayer mistake if they're available.

I don't know a good article other than the LWN one I linked to.

In general, one example is that if you have subsystem functionality that needs to do do_thing_1(), do_thing_2(), and do_thing_3(), and a higher layer that wants to call into that subsystem, you shouldn't create hook points like between_1_and_2 or between_2_and_3 or modify_2_behavior. You'll end up with a never-ending need for different kinds of hooks. Instead, you should provide library functions to do each thing, and then your higher layer can look like this:

do_thing_1();
my_between_1_and_2();
my_modified_thing_2();
my_between_2_and_3();
do_thing_3();

That does mean that you may have multiple higher layers that each need to call some of the same library functions in the same order, but you can factor out patterns from those higher layers rather than trying to modify an inflexible lower layer.

In the context of the Rust standard library, the equivalent would be to factor out helper functions and helper crates that make it easier to reimplement pieces of the standard library with as little duplicated code as possible.

@ethindp

Also, I, too, would love to see UEFI support in libstd. The UEFI specification provides all the functionality we need (and then some), so with UEFI support could -- in theory -- come more modules, maybe. (I know that putting something in libstd is deliberately difficult, and I know why.) As an example, UEFI provides quite a large number of networking protocols for raw packets, wi-fi, bluetooth, VLANs, DHCP, TCP, and UDP, as well as security functionality, human interface infrastructure (HII) support for displaying GUIs, and so on. (Most of that could be put under std::os , though. The name os would be a bit misleading -- UEFI isn't an operating system -- but it seems pointless to add a fw module just for UEFI support.) Should I create a new topic to specifically discuss UEFI support or is it okay if this continues here?

I would suggest creating a separate topic for improving UEFI support.

In general, I don't think we'd want to add much UEFI-specific functionality to the standard library; for instance, we're unlikely to add things like wifi, bluetooth, VLANs, or DHCP. That kind of functionality belongs in a separate crate; for instance, a uefi_dhcp or uefi_wifi crate.

However, we should absolutely add UEFI support for TCP, UDP, and anything else we have an established interface for. Sockets, filesystem support, almost all of std should be an option. (Threads will be a challenge, but potentially doable.)