Non-Send Futures

[tvallotton]

You know, reading this reminded me of this thread. It is a green threading library that implements a work stealing executor. It seems it cannot soundly implement work stealing because of this same local storage thing.

Maybe if it could be temporarily disabled for !Send types at runtime this could be sidestepped. Like, only being disabled in a runtime context.

[steffahn]

@steffahn

Now introducing capsules could in principle work the same. A new effect capsule (naming TBD) would mark code that can be executed in the a context of a Capsule::with callback. It cannot access legacy thread_local, but only a new thread_local with a CSync constraint. (Whether or not there are ways to still have these two thread_local versions merged into one is something I’m not sure yet - probably yes.)

In either case, the obvious downside is that there’s a lot of code to mark then. This same problem applies with const already, it’s slow to mark everything const as such, and it’s manual effort.

The analogy to multi-threading isn’t perfect. Unlike for that, for capsules, I think there’s another sound alternative to the effects-marker approach for achieving backwards compatibility:

Simply put, all existing types could be CSend + CSync. This includes … unfortunately … all generic parameters, so you’d need a ?Sized-style opt-out thing here, too. Yes, marking these cases is a considerable effort, too, but… it might be less bad than introducing an effect for this. I’m not certain.

If all existing types as CSend + CSync, it isn’t backwards-incompatible to require thread_local contents to be CSend.

How could all existing types be CSend + CSync, even though I’d proposed the opposite e.g. for Rc? The answer is following this argument:

@steffahn

Interestingly enough, as TSend and CSend stand for two orthogonal kinds of memory-boundary, one based on capsules and other thread-like boundaries, like tasks for futures; and one based on physical threads. I’m noticing that it’s also feasible to build … well let’s call the above a CSync-limited Rc … to build a TSync-limited version.

All existing !Send or !Sync types could be considered still bound to physical-threads, i.e. “TSync-limited”, i.e. they would implement !TSend and/or !TSync, but CSend + CSync could be fulfilled. Only new versions of Rc and the like would be “CSync-limited”. New versions of such types could probably be parametrized somehow, too, so instead of doubling all the API, it would “just” become more generic.

[fdasfe]

Alternative design: Only add TSend/TSync and make the current Sync and Send traits have the corresponding bound. This would be backwards compatible(because no existing traits would have their meaning change) while still allowing developers to relax bounds on functions from Sync/Send to TSync/TSend, or to add Tsync/Tsend impls to existing structs, like RC.

[RalfJung]

@steffahn

Simply put, all existing types could be CSend + CSync. This includes … unfortunately … all generic parameters, so you’d need a ?Sized-style opt-out thing here, too. Yes, marking these cases is a considerable effort, too, but… it might be less bad than introducing an effect for this. I’m not certain.

If all existing types as CSend + CSync, it isn’t backwards-incompatible to require thread_local contents to be CSend.

How could all existing types be CSend + CSync, even though I’d proposed the opposite e.g. for Rc? The answer is following this argument:

New auto / ? traits don't work entirely smoothly either, see this recent blog post.

[tvallotton]

Suppose we could somehow allow users to provide an allocation for TLS. Then async runtimes and stackful coroutine runtimes could provide an allocation of their own. TLS would become synonymous to the task_local! and coroutine_local! macros these libs provide. Would the problem go away like this?

I'd imagine that we would then introduce a more restrictive version of TLS that has guaranteed kernel threading semantics, and keep the old one as having user level threads semantics.