Placement NWBI<- FAQ (New/Box/In/Left Arrow)

[pnkfelix]

ugh the formatting is screwed up in various ways.

But the content is still essentially there. I’ll be back to try to correct the formatting in a few hours

[brson]

Great read. Thanks.

[kristof]

Good write up! A few comments.

There is clearly an advantage to being able to use my_vec.front <- item over my_vec.push(item), but the change in form is slightly jarring. An implementation detail has become a clear syntactic change even though the semantics are roughly the same. I know that rustc and llvm in general cannot optimize that kind of thing away but perhaps we could introduce a way to mark methods and functions as “placer” functions? Any use of the parameter would not be valid until the item has been placed/initialized within the body. And in fact, perhaps the <- syntax could be used for this within methods.

So specifically what I’m wondering is, is it possible perhaps to add a pragma #[unitialized(arg1, arg2, arg3)] to require the arguments listed in the method be placed before used? This would shelter users from implementation details. It would also change the order of side effects in a significant but in my opinion backwards compatible way.

Of course reimplementing vector.push(item) in terms of an uninitialized marker pragma would break builder patterns, so many of these should be new, “in place” versions of methods, in the same way we have mut versions. vector.push_ip(item) as shorthand.

Admittedly, hidden semantics are not the best choice, but I want to put this idea out here because I really think similar things should look similar, and pushing a value with a temporary stack allocation, and pushing a value without one, should both be transparent. Shouldn’t they?

Alternatively, we already have a very robust notion of lvalues and assignment. Currently, place = value is valid syntax where place is &mut. What’s wrong with extending the notion of lvalues to include places? Then instead of reference = (place <- value) we have reference = (place = value), where assignment returns the reference if it supports it as was mentioned in the RFC.

I am going to reiterate this because I think it is important. The fact that chaining assignment is currently possible but useless, the fact that we have such a flexible notion of lvalues but are not leveraging this concept for an ambitious and serious addition to Rust semantics, and the fact that rather than using existing syntax we have to add new syntax concerns me. I think reinvestigating the possibilities of overloading lvalues to include places is a lot more appealing.

The potential for introducing garden paths is a big deal, to me. To preserve left to right reading, there should be an additional left associative -> operator that does exactly what one might expect.

let reference = value -> some_vec.place();

Obviously this was the advantage of the box in place syntax, but this also means that no new keywords have to be introduced. It’s consistent with the other version. I think it’s quite common in most language constructs with this kind of directional semantics to offer two versions of an operator so one is not preferred over the other.

This is not about changing the syntax. I am just proposing to add a little more to it. I don’t think this will be ambiguous with type context -> because even when writing out closures, there is a clear and unambiguous difference between the places where one might expect an expression and where one might expect a type.

[pnkfelix]

perhaps we could introduce a way to mark methods and functions as “placer” functions? Any use of the parameter would not be valid until the item has been placed/initialized within the body. And in fact, perhaps the <- syntax could be used for this within methods

How would this work? Do you mean that the evaluation of the argument expressions would be delayed until … sometime in the middle of the body of the method being called?

Bascially I don’t really understand how the pragma you suggest would be implemented, unless you really do mean that it be some sort of call-by-name evaluation strategy

[kristof]

I do mean that, but in retrospect it isn’t a good idea.

I still stand by overloading =, however. But seeing as this RFC has already been accepted, I missed the final comment period and I think that’s fair.

[yigal100]

This topic is very similar to objective-C’s alloc and init methods:

SomeObject *myObject = [[SomeObject alloc] init];
// Or with parameters
SomeObject *myObject = [[SomeObject alloc] initWithFoo: foo];

Personally I lean towards the &uninit semantics where an allocator generates new uninitialized places (like sources in a directed graph) and the placement protocol is used for initialization of those places. I wonder if the pointer type has to be user facing (i.e. in the language) or just internal in the compiler and used for static analysis. In C# AFAIK the compiler uses analysis to track first initialization and prevents compilation of erroneous programs.

void foo() {
    int a;
    if (condition) {
        a = 5;
    } else { 
        a = 0;
    }
   // do something with a
}

foo above will not compile without the else branch.

[pnkfelix]

I still stand by overloading =, however. But seeing as this RFC has already been accepted, I missed the final comment period and I think that’s fair.

Well, to be fair, the real point where the ship sails and there’s no going back is when a feature is ungated and stabilized.

We haven’t even landed a patch yet for gated <- on the unstable release channel, let alone have it on the beta or stable channels.

So, regardless of my own opinion as to what path is best, there is certainly room for course correction in the short term as we go here.

[nikomatsakis]

Personally I lean towards the &uninit semantics where an allocator generates new uninitialized places (like sources in a directed graph) and the placement protocol is used for initialization of those places. I wonder if the pointer type has to be user facing (i.e. in the language) or just internal in the compiler and used for static analysis.

From my point of view, a struct implementing the Placer trait IS an uninitialized pointer, but a rich one, that understands cleanup and so forth.

[pnkfelix]

Some questions that need to be added to the FAQ (see Rust Issue 27779 and RFC Issue 1315):

• Q Why are is there both a Placer and a Place trait? Can we not make do with just one?

• Q How is fallible allocation supposed to be handled?

• Q Are we really planning to have Placer global constants (e.g. HEAP, RC, from some of the original discussions of the protocols).

[whataloadofwhat]

Shouldn’t the Place trait (or at least InPlace) be an unsafe trait if the placement protocol is relying on it returning a valid, non-null pointer?

[pnkfelix]

Shouldn’t the Place trait (or at least InPlace) be an unsafe trait if the placement protocol is relying on it returning a valid, non-null pointer?

Hmm, the original API designs I think predated unsafe traits … now that you mention this, I can’t think of a good reason not to do it.

[whataloadofwhat]

After I posted that I realised that all the protocol relies on is the Placer to return a valid Place, so if anything should be an unsafe trait it should probably be that. Sorry about that, I was a little confused about why 3 traits were necessary, but I think I understand now.

[rkjnsn]

It seems like making Placer unsafe would prevent creating a wrapper around a struct implementing Placer in safe code. E.g., say I want to create a custom type that I want to support placement into. It would be nice if I could, without using unsafe, have an internal Vec or TypedArena (depending on the desired semantics) to which I delegate my Placer::make_place implementation.

[pnkfelix]

It seems like making Placer unsafe would prevent creating a wrapper around a struct implementing Placer in safe code. E.g., say I want to create a custom type that I want to support placement into. It would be nice if I could, without using unsafe, have an internal Vec or TypedArena (depending on the desired semantics) to which I delegate my Placer::make_place implementation.

yes I agree.

I think the original suggestion of making Place itself an unsafe trait will cover the wrapper use-case, right?

[rkjnsn]

It would seem so to me.

[rkjnsn]

I was thinking about this, and I realized that while making Place an unsafe trait does allow the wrapper use case with safe code, it does make the abstraction leaky. That is, without being able to also wrap Place, code using the wrapper will be able to see that the Place returned is a VecPlaceBack (or whatever), and treat it as such. Thus, changing the inner container could technically cause breakage.

One way to solve this problem would be to introduce some kind of abstract associated type to go along with the proposed idea for abstract function return types. This would prevent the code using the wrapper from relying on the specific type implementing Place.

[glaebhoerl]

One way to solve this problem would be to introduce some kind of abstract associated type to go along with the proposed idea for abstract function return types.

FWIW, I think this would be a natural extension of the abstract type feature discussed in/on the new RFC by @Kimundi (that is, allowing an associated type to be defined as an abstract type in an impl). Basically the same idea is described in the “Translating Abstract Types” section of the paper ML Modules and Haskell Type Classes: A Constructive Comparison (longer version). (I suspect this may also be the same thing @eddyb and @aturon were discussing before I had even understood what it’s about - or in @eddyb’s case, implementing.)

My head is too small for the placer protocol to fit inside, but a different direction that may also be applicable would be, considering that our “associated types” correspond to Glasgow Haskell’s “type families”, to add the equivalent of data family as well, which in Rust parlance would probably be called “associated structs”. In that case there would be a direct 1-to-1 mapping between types implementing the trait and their associated structs (or enums), instead of an associated type which maps onto a previously-declared type. (I’m not sure whether this obviates the “changing the inner container” possibility.)

[vadimcn]

Having recently re-read this FAQ, I’ve developed a couple of new-ish thoughts :

Ignoring the issue that this relies on return-value optimization actually kicking in (which @pnkfelix found to not be generally reliable, and in any case we wouldn’t want debug builds to differ so much in runtime behavior that they could e.g. stack overflow with ease)

I don’t see how this could work without relying on RVO. What if the VALUE is a function call? IIRC, the idea was that large values are returned by pointer into the caller-allocated space anyways, so all that box expression needs to do is ensure that pointer passed to the callee is the one returned by the placer. Why wouldn’t this work with closures?

the most obvious place where this completely falls down is that it does not do the right thing for something like this:

let b: Handle<T> = placer <- try!(run_code()

because under the once-function desugaring, that gets turned into something like:

placer.make_place(|| -> T { try!(run_code()) })

Yeah, but if run_code() returns Result<T,E>, how can we avoid creating a temporary value for it? We can’t put it into the memory allocated by the placer, since Result<T,..> is larger than T (in most cases), so it’d have to go on the stack, which kinda defeats the whole purpose of using placement operator…

[rkruppe]

Yeah, but if run_code() returns Result<T,E>, how can we avoid creating a temporary value for it? We can’t put it into the memory allocated by the placer, since Result<T,…> is larger than T (in most cases), so it’d have to go on the stack, which kinda defeats the whole purpose of using placement operator…

This point is not about space overhead. The point is that the try! expands to control flow (specifically a return in the error case), and therefore putting it into the closure is incorrect: It won’t return from the function that defines b, it only returns from the closure. And what’s worse, it doesn’t even compile because the closure must return T but in the error case try! would return Result<T>.

[vadimcn]

This point is not about space overhead.

I disagree. Elimination of temporaries is the whole point of the placement operator. Otherwise, you might as well write it as Box::new(try!(make_value()))