Pass `&` references by value

[noamraph]

That’s exactly what I mean.

[noamraph]

It might make sense if those were the rules, but I believe that the current rules are that everything is passed by value, no matter its size, and if a type has a destructor it is “moved”, that is, you can’t use it again from the calling function (and the called function is responsible for calling the destructor).

[noamraph]

Indeed, according to this idea, you’d copy all Foo's inner values instead of pass a pointer to foo. This would mean more work when calling the function, but less work inside the function, as it would be able to access the members directly without an indirection. I think that in most cases this would be a performance gain. If a type is large and isn’t accessed much, then you’d use Ref<T>.

Perhaps it would make the most sense to construct a Ref<T> by ref t, just like you construct a Box<T> by box t. However the ref keyword might already by taken.

[shadowmint]

I’ll have to bow out I guess.

according to this idea, you’d copy all Foo’s inner values instead of pass a pointer to foo

Sounds exactly like you’re derefferencing the pointer and copying the (potentially huge) memory block it points to.

…but apparently you’re not. So disregard me I guess~

[noamraph]

Perhaps I wasn’t clear enough. Anyway, I meant that you’d copy all the size_of<Foo>() bytes of foo into the function. If some of the bytes are pointers, only the pointers would be copied, not the data they point to. So you won’t copy any huge memory block.

[cptroot]

What if it’s a large complex structure, or has a fixed length array in it? What does passing &Foo mean now?

This would be my one concern with indiscriminately copying Foo to the function. Making choices about whether to pass a (possibly large) fixed size array sounds like a hard problem that we should leave the programmer to decide. If they can’t use &Foo as a reference type, then what would be the alternative to force the struct to be passed by reference?

[noamraph]

I agree that the programmer should be able to decide whether he wants to pass by value or by reference. Currently, if a type has a destructor, the programmer is forced to pass by reference, even if it doesn’t make sense, except for two special types: a string and a vector.

I propose that &foo would create a view of foo by copying its bytes, and ref foo would create a view by passing a pointer. I’m suggesting the shorter syntax for passing by value because I think that in most cases it would result in better performance, and because I think it’s more consistent with the rest of Rust.

[Ericson2314]

It sounds like you want a shallow copy of objects where any owned ptrs they contain are converted to borrowed ptrs.

Vaguely related: is the plan for Vec<T> to be redefined with a Box<[T]> once we get that?

[mbrubeck]

Programmers can (sort of) opt in to pass-by-value without move semantics, by implementing and using the Clone trait. This isn’t the same level of support/sugar that your proposal would offer, but it does give some control to the programmer when it’s really needed.

[tbu]

This doesn’t work for things with destructor – Rust insists that pass-by-value things are destructed at the end of the function.

[steven099]

This sounds like it’s trying to reintroduce passing modes. I don’t even know what the sigils stood for at the time, I just remember it being deemed too confusing. It might be nice to be able to say “borrowed by value” or “borrowed as word sized, indirecting if necessary”, but I figure it’s a lot of complexity to add just to avoid indirecting through a stack pointer sometimes, and anything short of that means introducing “smart” behaviour that can’t be opted out of.

[tbu]

You already have that smart behaviour which cannot be opted out of when passing things ‘by-value’ or returning things.

[noamraph]

I want to explain my idea again, in the hope that I’ll manage to make it clearer this time. I also have a refinement of the idea, which makes it applicable without any additional syntax, and immediately allows to throw the special (and confusing) types &str and &[T] away, while making the language more performant.

First, the explanation. Every type Foo has its size_of<Foo>() bytes. Unless some of them are of type Unsafe (usually they aren’t), whenever a view &foo is alive, the compiler makes sure that those bytes are not mutated. If they are not mutated, it means that it doesn’t matter whether you have a pointer to those bytes, or if you have a copy of those bytes. This means that the question of whether the type &Foo is a pointer or a copy of Foo's bytes is just an implementation detail. Take for example the code

let v = &foo;
let x = v.x

Currently the compiler translates it to

Foo* v = &foo;  // v is a pointer to foo
int x = (*v).x; // to access v.x you need to follow the pointer

If &Foo were implemented by copy, the code would be translated to:

Foo v = foo;  // to create v you copy foo's bytes
int x = v.x;  // you can now access v.x immediately

As you can see, it is only a matter of code generation. This would be entirely transparent to a Rust program, unless it checked what is size_of<&Foo>().

 

Now, based on this, let me propose a variant of the original idea, which doesn’t require any new syntax.

• Types could be decorated, to affect the implementation of their view.
• If a type isn’t decorated, its view is a copy of its bytes.
• A type could be decorated by #[ref_view], which would make the compiler implement its view as a pointer. Types which include Unsafe members must be decorated with #[ref_view].
• Other types could be decorated to specify fields which are not copied into the view. For example, Vec would be defined like this:

#[not_in_view(capacity)]
pub struct Vec<T> {
    ptr: *mut T
    len: uint,
    capacity: uint,
}

• Fields listed as not_in_view must be private, so code outside the type’s module won’t be bothered by the discrepancy between the type and its view.

This has a lot of benefits:

• No extra syntax to the language, just some more work for the machine code generator.
• No need for the special types &str, &[T], which are confusing a lot of people. There would be just String and &String, Vec and &Vec.
• No need for dynamically sized types (I confess that I didn’t manage to really understand those, so maybe I’m wrong here.)
• Virtually all existing code would continue to run, and I believe most of it would become faster. If some code becomes slower, you can always add #[ref_view] where it’s needed to get back the original performance.

Did I manage to explain myself? What do you think of this idea?

[P1start]

@noamraph It’s an interesting idea, but I see this as two largely unrelated proposals:

• Add an optimisation to & pointers to copy for small Copy types
• Remove &[T] and &str and just use &Vec and &String instead. (This is optimised with #[not_in_view] so that the capacity doesn’t need to be included in references, but AFAICT this isn’t strictly necessary.)

The first proposal makes sense (and is probably a decent idea IMO) but the second one isn’t really practical, unfortunately—when allocation is not available, Vec cannot be used. Also, DST is still needed—trait objects with custom pointers (like Rc<Show>) need DST to work, and that isn’t fixed by your proposal.

[tbu]

Those types don’t need Copy, having no Unsafe elements in them suffices.

[rkjnsn]

when allocation is not available, Vec cannot be used.

I don’t follow this. The only operations that require allocation or access to “capacity” are ones that modify the Vec, but you can’t do those with a &Vec, anyway. You can still read and iterate through the elements, and even clone the Vec if you need a mutable copy.

Unless you mean Vec couldn’t be made Copy, but it wouldn’t have to be. The only restriction would be that it could not contain internal mutability (no Unsafe elements, as tbu mentioned), which it doesn’t.

[rkjnsn]

How would Vec and String operations that return a view into the object be implemented with your proposal?

[noamraph]

@P1start You are right that the idea can be separated into two proposals, but I don’t think they are unrelated. The second, for dropping the special types &str and &[T]` requires the first, and I think it makes the first idea very compelling.

As @tbu and @rkjnsn noted, there’s no need for Copy, the only requirement is that there won’t be any Unsafe elements.

As @rkjnsn said, I don’t think there’s a problem with using &Vec and &String in code without allocation. String and Vec could be defined in core, along with the operations on &Vec and &String which don’t require allocation. Methods which create a new Vec or String, which require allocation, would be defined in std, where allocation is available.

@rkjnsn Operations that return a view into an object could be implemented just like they are implemented now, using mem::transmute. For example, the current implementation of Vec<T>::as_slice is:

fn as_slice<'a>(&'a self) -> &'a [T] {
    unsafe { mem::transmute(Slice { data=self.as_ptr(), len=self.len }) }
}

If my proposal is accepted, you’d just need to replace &'a [T] with &Vec<'a, T> and it would work.

[P1start]

I still don’t see why the first proposal is necessary for the second. Right now &str is represented as (ptr, len). With your second proposal, &String is represented as str where str points to (ptr, len). Sure, this is less efficient, but I don’t think it’s strictly required. And if the & optimisation is required, how do you handle &mut [T] with your system (because there is no such optimisation for &mut)? The #[not_in_view] optimisation also seems unnecessary (but obviously useful).

With those two additions (#[not_in_view] and by-value views/references), &String is now significantly different from String. It doesn’t have the capacity field, and it’s not actually a reference to a string as the name suggests. What you’ve effectively done is renamed &str to &String, which is both a good and a bad thing. Good: it’s less confusing—there is only one ‘base’ string type. Bad: it’s inconsistent—&String is basically just a String but without a capacity field, and when allocation is not available, String is not usable, while &String is. At the same time, you’ve added a new pointer type (or really, renamed one and added another), and Ref<'a, T> could easily turn out to be more common than the new &'a T and is counter-intuitive to what people expect (& is a reference (effectively a pointer AFAIK) in C++, and is used to construct a pointer in C/C++).

I can see the advantages, but I’m not sure they outweigh the disadvantages. It’s certainly a very interesting and original idea, though.

[noamraph]

I can see the advantages, but I’m not sure they outweigh the disadvantages. It’s certainly a very interesting and original idea, though.

Thanks! That’s very nice to hear!

I still don’t see why the first proposal is necessary for the second.

You are right that it’s not strictly required, but as Rust is a systems programming language, no one would agree to add an unneeded level of indirection just for the sake of consistency. So keeping the same performance is very important.

I haven’t thought about &mut [T]. I wonder how frequently it’s being used. Perhaps it can be replaced with a struct that contains a &mut T reference to the first element and a length? If it can’t, then my proposal doesn’t handle &mut [T]. Too bad!

you’ve added a new pointer type (or really, renamed one and added another), and Ref<'a, T> could easily turn out to be more common than the new &'a T and is counter-intuitive to what people expect

I have dropped my idea about Ref<'a, T>, to avoid extra complexity in the language. So now &Foo would always be a view of Foo, and would always behave the same way, but may be implemented either as a value copy or as a pointer, according to the type attributes. I agree that it would make the implementation more opaque than it is today. It can be seen as an optimization - think about &Foo as a pointer to Foo, and if the author of the library thought that it would be more efficient to pass it by copying the bytes, it would work that way.

it’s inconsistent—&String is basically just a String but without a capacity field, and when allocation is not available, String is not usable, while &String is.

I don’t see &String and String as inconsistent. According to my proposal, &String would have just the same functionality as &String has now - you won’t be able to distinguish between the two without size_of. It’s just an optimization - if you are not allowed to mutate the object, you have no interest in the capacity, so there’s no need to copy it.

Edit: Currently Vec has a method pub fn capacity(&self) -> uint, which would of course not work with my proposal. From what I can see, it could be replaced by pub fn capacity(&mut self) -> uint without causing any trouble.