[pre-RFC] Allowing string literals to be either &'static str or String, similar to numeric literals

[Ixrec]

@phaux It’s not clear to me why “interpolated string literals” should only produce String and not &'static str (when given const/static arguments of course). I think features like interpolation probably ought to remain orthogonal to how literal typing works.

@arthurprs I agree that “special-casing String to allow static backing data” is way too magical, not to mention totally unnecessary for solving the original problem. But in the interests of moving the conversation forward, which of the alternative solutions do you prefer? New syntax for “owned string literals”? A macro with a very short name? Doing nothing at all?

@elszben The immediate problem is that with a proposal that broad, you need far stronger motivation than what we have in this thread. I want the vast majority of potentially-expensive methods like .to_string() and .into() to be explicit most of the time (especially since .into() could mean practically anything), and don’t know of any compelling reason why that principle should be changed. And that’s before we get to any of the tricky questions about how this would actually work with generics and inference and the other implicit conversions that Rust does do today.

@Nemo157 “User-defined literals” (as C++ calls this feature) are something I’d probably support if designed well and given enough compelling examples, but I see it as solving a different set of problems. What makes to_string() on literals annoying to me isn’t the number of characters to type, it’s that we have to provide any kind of explicit annotation at all when it’s perfectly obvious to the type system that this string literal has to be a String instead of a &'static str, but it still makes me do one extra round of see error message -> edit code -> recompile before letting me see useful output again. To me user-defined literals are most compelling for libraries dealing with types like dates, times, SQL queries, etc. which won’t get dedicated literal syntax in the core language, but can reasonably be represented by a single string or number.

(I haven’t read gankro’s new post yet, but since I already typed up all this and that’s gonna take a while to digest I’ll just post this now)

[ExpHP]

I don’t imagine String is currently a lang item? (and this proposal seems to me like the sort of thing that would require it to become one)

(note: I am referring to the OP’s proposal, not the static string optimization)

[comex]

@Gankro Good post.

Regarding capacity: Searching the standard library, I find:

• foo.capacity() - foo.len() on Vec<u8> in two places (libstd/sys/windows/pipe.rs, libstd/sys_common/io.rs), both as part of unsafe code that passes the uninitialized portion of the Vec to OS read functions;
• foo.capacity() == foo.len() on Vec<u8> in the same libstd/sys/windows/pipe.rs, and on Vec<T> in libcollections/vec.rs. (In the latter case, capacity ≠ len is taken to mean that an element can be appended without reallocating.)

If capacity() returns 0, the first will overflow, and the second will allow out-of-bounds writes – both really bad. Of course, the stdlib itself can be changed, but external code may include similar logic. So I think capacity() returning 0 is unviable at least for Vec, and probably for String as well.

(But then, I agree that supporting statics in Vec is a bad idea anyway because of the cost on deref_mut; I didn’t think of doing it only for String.)

Also, nit: I don’t think any of pop, truncate, or clear would need to special-case static strings, unless you’re really concerned about capacity() staying the same; the static pointer is still valid with a reduced length.

[Gankro]

Thanks for the sleuthing! I hadn’t thought about those particular cases; definitely a good warning against making this change.

Also great point on the truncation family of methods! Talking about COW got me into a mental model where mutation requires reallocation.

[tmccombs]

I started a topic on something like this at [pre-RFC] custom string literals. The rfc I posted was very rough, but if you’re interested in custom string literals I’d like you’re feedback.

[Storyyeller]

StableAddress trait currently assumes all bets are off if you take a mutable reference, so that’s encouraging.

Actually, that’s not true. owning_ref was updated to support mutable references, which implicitly added the requirement that implementers of StableAddress + DerefMut are stable under deref_mut().

I tried to document the requirements here as best I could.

Anyway, I am extremely opposed to changing the implementations of core types like String and Vec. Tons of code has been written under the assumption that they work the way they do. Not only will changing things break code and make the language more confusing, it will also (rightfully) promote the impression that Rust is unstable and immature.

As for the original suggestion, I think special casing String is the wrong way to go about it. In cases where performance isn’t an issue, I want everything to default to easy_strings::EZString, not String. I think user defined string literals are the best solution.

P.S: IMO, defaulting ints to i32 was a bad idea - I’ve already encountered bugs due to values unexpectedly becoming i32s.

[Kimundi]

@Storyyeller, @Gankro: Ah right, I forgot about the recent-ish DerefMut patchs to StableAddress, my bad.

@Gankro: Regardless of anything else - why would pop() and truncate() need to allocate? Couldn’t they just slice the static string?

[withoutboats]

Actually, that’s not true. owning_ref was updated to support mutable references, which implicitly added the requirement that implementers of StableAddress + DerefMut are stable under deref_mut().

What does “stable under deref_mut” mean? The docs say:

More specifically, implementors must ensure that the result of calling deref() is valid for the lifetime of the object, not just the lifetime of the borrow, and that the deref is valid even if the object is moved. Also, it must be valid even after invoking arbitrary &self methods or doing anything transitively accessible from &Self. If Self also implements DerefMut, the same restrictions apply to deref_mut() and it must remain valid if anything transitively accessible from the result of deref_mut() is mutated/called. Additionally, multiple calls to deref, (and deref_mut if implemented) must return the same address.

I feel like I must be totally misunderstanding this paragraph, because I don’t see how Vec<T> could meet them given that pushing to it can reallocate. What am I missing?

EDIT: It seems that specifically you can’t have mutated the object, but you can deref it to a mutable reference? I guess that’s what this means:

Basically, it must be valid to convert the result of deref() to a pointer, and later dereference that pointer, as long as the original object is still live, even if it has been moved or &self methods have been called on it.

(Not to be pedantic but you can mutate objects without calling &mut self methods through reassigning visible fields, but I think I understand what you mean.)

[Storyyeller]

(Not to be pedantic but you can mutate objects without calling &mut self methods through reassigning visible fields, but I think I understand what you mean.)

Yes, but OwningRef/Value take ownership of the container and never expose a mutable reference to it. The only mutable access on it is deref_mut() (and technically, drop)

Anyway, if you have ideas for how to clarify the documentation, feel free to suggest them.

[matthieum]

Was any thought given to simply switch the type of string literals from &'static str (resp. &'static [u8]) to &'static String (resp. &'static Vec<u8>)?

This doesn’t allow let v: Vec<String> = vec!["a", "b"];, as it’s instead let v: Vec<&String> = vec!["a", "b"]; however:

• it allows punting on introducing the distinction between str and String,
• it does not require any magic in String or Vec: &String and &Vec do not allow mutation, so the pointed to memory can be stored in ROM as usual (one extra word),
• it does not require inference specialization, &'static String can naturally Deref into &'static str.

Did I miss any discussion? Would it not be good enough?

[Ixrec]

That’s obviously not backwards compatible, so it’s not really an option.

[yazaddaruvala]

I am not entirely sure how to formulate this or if it is at all feasible, but I want to put out an idea.

Currently, allocators return mutable references, therefore they need to point to the heap. However, could we design custom allocators to potentially return immutable references? Then we could have an “allocator” into program memory, an “allocator” into the stack.

Could we solve this &str vs &String discrepancy with custom allocators?

let s = "Hello World".as_string();
// s: &'static String<immutable_static_program_memory_allocator<'static>>
// s.vec: &'static Vec<u8, immutable_static_program_memory_allocator<'static>>

[withoutboats]

I think we should revisit this next year. Currently I’m in favor of performing this optimization/coercion with these modifications:

• Only apply it to &'static str & String, not &'static [T] and Vec<T>.
• String::capacity returns max(len, capacity), so string.capacity() - string.len() will never underflow.

[atagunov]

Suppose "x" produced an immutable String when needed

Would that avoid any change to String/Vec

except a trick to avoid free()-ing static mem?

[withoutboats]

There is no such thing as an immutable String. If users have ownership they can mutate it.

[ExpHP]

Even with that kind of desugaring, immutability still is not an intrinsic property of values in rust, only of name bindings… What would prevent the user from doing this?

fn foo(mut s: String) {
   // Use s mutably
}

fn main() {
    foo("hello");
}

[newpavlov]

I don’t like implicit coercion proposals in this thread. If the main problem is annoyance of .to_string(), then in my opinion using explicit prefixes like s"foo" is a much better solution and it’s in line with already existing syntax b"foo" and r"foo".

On a slightly different note: I would also like very much to have something like h"00 01 0a ff" to define binary arrays using hex notation. (spaces are optional and just for readability)

[atagunov]

Could "x" then possibly desugar into & 'static String?

Where the actual String would be in static data?

Or maybe even into *static_var where static_var has got type & 'static String?

[withoutboats]

That would make it doubly indirect and not solve the problem of people trying to pass string literals to APIs that take String. Moreover it would not be beneficial: the branch on null that we have to do is almost certainly not a performance problem, and if it is we just won’t do this.

[ExpHP]

Or maybe even into *static_var where static_var has got type & 'static String?

I suspect that would error due to trying to move out of the static.

Of course, we could try .clone() instead of * but then that clone would either need to (a) point into the static memory (generating the same issue as before) or (b) allocate on the heap (entirely defeating the optimization).