Missing String->CString without copy

libs
1

We have a impl TryFrom<CString> for String, but there is no impl TryFrom<String> for CString. The later is badly needed because it would allow converting Strings into CStrings almost always without an extra re-allocation.

Proposed API

impl TryFrom<CString> for String {
    type Error = NulError;
    /// Converts a [`String`] into a [`CString`] by trying
    /// to append `\0` if it is not there already.
    #[inline]
    fn try_from(s: String) -> Result<CString, Self::Err> {
        // check if s ends with `\0` - if it does, this is allowed
        // if it does not, append `\0`
        // cast the result to CString and return
    }
}
2

CString does not support extra unused capacity, so I'm not so sure about the "almost always without extra re-allocation" claim.

That being said, it looks like the functionality you describe is (almost) already present in the form of CString::new, right? Including the lack of re-allocation if the capacity is right - just not following the locig of optionally accepting a trailing zero to already be in-place (instead it always wants to add it, and expects a zero-free input).

3 Likes
3

@steffahn thanks! I don't know how I could have overlooked the (essentially) new(Vec<u8>) that tries to prevent re-alloc.

What happens if the capacity is too high? Would it simply keep extra capacity unchanged, or would it first try to shrink it?

4

It needs to shrink in this case. As mentioned, CString doesn’t support unused capacity. This is in line with supporting a raw representation (here's the inverse conversion) without any separate capacity information.

The current implementation seems to use Vec<u8> to Box<[u8]> conversion for this (on the Vec after the terminating 0 is added). This avoids re-allocation if the Vec (or String) you started with had exactly 1 byte of spare capacity. It looks like the implementation also makes sure to avoid double re-allocation when no capacity was present (using reserve_exact(1) to avoid too much new capacity from the push(0)).

2 Likes
5

Is allocated capacity actually used by the de-allocator? If not, why would it matter that some extra capacity has been allocated by the Vec, even though it is not being used? Do note that shrink_to_fit may still keep some extra allocated space, which seems to suggest that shrink_to_fit call is a memory optimization rather than a requirement for proper operations.

The resulting vector might still have some excess capacity, just as is the case for with_capacity

6

Doesn't this make the API somewhat redundant with Box<CStr> and Box::into_raw? I am probably out of the loop here but I am noticing CString::into_raw was stabilized in 1.4.0 so perhaps this is just one of those awkward historical choices.

7

We can't answer that in general, since the allocator can be provided externally. It's in the API for the allocator, so they are allowed to use and depend on that.

3 Likes
8

As far as I understand, the intention expressed in the API is that CStr is supposed to become a thin pointer eventually, and then Box<Cstr> would be thin, vs CString being a fat pointer. Correspondingly, there’s e.g. method names as_bytes (as_… = cheap conversion) for CString but only to_bytes (not to_ = potentially expensive, needs to scan for the 0) on CStr and thus Box<CStr>.

9

Does this mean there is no intention to add appending methods to CString, like CString::push_cstr? [1] I had assumed it was just waiting for a motivated person to write an ACP, but your comment makes me think otherwise.

That would be unfortunate, as it relegates the job of building C strings to a hand rolled wrapper around Vec<u8> to enforce the same invariants.

  1. Well, I know anything is possible but we tend to not add methods with surprising runtime costs. If we can't make the same amortized guarantees as String::push_str I don't see a CString::push_cstr happening. ↩︎

1 Like
10

There is a bstr crate for better byte strings (arguably, the byte strings that should be in the standard library). Perhaps an equivalent crate for C strings would make sense.

Of course, bstr is primarily extension traits (plus optional newtypes). It would not be quite as straightforward here. You would need a new separate "CStringBuilder" or similar, which lowers Interoperability.

1 Like
11
3 Likes
12

Nice that seems to be relatively new. And I don't know how comparable with the bstr crate it is when it comes to features, hopefully at least as good.

13

Is there a specific reason that CString::try_from(some_string) does not work and behave equivalently to CString::new? We already have plenty of other Into/From conversions that allocate so I don't think that can be the blocker.

3 Likes