# Planned CStr changes seem inefficient

**URL:** <https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332>\
**Category:** libs\
**Created:** [February 2, 2025, 3:27pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332 "2025-02-02T15:27:31Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![zackw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/zackw/32/2071_2.png) [@zackw](https://internals.rust-lang.org/u/zackw)\
**Post date:** [February 2, 2025, 3:27pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/1 "2025-02-02T15:27:31Z")

</div>

For a long time there have been warnings in the documentation of `ffi::CStr` that say that certain methods are constant-time _now_, but in the future they will do a `strlen` operation. For example, [`CStr::to_bytes`](https://doc.rust-lang.org/std/ffi/struct.CStr.html#method.to_bytes):

> **Note** : This method is currently implemented as a constant-time cast, but it is planned to alter its definition in the future to perform the length calculation whenever this method is called.

So, right now this function is optimal (in the sense that it performs only one `strlen` operation), but in the future, it may perform two -- one in `from_bytes_until_nul`, another in `to_bytes`.

```rust
/// Extract a C string from the first portion of a byte slice.
fn cstr_from_buffer(buf: &[u8]) 
    -> Result<(&[u8], &[u8]), ffi::FromBytesUntilNulError>
{
    let cslice = ffi::CStr::from_bytes_until_nul(buf)?.to_bytes();
    Ok((cslice, &buf[cslice.len() + 1..]))
}

```

And I don't see any good alternative. With unstable library features you can use `split_once` instead and it's easier to _read_ but you don't get the hand-optimized `memchr` that backs `from_bytes_until_nul`; as far as I can tell, there's no way to access that hand-optimized `memchr` short of `feature(slice_internals)`, which the compiler yells at you for using at all, even on nightly. Without unstable library features, I think I'd have to use either the `memchr` crate, or `libc::strlen`, to get equivalent efficiency to `from_bytes_until_nul`.

It seems to me that this means the plan to make `CStr::to_bytes` not constant-time is maybe not a good plan and it should be reconsidered.

([Playground link with all three variants of the above function I've come up with so far](https://play.rust-lang.org/?version=nightly&mode=release&edition=2021&gist=47160392eed3a563c5ae15faa6fed4a3).)

---

<div class="post-metadata">

**Author:** ![pitaj](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/pitaj/32/11262_2.png) [@pitaj](https://internals.rust-lang.org/u/pitaj)\
**Post date:** [February 2, 2025, 4:01pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/2 "2025-02-02T16:01:13Z")

</div>

The idea behind this is to make `CStr` a single pointer. This makes conversion to and from a `char*` free. Then getting the length would be a lazy operation, counting the bytes.

I think this would be the way to do it with a "thin" CStr

```rust
/// Extract a C string from the first portion of a byte slice.
fn cstr_from_buffer(buf: &[u8]) 
    -> Result<(&[u8], &[u8]), ffi::FromBytesUntilNulError>
{
    let cslice = unsafe {
        ffi::CStr::from_ptr(buf.as_ptr().cast())?.to_bytes()
    };
    Ok((cslice, &buf[cslice.len() + 1..]))
}

```

If we do move this direction, I think it would make sense for us to add variants of the `from_bytes_*` methods that just return the length alongside the `CStr`.

```rust
pub const fn from_bytes_until_nul_with_length(
    bytes: &[u8],
) -> Result<(&CStr, usize), FromBytesUntilNulError>

```

---

<div class="post-metadata">

**Author:** ![zackw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/zackw/32/2071_2.png) [@zackw](https://internals.rust-lang.org/u/zackw)\
**Post date:** [February 2, 2025, 4:18pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/3 "2025-02-02T16:18:06Z")

</div>

> [@pitaj](#):
>
> ```rust
> let cslice = unsafe {
> ffi::CStr::from_ptr(buf.as_ptr().cast())?.to_bytes()
> };
> 
> ```

This has undefined behavior if there isn't a NUL within `buf`, unlike `from_bytes_until_nul`.

---

<div class="post-metadata">

**Author:** ![kpreid](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kpreid/32/8484_2.png) [@kpreid](https://internals.rust-lang.org/u/kpreid)\
**Post date:** [February 2, 2025, 4:22pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/4 "2025-02-02T16:22:50Z")

</div>

I think that broadly, all this comes down to the two use cases of “parsing data in a buffer” vs. “FFI”, and the intent has always been that `CStr` is _for_ FFI, where:

- You want to be able to declare “my function's parameter is a null-terminated string” and get a _safe_ Rust type out of that, so only the function declaration is `unsafe`. This requires that `CStr` be a thin pointer (without length).
- You have to rely on the null being present anyway; there’s no UB-free way to check.

---

<div class="post-metadata">

**Author:** ![dlight](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/dlight/32/8462_2.png) [@dlight](https://internals.rust-lang.org/u/dlight)\
**Post date:** [February 2, 2025, 4:24pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/5 "2025-02-02T16:24:26Z")

</div>

> [@zackw](#):
>
> > **Note** : This method is currently implemented as a constant-time cast, but it is planned to alter its definition in the future to perform the length calculation whenever this method is called.

It would be great that if/when those changes are actually done, the stdlib added another string type that represents a C string with length. It's often convenient to carry around the length even for nul-terminated strings, like C++ strings do; and also it may be error prone to do this manually (like is done in C by some programs); and also it would be a hassle to introduce a new dependency for it, specially because the stdlib had this behavior for at least 10 years.

---

<div class="post-metadata">

**Author:** ![zackw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/zackw/32/2071_2.png) [@zackw](https://internals.rust-lang.org/u/zackw)\
**Post date:** [February 2, 2025, 4:49pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/6 "2025-02-02T16:49:05Z")

</div>

Is it _really_ a safe-to-call function if `CStr` doesn't enforce the presence of the NUL upon construction?

---

<div class="post-metadata">

**Author:** ![Jules-Bertholet](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jules-bertholet/32/10671_2.png) [@Jules-Bertholet](https://internals.rust-lang.org/u/Jules-Bertholet)\
**Post date:** [February 2, 2025, 4:51pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/7 "2025-02-02T16:51:54Z")

</div>

I agree that it’s not a good plan, and in fact I suspect it won’t happen. This is because, in current Rust, it’s assumed that you can get the size of any value from a pointer to it, without dereferencing said pointer. This would no longer be true with a thin `CStr`, though. And that means that, if the thin `CStr` is inside a mutex of some sort, you can no longer soundly get its length without locking the mutex. Therefore, the proposed change would be breaking.

---

<div class="post-metadata">

**Author:** ![kpreid](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kpreid/32/8484_2.png) [@kpreid](https://internals.rust-lang.org/u/kpreid)\
**Post date:** [February 2, 2025, 5:07pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/8 "2025-02-02T17:07:45Z")

</div>

> [@zackw](#):
>
> Is it _really_ a safe-to-call function if `CStr` doesn't enforce the presence of the NUL upon construction?

`CStr` has an invariant, which can be unsafely asserted or checked at construction time. Either is a valid approach. When accepting a C string from foreign code, it is _necessary_ to make the unsafe assumption that a NUL is present; there is no way to check the assumption without reading out-of-bounds if it is false.

---

<div class="post-metadata">

**Author:** ![zackw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/zackw/32/2071_2.png) [@zackw](https://internals.rust-lang.org/u/zackw)\
**Post date:** [February 2, 2025, 5:09pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/9 "2025-02-02T17:09:55Z")

</div>

Oh, I see, I was imagining the FFI going the other way around (Rust calling C).

Sometimes there's a higher level structure telling you when to stop scanning for the NUL, but you're probably stuck trusting the caller to set _that_ correctly, too.

---

<div class="post-metadata">

**Author:** ![dlight](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/dlight/32/8462_2.png) [@dlight](https://internals.rust-lang.org/u/dlight)\
**Post date:** [February 2, 2025, 5:43pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/10 "2025-02-02T17:43:51Z")

</div>

Again, if it won't happen, there's another string type that could be added: a c string _without_ length, and then use _that_ for zero-overhead FFI. The stdlib already has many string types, but it's probably a mistake to not distinguish between those two strings.

---

<div class="post-metadata">

**Author:** ![comex](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/comex/32/2587_2.png) [@comex](https://internals.rust-lang.org/u/comex)\
**Post date:** [February 3, 2025, 7:16am UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/11 "2025-02-03T07:16:31Z")

</div>

> [@zackw](#):
>
> Is it _really_ a safe-to-call function if `CStr` doesn't enforce the presence of the NUL upon construction

I thought it did enforce that. Is there some way to create a non-nul-terminated `CStr` without unsafe?

---

<div class="post-metadata">

**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\
**Post date:** [February 3, 2025, 10:14am UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/12 "2025-02-03T10:14:41Z")

</div>

I agree with the opinions above that we probably should have two separate types. Wouldn't it be better to introduce a separate `CStrPtr` type and keep the current `CStr` behavior and layout to prevent potential unpleasant surprises?

---

<div class="post-metadata">

**Author:** ![jdahlstrom](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jdahlstrom/32/3351_2.png) [@jdahlstrom](https://internals.rust-lang.org/u/jdahlstrom)\
**Post date:** [February 3, 2025, 11:39am UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/13 "2025-02-03T11:39:58Z")

</div>

Would be nice if the hypothetical `CStrPtr` could be just a regular `&c_char` or `*const c_char`. I guess it's not feasible given that `c_char` is just `i8` and changing it to an opaque type would break things.

Generalizing a bit, might be useful to have something like `&thin Unsized` or `Thin<Unsized>` that you couldn't do much with except pass to FFI or unsafely construct a fat reference/pointer given the metadata part.

---

<div class="post-metadata">

**Author:** ![newpavlov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/newpavlov/32/3290_2.png) [@newpavlov](https://internals.rust-lang.org/u/newpavlov)\
**Post date:** [February 3, 2025, 11:44am UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/14 "2025-02-03T11:44:56Z")

</div>

> [@jdahlstrom](#):
>
> the hypothetical `CStrPtr` could be just a regular `&c_char` or `*const c_char`, couldn't it?

Well, a separate type is useful as a "container" for string-specific methods like `to_str`, `to_bytes_with_nul`, etc.

---

<div class="post-metadata">

**Author:** ![jdahlstrom](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jdahlstrom/32/3351_2.png) [@jdahlstrom](https://internals.rust-lang.org/u/jdahlstrom)\
**Post date:** [February 3, 2025, 11:49am UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/15 "2025-02-03T11:49:49Z")

</div>

Yes, but you could impl those for `&c_char`. That's why it would be good to have `c_char` be something else than `i8`, it would be confusing if `&i8` had a bunch of string-related methods. The API would of course be almost entirely unsafe because there's not much that you can safely do with a thin reference to a C string, besides accessing the first character.

---

<div class="post-metadata">

**Author:** ![Nemo157](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/nemo157/32/11585_2.png) [@Nemo157](https://internals.rust-lang.org/u/Nemo157)\
**Post date:** [February 3, 2025, 11:54am UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/16 "2025-02-03T11:54:59Z")

</div>

> [@jdahlstrom](#):
>
> Yes, but you could impl those for `&c_char`

A `&c_char` doesn’t have the invariant that it points to a zero-terminated slice (and might have memory model implications on whether it can read more than one byte).

---

<div class="post-metadata">

**Author:** ![jdahlstrom](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jdahlstrom/32/3351_2.png) [@jdahlstrom](https://internals.rust-lang.org/u/jdahlstrom)\
**Post date:** [February 3, 2025, 11:56am UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/17 "2025-02-03T11:56:24Z")

</div>

Ah, true. I overlooked the fact that it's actually possible to uphold that invariant if you can only ever safely create a thin pointer from an existing `CStr` or something else that guarantees the nul terminator. I tacitly assumed that such a pointer would only be used/useful in unsafe FFI contexts where the thinness matters, and safe APIs on top would give and take normal fat `&CStr`s.

---

<div class="post-metadata">

**Author:** ![zackw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/zackw/32/2071_2.png) [@zackw](https://internals.rust-lang.org/u/zackw)\
**Post date:** [February 3, 2025, 2:16pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/18 "2025-02-03T14:16:26Z")

</div>

> [@comex](#):
>
> > [@zackw](#):
> >
> > if `CStr` doesn't enforce the presence of the NUL upon construction
> 
> I thought it did enforce that.

It does _now_, but if we made it a thin pointer we'd either have to give that up at least some of the time, or eat the cost of even more `strlen` scans.

---

<div class="post-metadata">

**Author:** ![scottmcm](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/scottmcm/32/2355_2.png) [@scottmcm](https://internals.rust-lang.org/u/scottmcm)\
**Post date:** [February 3, 2025, 6:00pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/19 "2025-02-03T18:00:32Z")

</div>

> [@Jules-Bertholet](#):
>
> This is because, in current Rust, it’s assumed that you can get the size of any value from a pointer to it, without dereferencing said pointer. This would no longer be true with a thin `CStr`, though.

Work's under way to address that: [Hierarchy of Sized traits by davidtwco · Pull Request #3729 · rust-lang/rfcs · GitHub](https://github.com/rust-lang/rfcs/pull/3729)

---

<div class="post-metadata">

**Author:** ![Jules-Bertholet](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jules-bertholet/32/10671_2.png) [@Jules-Bertholet](https://internals.rust-lang.org/u/Jules-Bertholet)\
**Post date:** [February 3, 2025, 6:02pm UTC](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332/20 "2025-02-03T18:02:43Z")

</div>

That would address the issue for a _new_ `ThinCStr` type, but wouldn’t make it any less breaking to change the extant `CStr`.

[Next page](https://internals.rust-lang.org/t/planned-cstr-changes-seem-inefficient/22332.md?page=2)