# Storing A Smaller Length Of A Slice And Accessing Beyond The Stored Length (But Inside The Actual Length)
**URL:** https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234
**Category:** Unsafe Code Guidelines
**Created:** [April 27, 2020, 3:32am UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234 "2020-04-27T03:32:42Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@brunoczim](https://internals.rust-lang.org/u/brunoczim)
#### Post date: [April 27, 2020, 3:32am UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/1 "2020-04-27T03:32:43Z")
Is this code UB?
```rust
let x = [0,1,2,3];
let y = &x[0..2];
unsafe {
let z = *y.as_ptr().add(3);
}
```
---
### Author:  [@RustyYato](https://internals.rust-lang.org/u/RustyYato)
#### Post date: [April 27, 2020, 3:48am UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/2 "2020-04-27T03:48:45Z")
Yes, it is UB. This is better posted on [users.rust-lang.org](http://users.rust-lang.org), internals is meant for the development of Rust, not general questions about Rust.
---
### Author:  [@brunoczim](https://internals.rust-lang.org/u/brunoczim)
#### Post date: [April 27, 2020, 4:08am UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/3 "2020-04-27T04:08:30Z")
I thought unsafe coda guidelines was supposed to discuss UB... I mean, why is this UB?
Anyway: [https://users.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/41630](https://users.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/41630)
---
### Author:  [@daboross](https://internals.rust-lang.org/u/daboross)
#### Post date: [April 27, 2020, 5:40am UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/4 "2020-04-27T05:40:08Z")
> [@brunoczim](#):
>
> I thought unsafe coda guidelines was supposed to discuss UB... I mean, why is this UB?
I think it would be on-topic to discuss whether or not this should be UB, as it would be to have a discussion on writing documentation on what is and isn't UB.
However, this question comes across much more as "can I write this code?" than "should rustc count this code as valid, theoretically?", and as such, it seems much, much more suited for [users.rust-lang.org](http://users.rust-lang.org). Sure, it's about unsafe code, but that doesn't change the fact that this question is about using rust, and how rust behaves. You might be asking about the internal behavior of rust, but you're still asking as someone writing rust code, not as someone developing it, so the question belongs in the users forum.
This doesn't mean the question isn't important. It's just that this forum is most frequented by people wanting to discuss the internals of rust and plans for changing them, and questions like this are simply off-topic. This isn't a matter of priority, either - you'll get much better answers somewhere where the question is actually on-topic, and [users.rust-lang.org](http://users.rust-lang.org) is in general much more active than internals.
---
### Author:  [@kornel](https://internals.rust-lang.org/u/kornel)
#### Post date: [April 27, 2020, 10:24am UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/5 "2020-04-27T10:24:53Z")
> [@brunoczim](#):
>
> why is this UB?
I presume it's because it makes reasoning about slices simple. It makes slices only what they are on the surface: `(data, len)`, and there's no need to track where they originated from, and what length they _really_ have.
If access outside of a slice was allowed, it'd make `slice.split_at_mut()` unsafe. One half could peek into the other, causing mutable aliasing. It might be possible to define rules around that, but that would be creating new type of shadow-slice with invisible extra lifetimes to track how it was created.
---
### Author:  [@ckaran](https://internals.rust-lang.org/u/ckaran)
#### Post date: [April 27, 2020, 12:53pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/6 "2020-04-27T12:53:56Z")
Beyond what @kornel said, in your particular case both arrays are constants, and the compiler is aware of this. That means that under the right circumstances (all uses of `x` are known at compile time, and `x` is never used except through `y`), the compiler could decide to reuse the trailing portion of the slice with the knowledge that attempts to access beyond the slice are actually illegal.
Now, before anyone gets the wrong idea, as far as I know the compiler does not do this at the current time. But it could be a useful optimization in the future for small embedded systems.
---
### Author:  [@steffahn](https://internals.rust-lang.org/u/steffahn)
#### Post date: [April 27, 2020, 12:55pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/7 "2020-04-27T12:55:50Z")
I don’t think these comments about “what if the slice came from `split_at_mut`” are helping the discussion at all. On the [user.rust-lang.org](http://user.rust-lang.org) thread there’s a similar answer. The question is not about what-ifs regarding any different scenarios, it is also not about soundness if this kind-of code was provided in some API. The question just is: is this concrete piece of code UB or not.
---
### Author:  [@kornel](https://internals.rust-lang.org/u/kornel)
#### Post date: [April 27, 2020, 12:59pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/8 "2020-04-27T12:59:25Z")
I think it's relevant, because roughly speaking this code is UB, _because_ of `split_at_mut` existing. Rust has one set of rules for all code. Rules that require making things like `split_at_mut` safe also declare OP's benign code as UB, just because what is UB is a general definition without exceptions for innocent cases.
---
### Author:  [@steffahn](https://internals.rust-lang.org/u/steffahn)
#### Post date: [April 27, 2020, 12:59pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/9 "2020-04-27T12:59:55Z")
> [@kornel](#):
>
> roughly speaking this code is UB, _because_ of `split_at_mut` existing
I disagree.
---
### Author:  [@RalfJung](https://internals.rust-lang.org/u/RalfJung)
#### Post date: [April 27, 2020, 1:01pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/10 "2020-04-27T13:01:18Z")
> [@steffahn](#):
>
> The question just is: is this concrete piece of code UB or not.
Yes it is, because `y` is a reference to just the first 2 elements of the array. When a reference is cast to a raw pointer (such as with `as_ptr` here), that raw pointer may only be used for the memory the reference "points to", as determined by `size_of_val(y)`. The code violates this by using the raw pointer outside of the memory the reference is valid for.
Whether or not this rule should be weakened, and whether it even _can_ be weakened without sacrificing many optimizations, is being discussed at
>
>
> Currently, the following is illegal according to Stacked Borrows:
> \`\`\`rust
> let …val = \[1u8, 2\];
> let ptr = &val\[0\] as \*const u8;
> let \_val = unsafe { \*ptr.add(1) };
> \`\`\`
> The problem is that the cast to \`\*const u8\` creates a raw pointer that may only be used for the \`u8\` it points to, not anything else. The most common case is to do \`&slice\[0\] as \*const \_\` instead of \`slice.as\_ptr()\`.
>
> This has lead to problems:
>
> \* \[rand did the \`&slice\[0\]\` thing\](https://github.com/rust-random/rand/issues/779).
> \* \[Same for hashbrown\](https://github.com/rust-lang/hashbrown/pull/80).
> \* \[\`Rc::into\_raw\`+\`Rc::from\_raw\` don't work well together because of this\](https://github.com/rust-lang/unsafe-code-guidelines/issues/134#issuecomment-496469397).
> \* \[capnproto also used the \`&slice\[0\]\` pattern\](https://github.com/capnproto/capnproto-rust/commit/72480efb3514d32278bd2502a7b90b22a34d12b8)
>
> Maybe this is too restrictive and raw pointers should be allowed to access their "surroundings"? I am not sure what exactly that would look like though. It would probably require having the raw pointer fully inherit all permissions from the reference it is created from.
>
> I'll use this issue to collect such cases.
Miri currently fails to detect this particular code as UB because Miri does not fully precisely track raw pointers. Doing so requires figuring out better what we want to do with integer-pointer casts, and also getting `&raw` used more throughout the ecosystem.
---
### Author:  [@Ixrec](https://internals.rust-lang.org/u/Ixrec)
#### Post date: [April 27, 2020, 1:11pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/11 "2020-04-27T13:11:29Z")
While `split_at_mut` is not directly relevant to the "_is_ it UB?" question, it is extremely relevant for answering the "_should_ it be UB?" or "could it ever not be UB?" follow-up questions that people almost always end up asking (and often kinda already intended in their original question) on "is it UB?" threads.
I can't really imagine us defining this behavior only for const raw pointers but leaving it UB for mut raw pointers, and IIUC aliasing mut raws is (in Stacked Borrows) already categorically UB unless there's an `UnsafeCell` involved. So "things like `split_at_mut` need to be implementable" _seems_ like a knockdown argument here, unless I'm missing something.
---
### Author:  [@RalfJung](https://internals.rust-lang.org/u/RalfJung)
#### Post date: [April 27, 2020, 1:14pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/12 "2020-04-27T13:14:57Z")
FWIW I honestly do not see the connection to `split_at_mut` here. `split_at_mut` could be sound even if this code was allowed, as the code does not even call `split_at_mut`. Sure, doing _both_ this _and_ `split_at_mut` would be UB, but that in no way implies that doing _just_ this has to be UB.
---
### Author:  [@steffahn](https://internals.rust-lang.org/u/steffahn)
#### Post date: [April 27, 2020, 1:15pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/13 "2020-04-27T13:15:24Z")
Another question: Aren’t pointers to slice entries allowed to point one past the end so that you can have a pointer to compare to if you finished iteration? ~~Then, this whole code would possibly _not_ be UB...~~
Edit: To elaborate, the documentation for pointer `add` says
> Both the starting and resulting pointer must be either in bounds or one byte past the end of the same allocated object.
where the explicit mention of “byte” confuses me a bit, but in essence the reason for this specification is stuff like iterating over a slice, isn’t it?
Edit2: I miscalculated.. the smaller slice only goes to `2` _not_ inclusive 😃
Edit3: This however proves my previous point (of this having nothing to do with `split_at_mut`) if this means that `.add(2)` would _not_ have been UB.
Edit4: Damn I’m blind, I didn’t spot the `*` dereferencing the thing all the time. I thought this was about if the pointer `add` is legal.
@RalfJung Would you say that the pointer addition `y.as_ptr().add(3)` itself is already UB in the code?
---
### Author:  [@Ixrec](https://internals.rust-lang.org/u/Ixrec)
#### Post date: [April 27, 2020, 1:20pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/14 "2020-04-27T13:20:16Z")
> [@RalfJung](#):
>
> FWIW I honestly do not see the connection to `split_at_mut` here. `split_at_mut` could be sound even if this code was allowed, as the code does not even call `split_at_mut` . Sure, doing _both_ this _and_ `split_at_mut` would be UB, but that in no way implies that doing _just_ this has to be UB.
Ah, I think I see the confusion. I was imagining this code snippet being exposed directly to safe code as a safe API which then could be combined with `split_at_mut` by additional safe client code to cause problems. But if this pointer and anything derived from it stay within the module, then you're right that there's tons of wiggle room here.
---
### Author:  [@RalfJung](https://internals.rust-lang.org/u/RalfJung)
#### Post date: [April 27, 2020, 3:12pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/15 "2020-04-27T15:12:52Z")
> [@steffahn](#):
>
> Would you say that the pointer addition `y.as_ptr().add(3)` itself is already UB in the code?
No it is not UB. The documentation for `add` says that only the limits of the allocation itself are relevant for pointer addition.
The concerns I raised (about where the pointer "comes from") only enter the picture once you actually _use_ the pointer.
> [@Ixrec](#):
>
> Ah, I think I see the confusion. I was imagining this code snippet being exposed directly to safe code as a safe API which then could be combined with `split_at_mut` by additional safe client code to cause problems. But if this pointer and anything derived from it stay within the module, then you're right that there's tons of wiggle room here.
Ah I see. That would be the question of whether this code is _sound_, and to ask that question we'd have to see at which types the code is exposed to safe code.
But the OP was (I think) asking about whether the code _has UB_, not whether it can be soundly exposed to arbitrary safe code.
---
### Author:  [@steffahn](https://internals.rust-lang.org/u/steffahn)
#### Post date: [April 27, 2020, 3:27pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/16 "2020-04-27T15:27:29Z")
> [@RalfJung](#):
>
> No it is not UB. The documentation for `add` says that only the limits of the allocation itself are relevant for pointer addition.
Interisting analysis. My next question: Would that mean it’s _not a valid optimization_ for a compiler when given code like
```rust
let x = [0; 10000];
let y = &x[0..10]
// ... rest of code only uses y, not x
```
to argue, “only the first 10 entries of `x` are ever used, let’s save **loads of** stack space and turn this into
```rust
let x = [0; 10];
let y = &x[0..10]
// ... rest of code only uses y, not x
```
”?
---
### Author:  [@RalfJung](https://internals.rust-lang.org/u/RalfJung)
#### Post date: [April 27, 2020, 3:40pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/17 "2020-04-27T15:40:17Z")
As a source-to-source transformation, that would indeed be incorrect. In a lower-level IR, after removing some requirements about inbounds pointer arithmetic, such transformations could still be possible.
---
### Author:  [@kornel](https://internals.rust-lang.org/u/kornel)
#### Post date: [April 27, 2020, 5:15pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/18 "2020-04-27T17:15:01Z")
I'm surprised you think it's not relevant. So does this mean that in this example `reach_beyond` may or may not be UB depending on which codepath is taken?
```rust
fn split_shared(x: &mut [0; 1000])
reach_beyond(x.split_at(500).0)
}
fn split_mut(x: &mut [0; 1000])
reach_beyond(x.split_at_mut(500).0)
}
fn reach_beyond(x: &[i32]) {
unsafe {
*x.as_ptr().add(x.len()+1);
}
}
```
---
### Author:  [@RalfJung](https://internals.rust-lang.org/u/RalfJung)
#### Post date: [April 27, 2020, 10:55pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/19 "2020-04-27T22:55:38Z")
> [@kornel](#):
>
> `reach_beyond` may or may not be UB
_Functions_ aren't UB, _programs_ are. It makes no sense to ask if a function is UB, just like it makes no sense to ask of a function terminates. You have to say for which inputs.
For functions, we can just ask if they are _sound_, which means "cannot cause UB when invoked from safe code" (or, equivalently, "cannot cause UB when invoked with inputs that satisfy the [safety invariant](https://www.ralfj.de/blog/2018/08/22/two-kinds-of-invariants.html) of the respective types").
For your example, both `split_mut` and `split_shared` are unsound for the reasons I explained above (using a raw pointer outside its valid range). I don't see how `split_at_mut` would make a difference here.
---
### Author:  [@steffahn](https://internals.rust-lang.org/u/steffahn)
#### Post date: [December 22, 2024, 5:04pm UTC](https://internals.rust-lang.org/t/storing-a-smaller-length-of-a-slice-and-accessing-beyond-the-stored-length-but-inside-the-actual-length/12234/20 "2024-12-22T17:04:10Z")
This topic was automatically closed 540 days after the last reply. New replies are no longer allowed.