# Byte-aligned Vec

**URL:** <https://internals.rust-lang.org/t/byte-aligned-vec/7200>\
**Category:** language design\
**Created:** [March 31, 2018, 5:33pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200 "2018-03-31T17:33:07Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![andygrove](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/andygrove/32/3891_2.png) [@andygrove](https://internals.rust-lang.org/u/andygrove)\
**Post date:** [March 31, 2018, 5:33pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/1 "2018-03-31T17:33:07Z")

</div>

Hi,

I’m working on a rust implementation of Apache Arrow and one of the requirements is to have 64-byte aligned arrays of primitive types so I cannot simply use `Vec<T>` but have had to implement my own type that uses libc and lots of unsafe code.

I saw that Rust 1.25 introduced byte-aligned structs with `#[repr(align(16))]` and if I could somehow do the same for Vec I would be very happy. I’m wondering if any language features are in the works for this or whether there are other strategies for this?

Thanks,

Andy.

---

<div class="post-metadata">

**Author:** ![jethrogb](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jethrogb](https://internals.rust-lang.org/u/jethrogb)\
**Post date:** [March 31, 2018, 5:35pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/2 "2018-03-31T17:35:10Z")

</div>

I’m assuming you don’t need to align the `Vec` itself but need to have a `Vec` where the elements are aligned? You can just define a type with the correct alignment and create a `Vec` of that:

```rust
#[repr(align(64))]
struct MyType {
    /* ... */
}
type Aligned = Vec<MyType>;

```

---

<div class="post-metadata">

**Author:** ![andygrove](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/andygrove/32/3891_2.png) [@andygrove](https://internals.rust-lang.org/u/andygrove)\
**Post date:** [March 31, 2018, 5:37pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/3 "2018-03-31T17:37:23Z")

</div>

No, I actually need the start of the data aligned. So if I have a Vec I need the data to start on a 64-byte aligned address but then have the items 4-byte aligned in the i32 example.

---

<div class="post-metadata">

**Author:** ![jethrogb](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@jethrogb](https://internals.rust-lang.org/u/jethrogb)\
**Post date:** [March 31, 2018, 5:40pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/4 "2018-03-31T17:40:49Z")

</div>

Oh, that’s an interesting requirement. I don’t think that’s supported in any standard/safe way. Can you allocate a `Vec` that’s too big and start indexing at a particular element offset to get your required alignment?

---

<div class="post-metadata">

**Author:** ![andygrove](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/andygrove/32/3891_2.png) [@andygrove](https://internals.rust-lang.org/u/andygrove)\
**Post date:** [March 31, 2018, 5:48pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/5 "2018-03-31T17:48:07Z")

</div>

That could work I guess. Seems kinda hacky and wasteful, but definitely safer than what I’m doing now.

---

<div class="post-metadata">

**Author:** ![andygrove](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/andygrove/32/3891_2.png) [@andygrove](https://internals.rust-lang.org/u/andygrove)\
**Post date:** [March 31, 2018, 5:53pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/6 "2018-03-31T17:53:58Z")

</div>

I guess what I asking for is `Vec::with_capacity(n: usize, align: usize)`. I wonder how much work it would be to add something like that to Rust? It is probably not something I can tackle since I’m not a C expert.

---

<div class="post-metadata">

**Author:** ![sfackler](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/sfackler/32/9_2.png) [@sfackler](https://internals.rust-lang.org/u/sfackler)\
**Post date:** [March 31, 2018, 5:57pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/7 "2018-03-31T17:57:26Z")

</div>

We wouldn’t want to add that API to `Vec` itself because it’d need to track the requested alignment as a new field. We’re getting close to stabilizing some of the allocator API which will let you make a custom type that does this though.

---

<div class="post-metadata">

**Author:** ![andygrove](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/andygrove/32/3891_2.png) [@andygrove](https://internals.rust-lang.org/u/andygrove)\
**Post date:** [March 31, 2018, 6:01pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/8 "2018-03-31T18:01:22Z")

</div>

allocator API does sound like the way to go for this. Are these stabilizations being targeted to a particular Rust release yet?

---

<div class="post-metadata">

**Author:** ![sfackler](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/sfackler/32/9_2.png) [@sfackler](https://internals.rust-lang.org/u/sfackler)\
**Post date:** [March 31, 2018, 6:32pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/9 "2018-03-31T18:32:51Z")

</div>

It’s not totally pinned down, but hopefully sometime near 1.28 or so?

How does the Java and C++ implementations deal with the alignment requirement? You can always directly talk to the C allocator through `posix_memalign`/`free` to make these allocations now.

---

<div class="post-metadata">

**Author:** ![andygrove](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/andygrove/32/3891_2.png) [@andygrove](https://internals.rust-lang.org/u/andygrove)\
**Post date:** [March 31, 2018, 6:47pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/10 "2018-03-31T18:47:56Z")

</div>

OK, thanks. Yes, I’m using `posix_memalign` for now.

---

<div class="post-metadata">

**Author:** ![jmst](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jmst](https://internals.rust-lang.org/u/jmst)\
**Post date:** [March 31, 2018, 9:14pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/11 "2018-03-31T21:14:40Z")

</div>

Just make a type that is a 64-byte aligned array of 16 i32s (or just 64 u8s, although if you are using it as i32s it’s more elegant to use i32s), make a Vec of that, and then a newtype over that Vec that implements Deref to &i32 by transmuting.

Add an extra size field to the newtype if you need to keep track of size at primitive granularity.

---

<div class="post-metadata">

**Author:** ![andygrove](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/andygrove/32/3891_2.png) [@andygrove](https://internals.rust-lang.org/u/andygrove)\
**Post date:** [March 31, 2018, 9:43pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/12 "2018-03-31T21:43:11Z")

</div>

That is pure genius. Simple and elegant. Thank you!

---

<div class="post-metadata">

**Author:** ![kornel](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kornel/32/2711_2.png) [@kornel](https://internals.rust-lang.org/u/kornel)\
**Post date:** [April 2, 2018, 3:34pm UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/13 "2018-04-02T15:34:41Z")

</div>

Just a safety note: that can be done only with read-only `Deref` for `Vec`. Any push to transmuted vec could re-allocate storage as `i32`s.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\
**Post date:** [March 25, 2019, 8:30am UTC](https://internals.rust-lang.org/t/byte-aligned-vec/7200/14 "2019-03-25T08:30:00Z")

</div>

Closed on March 25, 2019, 8:30am UTC

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.