Arrays as const generics

ive been hitting my head around this problem for a while: there is absolutely no way of statically encoding the shape of an n-dimensional array in the type system. with const generics you can encode how many axes your array can have, for example:

struct Array<T, const N: usize> {
    shape: [usize; N],
    data: Box<[T]>,
}

but you will have to check for compatible shapes whenever you operate on two arrays.

what im wondering, is whether something like

struct Array<T, const N: usize, const S: [usize; N]> {
    data: Box<[T]>,
}

with all the necessary restrictions to make extremely sure you do not go out of bounds in the S parameter, would be feasible in any way

2 Likes

saying it as a reply, this is my first time posting here and i think my spotty internet connection might have made me accidentally post something with the same title, but it was an incomplete post and i deleted it. sorry if this is not how this sort of thing is usually handled here

This is confusing. In your first example, N is the length of the array, not the number of axes. Typically "shape" is used when referring to multi-dimensional arrays. That is, [[usize; 2; 7] would have one shape but [[usize; 9]; 3] would have a different shape.

The presence of a boxed slice in each of your examples is also confusing. Did you mean to use the S shape parameter with the Box?

in both examples, N is referring to the number of axes. the difference is that in the first example i am forced to store the shape of the array as a field of the struct, which makes the array not type safe in its shape

the boxed slice contains the raw data of the array, to be represented with the given shape

Why not just represent the whole thing as a normal type parameter and feed that into the box?

struct Array<T> {
    data: Box<T>,
}

// Usage
let x: Array<[[[i32; 5]; 7]; 22]>

because this isnt a good way to implement a generic n dimensional array. this makes is impossible to define shared behaviour for such objects (tensor calculus, linear algebra, etc).

what im wondering is: is there a practical reason to not have the power to use an array as a const generic, in the way i showed in my second example?

1 Like

also, this syntax is 1. ugly, 2. not ergonomic, 3. error prone and 4. doesnt allow any abstraction over it

To answer your actual question: there is effort to enable exactly this. Not just for arrays, but any arbitrary datatype. The tracking issue for this is #95174

2 Likes

thank you so much for linking to that, i could not find it earlier

do you think having one of the const generics referring to the other in its declaration would lead to added complexity, implementation-wise? still referring to my second example, where S uses N, another const generic parameter, as part of its definition

While Iā€™m looking forward to a language solution, here is one neat multi-dimensional array library which supports shapes with constant entries and which works right now:

The mdarray library even allows to mix constant and dynamic-sized dimensions within one shape, like in this complete program:

use mdarray::{array, Const};

fn main() {
    let a = array![[0.0, 1.0], [1.0, 1.0]];
    dbg!(std::any::type_name_of_val(&a));
    let a = a.reshape((Const::<2>, !0));
    dbg!(std::any::type_name_of_val(&a));
}

This creates a constant-shaped array and then a view into it that is constant-shaped in one dimension, and dynamically-shaped in the other. Here is the output:

[src/main.rs:5:5] std::any::type_name_of_val(&a) = "mdarray::array::Array<f64, (mdarray::dim::Const<2>, mdarray::dim::Const<2>)>"
[src/main.rs:7:5] std::any::type_name_of_val(&a) = "mdarray::view::View<f64, (mdarray::dim::Const<2>, usize)>"

Under the hood this is realized using macros. Non-dynamic shapes work up to a maximum number of dimensions of 6.

1 Like

this is very cool, i wish i found this earlier!! i hope language support for dynamic dimensionality lands at some point, but this is a very nice implementation. thank you so much for the link

I see that it was not mentioned in the thread, but there are no objections against arrays as const generics. Even more, Rust will eventually support a lot of different types as const generics. It's just not trivial to implement and there are more priorities, blockers, limited development capacity.

2 Likes