[Feature Request] Add async indexing

language design
1

Rust does a good job of clean syntax for indexing like most other languages in allowing one to do:

let value_ref = &data[1];
let values_ref = &data[1..10];
let value = data[1].clone();
let values = data[1..10].to_vec();
..etc

What would be great though is an interface that would allow for similar functionality but for async use. With async calls though you aren't just trying to return a reference to a point in memory, you are doing more than that so an option to return a value rather than just a reference would be good here. Something along the lines of:

pub trait Index<Idx: ?Sized>: !IndexAsync<Idx> {
    type Output: ?Sized;

    fn index(&self, index: Idx) -> &Self::Output;
}

#[async_trait]
pub trait IndexAsync<Idx>: !Index<Idx> {
    type Output;

    async fn index(&mut self, index: Idx) -> Self::Output;
}

A struct that would inherit this interface would then be able to do:

let value_ref = &data[1].await;
let values_ref = &data[1..10].await;
let value = data[1].await;
let values = data[1..10].await;
..etc
2

Can you elaborate more on when you'd like to use this?

My instinct here is that if it's doing enough to need to be async, then it can just be a normal method and doesn't need the terseness of the [] sugar.

3 Likes
3

I want to create a library for data loading from as many sources as possible and want it to have a super simple interface so new-comers from python etc have something close to what they were working with in their own languages. I'd like to support two different types of data loading: In memory and out of memory. In memory can be sync so indexing can be done the normal way. However for out of memory data loading I'd like to keep the ability to do indexing although due to io blocking it needs to be async. I still want the same nice syntax though hence this feature suggestion. Hope that clears this up a little? (A note on &mut self requirement also: This is to allow for caching support to be built in further down the line)

4

Note that caching tends to imply sharing, and thus will generally be done with &self + interior mutability, not &mut self.

5

How would this be any different than implementing Index<Output<Box<dyn Furure>>>?

6

Implementing that is not possible because the Index trait is required to return a reference, whereas the future needs to be owned.

7

I believe this falls victim to the same problem that GATifying the Index trait falls into: the arr[ix] syntax is not sugar for a function call. It's much more complicated than that.

The result of an expression arr[ix] is, in Rust terminology, a place. (In C++ terms, an lvalue.) In Rust, this isn't a "thing" that you can return from a function or bind in a binding, so it's kind of difficult to talk about. (In C++, you can have an lvalue reference, T&, which makes this kind of discussion easier. In fact, in C++, indexing is just sugar for a function returning an lvalue reference (with overloading semantics).) A place is effectively "what you get from a dereference operator".

What makes a place special is that while it's typed at T, you can use it without moving the value. When you take a reference to a place, you get the same reference that was dereferenced to obtain the place.

So, okay, indexing is sugar for not arr.index(ix), but *arr.index(ix), so that you can take it's address with & or &mut,,, and it... somehow decides whether to call index_mut instead, if you "use the place via mutable reference".

And this interaction with the context is what really drives the nail in the coffin of returning anything other than references from index, imho. You could maybe get around the immediate dereference by just saying that &arr[ix] returns your special handle, rather than a regular reference, and the same for &mut arr[ix].

But how do you resolve method syntax?

Via autoref, .method() can either use the place by-move, by-ref, or by-mut. This is resolved by the signature of T::method and what the receiver type is, and then that receiver type is back-propagated to decide whether to call Index::index or IndexMut::index_mut.

There is no way to resolve this for custom handle types. The behavior of indexing syntax is tied to the details of returning built-in references in order to be a place expression and act how people expect.

I'm pretty sure I said it the last time I talked about this, but I really should submit a design notes document for "why Index can't be GATified".

And now I have to note that you've somewhat sidestepped this issue with the OP. Notably, you've:

  • Used a theoretical mutually exclusive trait to the Index trait, avoiding Index compatibility hazards;
  • Mandated &mut access to the indexee, avoiding resolution concerns;
  • Mandated returning a (boxed dyn future returning an) owned value, avoiding the problem of returning a place; and
  • Hidden complexity behind #[async_trait] (though this (async fn in traits) is a likely future feature).

So maybe my whole rant about GAT Index doesn't apply. But I think the spirit of the issue remains: Index is special, and your IndexAsync isn't special, it's just a regular function call.

So really, the question stands: why should this be put on the [] syntax, when it behaves quite unlike the existing [] syntax? If your goal isn't to produce a place, what you want is a function call.

8 Likes
8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.