Taking slices

language design
1

Not sure if this is the place to but I have an company-internal library where I am walking across byte array with an iterator and passing the iterator down by reference to different functions to do slightly different things to subsets of the bytes without any copying etc... I then want to pass subslices of the data to functions in external crates without copying the data into a Vec first, something like this:

fn some_fun(data: &[T]) -> Result<Vec<U>> {
     let mut data_it = data.iter();
     pull_metadata_off_first_n_bytes(data_it.by_ref());
     let x = deserialize_some_value::<usize>(data_it.by_ref());

     let y: &[T] = data_it.take(x).as_slice(); // this doesn't exist
   
     let z = some_crate::do_func(y);
   ...
}

To make it all work with no copies I ended up creating a trait

pub(crate) trait SliceIterExt<'a, T>: std::iter::Iterator {
    fn as_slice(&self) -> &'a [T];

    fn take_slice<'b>(&'b mut self, n: usize) -> &'a [T] {
        let return_val = &self.as_slice()[0..n];
        // advance iterator by n elements
        self.nth(n - 1);
        return_val
    }
}

impl<'a, T> SliceIterExt<'a, T> for std::slice::Iter<'a, T> {
    fn as_slice(&self) -> &'a [T] {
        self.as_slice()
    }
}

Which is simple enough but also left me feeling this should probably already be in the language?

Maybe std::iter::Iterator could have take redefined to return something that implements "Takeable" then we implement Takeable for the existing Take struct and in std::slice::Iter we return std::slice::SliceableTake which also implements Takeable and has as_slice implemented.

1 Like
2

But plenty of iterators produce items from sources other than underlying slices, so they can't return a borrowed slice into the backing store (as it doesn't exist).

This seems like something that might only really be applicable to slice::Iter etc, so something along the lines of your current solution seems fairly sensible? I guess if there's enough community demand for it, it could be added to the standard library (either in trait form or perhaps just as an intrinsic method).

3

SliceableTake would only be implemented for the std::slice::Iter 's impl Iterator... so this would be specific to std::slice::Iter

Other non sliceable iterators could still just use Take but making it a trait empowers users to write custom iterators that have even more functionality that still interact with the standard iterator ecosystem.

4

This would be a breaking chance, since calling take on a core::slice::Iter currently returns Take<Iter>, which someone might be use by name or something like that.

Moreover core::slice::Iter already has a as_slice method that makes this possible to implement, so the functionality is not missing, just a bit less ergonomic.

5

Sadly this works less well than one would hope for some possible implementations due to complicated nuances about pointer provenance.

But yeah, it's still easily written as something like

pub fn take_first_chunk<'a, T, const N: usize>(x: &mut std::slice::Iter<'a, T>) -> Option<&'a [T; N]> {
    let chunk = x.as_slice().first_chunk()?;
    if N > 0 {
        x.nth(N - 1).unwrap();
    }
    Some(chunk)
}

And it optimizes great: https://rust.godbolt.org/z/xscEzdon6

1 Like
6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.