Ideas for Result/Error handling

[antonWetzel]

Problem

I got a file with independant chunks and want to do an expensive computation for every chunk.

chunks.into_par_iter().for_each_init(
    || std::fs::File::open(&path).unwrap(),
    |file, (chunk_start, chunk_length)| expensive_computation(file, chunk_start, chunk_length).unwrap(),
)

Now I want to handle the errors by returning, when an error occurs.

Trying to handle the errors

The error with the computations can be catched with an map instead of for_each

let res = chunks.into_par_iter().map_init(
    || std::fs::File::open(&path).unwrap(),
    |file, (chunk_start, chunk_length)| expensive_computation(file, chunk_start, chunk_length),
)
.find_any(|r| r.is_err()) // try find error
.unwrap_or(Ok(()) ?; // replace no Error with Ok(())

The error in the init is hard to catch, because it is always owned by rayon, not my code.

Questions

1. Did I overlook some clean obvious solution?
2. Has anyone similar problems (cancel on first error)?

Solutions?

(Note: concepts which may clash with rayon internals or other limitations)

For the computation

Let tasks without a return value return a generic TaskResult

trait TaskResult {
    fn cancel(&self) -> bool; // cancel the iterator to return this result
    fn ok() -> Self; // default if nothing cancels
}

impl TaskResult for () {
    fn cancel(&self) -> bool { false }
    fn ok() -> Self { () }
}

impl<V, E> TaskResult for Result<V, E> {
    fn cancel(&self) -> bool { self.is_err() }
    fn ok() -> Self { Ok(()) }
}

The for_each would check the TaskResult and maybe cancel. I hope this does not generate overhead for ().

For the init

The Result<_, Init> and the generic Init value can't be differentiated easily, so I don't see an easy solution.

It would be cool if the ..._with_init would be just a with_init method

// current
chunks
    .into_par_iter()
    .with_init(|| <init>)
    .for_each(|(<init>, chunk)| { ... })

and then maybe an error_filter, which collects the first encountered error

// dream
chunks
    .into_par_iter()
    .with_init(|| <result_init>)
    .error_filter() // no ? here, because erros did not occour yet
    .for_each(|(<init>, chunk)| { ... })
    .ok()?;

[cuviper]

I think try_for_each_init will make your computation errors easier to handle. I'm not sure about the init errors though. Something like this might work, though a little ugly:

chunks.into_par_iter().try_for_each_init(
    || Some(std::fs::File::open(&path)),
    |file, (chunk_start, chunk_length)| {
        let file = match file {
            Some(Ok(file)) => file, // use it!
            Some(Err(_)) => file.take().unwrap()?, // steal the option and return its inner error
            None => unreachable!(), // we shouldn't be called again after an error
        };
        // Now just return the computation's Result
        expensive_computation(file, chunk_start, chunk_length)
    },
)

It would be cool if the ..._with_init would be just a with_init method

We don't have a way to pass a borrowed reference as a regular Item in the method chain, similar to the problem of lending Iterator in the standard library (that may now be somewhat solved by GAT LendingIterator).