[docs] document the plumbing APIs and so forth that are used to make your own parallel iterators

[bluss]

This would be very helpful. Is there any existing documentation?

Also, for usability reasons, I guess the internal methods and parts are hidden in rustdoc. But it's working against those that want to implement parallel iterators.

[cuviper]

There's src/par_iter/README.md with a design discussion, and I'd also suggest people to work by example from existing iterators.

IMO it's hard to know what's lacking until people start asking questions, which is how we got the ProducerCallback explanation, for example.

I'm also not sure whether @nikomatsakis wants the internals to be considered part of the stable API, which might be another reason to hide their docs.

[bluss]

Thanks for that pointer. → Link: par_iter/README.md

I just started working on it, but the lack of starting point in the docs had put me off for a long time, this time I just decided to toss in the impl ParallelIterator for <existing Iterator> and see what the compiler said I needed to do. The result is more or less this (see par.rs)

https://github.com/bluss/rust-ndarray/compare/par-iter-wip-1?expand=1

No work needed to implement ParallelIterator for AxisIter, since it already supports iteration and splitting. That's great! Just needed to figure out the boiler plate then.

The import list tells you that there is a moderate amount of it:

+use rayon::par_iter::ParallelIterator;
+use rayon::par_iter::IndexedParallelIterator;
+use rayon::par_iter::ExactParallelIterator;
+use rayon::par_iter::BoundedParallelIterator;
+use rayon::par_iter::internal::{Consumer, UnindexedConsumer};
+use rayon::par_iter::internal::bridge;
+use rayon::par_iter::internal::ProducerCallback;
+use rayon::par_iter::internal::Producer;
+

Right at this point when I'm showing the WIP is when I realize that the Iterator and ParallelIterator have to be separate types, because the two traits overlap in their method names. I suppose that's reasonable.

[bluss]

It's been publically indicated it is unsure which parts of the library are for external consumption. Here and in https://github.com/nikomatsakis/rayon/pull/188#issuecomment-269626504

The crucial parts for implementing parallel iterators are also hidden from docs.

While this is unclear, it is not possible to depend on rayon for the purpose of implementing ParallelIterator and related traits. (Which is frustrating, because implementation in ndarray is now almost done, at least well enough for the first release.)

[cuviper]

See #209 -- I didn't actually add any new documentation yet, but I did un-hide some of those APIs required for implementing iterators, and this opens the door for writing proper documentation on them.

[bluss]

Using strategy "2" of an intermediate crate called ndarray-parallel is in the works rust-ndarray/ndarray#264

[nikomatsakis]

I'm bumping this up to 1.0 milestone -- I think at least some basic docs are needed, just because I want to turn on #[deny(missing_docs)] =)

[nikomatsakis]

Done