Skip to content

Add ExactSizeIterator impl for iter::Chain #66531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions src/libcore/iter/adapters/chain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,36 @@ impl<A, B> FusedIterator for Chain<A, B>
B: FusedIterator<Item=A::Item>,
{}

#[stable(feature = "chain_exact_size", since = "1.41.0")]
impl<A, B> ExactSizeIterator for Chain<A, B>
where
A: ExactSizeIterator,
B: ExactSizeIterator<Item = A::Item>,
{
/// Returns the exact number of times the iterator will iterate.
///
/// # Overflow Behavior
///
/// Calling this method on an iterator with more than [`usize::MAX`]
/// elements will result in a panic.
///
/// # Panics
///
/// This panics if the iterator has more than [`usize::MAX`] elements.
///
/// [`usize::MAX`]: ../../std/usize/constant.MAX.html
fn len(&self) -> usize {
match self.state {
ChainState::Both => {
self.a.len().checked_add(self.b.len())
.expect("called `len` on iterator with more than `usize::MAX` elements")
}
ChainState::Front => self.a.len(),
ChainState::Back => self.b.len(),
}
}
}

#[unstable(feature = "trusted_len", issue = "37572")]
unsafe impl<A, B> TrustedLen for Chain<A, B>
where A: TrustedLen, B: TrustedLen<Item=A::Item>,
Expand Down
24 changes: 24 additions & 0 deletions src/libcore/iter/traits/iterator.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
// ignore-tidy-filelength
// This file almost exclusively consists of the definition of `Iterator`. We
// can't split that into multiple files.

use crate::cmp::{self, Ordering};
use crate::ops::{Add, Try};

Expand Down Expand Up @@ -388,6 +392,26 @@ pub trait Iterator {
/// [`once`] is commonly used to adapt a single value into a chain of
/// other kinds of iteration.
///
///
/// # Overflowing behavior for long iterators
///
/// This method allows to easily build an iterator that yields more than
/// [`usize::MAX`] items. In that case, some methods that return `usize`
/// are not guarded against overflow. For example, this includes the
/// following methods:
///
/// - [`Iterator::count`]
/// - [`Iterator::enumerate`]
/// - [`Iterator::position`] and [`Iterator::rposition`]
/// - [`ExactSizeIterator::len`]
///
/// An overflow in those methods leads to a wrong result or a panic. If
/// debug assertions are enabled, a panic is guaranteed.
Comment on lines +408 to +409
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that we could guarantee a panic whether or not debug assertions are enabled? Reserving the right to give a wrong result doesn't seem helpful here.

Copy link
Contributor

@timvermeulen timvermeulen Jan 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reserving this right is "helpful" in the sense that it means we don't have any performance overhead from overflow checks in release builds.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems wrong that len() could return something smaller than size_hint().0 on an iterator that implements TrustedLen. What sort of overhead are we talking about in practice?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was referring to all methods listed in that doc comment, not just len. Iterator::count currently does not check for overflow in release builds because it simply adds 1 repeatedly. In the case of len, I agree that it should probably panic rather than return an incorrect value, if we were to land this change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

len guarantees a panic on overflow. I changed this in response to this comment.

This sentence here is a bit vague, yes. It's still technically correct for len, but maybe I should add something like "ExactIterator::len even guarantees a panic in release mode"? On the other hand, this behavior is already documented on that method (see above).

As for the other methods: I don't think it's in the scope of this PR to change them. And currently they only have debug checks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. In that case I am on board with this change. Want to propose FCP?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dtolnay I'm not quite sure if you want me to add another sentence to this part of the documentation or not. (Will rebase either way soon.)

Regarding FCP: the main concern is this one. I replied to that comment, but no one else did, so I don't think it has been resolved yet. But maybe FCP is a way to get the others to look at this?

///
///
/// [`usize::MAX`]: ../../std/usize/constant.MAX.html
///
///
/// # Examples
///
/// Basic usage:
Expand Down
44 changes: 44 additions & 0 deletions src/libcore/tests/iter.rs
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,50 @@ fn test_iterator_chain_size_hint() {
assert_eq!(iter.size_hint(), (0, Some(0)));
}

#[test]
fn test_iterator_chain_len() {
let xs = [0, 1, 2];
let ys = [30, 40, 50, 60];

// First iterator is exhausted first
let mut iter = xs.iter().chain(&ys);
assert_eq!(iter.len(), 7);
assert_eq!(iter.next(), Some(&0));
assert_eq!(iter.len(), 6);
assert_eq!(iter.next(), Some(&1));
assert_eq!(iter.len(), 5);
assert_eq!(iter.next_back(), Some(&60));
assert_eq!(iter.len(), 4);
assert_eq!(iter.next(), Some(&2));
assert_eq!(iter.len(), 3);
assert_eq!(iter.next(), Some(&30));
assert_eq!(iter.len(), 2);
assert_eq!(iter.next(), Some(&40));
assert_eq!(iter.len(), 1);
assert_eq!(iter.next(), Some(&50));
assert_eq!(iter.len(), 0);
assert_eq!(iter.next(), None);

// Second iterator is exhausted first
let mut iter = xs.iter().chain(&ys);
assert_eq!(iter.len(), 7);
assert_eq!(iter.next_back(), Some(&60));
assert_eq!(iter.len(), 6);
assert_eq!(iter.next(), Some(&0));
assert_eq!(iter.len(), 5);
assert_eq!(iter.next_back(), Some(&50));
assert_eq!(iter.len(), 4);
assert_eq!(iter.next_back(), Some(&40));
assert_eq!(iter.len(), 3);
assert_eq!(iter.next_back(), Some(&30));
assert_eq!(iter.len(), 2);
assert_eq!(iter.next(), Some(&1));
assert_eq!(iter.len(), 1);
assert_eq!(iter.next(), Some(&2));
assert_eq!(iter.len(), 0);
assert_eq!(iter.next(), None);
}

#[test]
fn test_zip_nth() {
let xs = [0, 1, 2, 4, 5];
Expand Down