Skip to content

Commit a62c795

Browse files
committed
Make the slice iterator optimise better.
Due to the null pointer optimisation, LLVM was leaving totally useless null pointer checks in tight loops using this iterator. We compile `Some(a_reference)` to a no-op, and matching against None in an `Option<&T>` is just comparing against 0, this means that the `None` branch in `match Some(a_reference) { None => ... }` (as appears effectively in a loop using `slice::Iter`) can't be eliminated unless `a_reference` is known to be non-null. LLVM currently doesn't seem to understand this in every situation (often due to inlining, I am told), particularly painfully in ones involving this iterator, but using `assume` means it understands almost always. for i in x { test::black_box(i); } Before: .LBB0_4: leaq 4(%rcx), %rsi testq %rcx, %rcx je .LBB0_5 movq %rcx, (%rsp) #APP #NO_APP cmpq %rsi, %rax movq %rsi, %rcx jne .LBB0_4 After: .LBB0_4: movq %rcx, (%rsp) addq $4, %rcx #APP #NO_APP cmpq %rcx, %rax jne .LBB0_4 This leads to significantly better unrolling and vectorisation too.
1 parent 583c5c5 commit a62c795

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

src/libcore/slice.rs

+10
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ use cmp::{Ordering, PartialEq, PartialOrd, Eq, Ord};
4040
use cmp::Ordering::{Less, Equal, Greater};
4141
use cmp;
4242
use default::Default;
43+
use intrinsics;
4344
use iter::*;
4445
use marker::Copy;
4546
use num::Int;
@@ -690,6 +691,13 @@ macro_rules! iterator {
690691
let old = self.ptr;
691692
self.ptr = self.ptr.offset(1);
692693

694+
// LLVM doesn't properly recognise that
695+
// the `&` is never null without this, and
696+
// so doesn't see that the Option we
697+
// create here is always Some, due to the
698+
// null-pointer optimisation.
699+
intrinsics::assume(!old.is_null());
700+
693701
Some(transmute(old))
694702
}
695703
}
@@ -723,6 +731,8 @@ macro_rules! iterator {
723731
} else {
724732
self.end = self.end.offset(-1);
725733

734+
// see above
735+
intrinsics::assume(!self.end.is_null());
726736
Some(transmute(self.end))
727737
}
728738
}

0 commit comments

Comments
 (0)