Skip to content

[LoopInterchange] Fix the vectorizable check for a loop #133667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 27 additions & 17 deletions llvm/lib/Transforms/Scalar/LoopInterchange.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1197,25 +1197,35 @@ LoopInterchangeProfitability::isProfitablePerInstrOrderCost() {
return std::nullopt;
}

/// Return true if we can vectorize the loop specified by \p LoopId.
static bool canVectorize(const CharMatrix &DepMatrix, unsigned LoopId) {
for (unsigned I = 0; I != DepMatrix.size(); I++) {
char Dir = DepMatrix[I][LoopId];
if (Dir != 'I' && Dir != '=')
return false;
}
return true;
}

std::optional<bool> LoopInterchangeProfitability::isProfitableForVectorization(
unsigned InnerLoopId, unsigned OuterLoopId, CharMatrix &DepMatrix) {
for (auto &Row : DepMatrix) {
// If the inner loop is loop independent or doesn't carry any dependency
// it is not profitable to move this to outer position, since we are
// likely able to do inner loop vectorization already.
if (Row[InnerLoopId] == 'I' || Row[InnerLoopId] == '=')
return std::optional<bool>(false);

// If the outer loop is not loop independent it is not profitable to move
// this to inner position, since doing so would not enable inner loop
// parallelism.
if (Row[OuterLoopId] != 'I' && Row[OuterLoopId] != '=')
return std::optional<bool>(false);
}
// If inner loop has dependence and outer loop is loop independent then it
// is/ profitable to interchange to enable inner loop parallelism.
// If there are no dependences, interchanging will not improve anything.
return std::optional<bool>(!DepMatrix.empty());
// If the outer loop is not loop independent it is not profitable to move
// this to inner position, since doing so would not enable inner loop
// parallelism.
if (!canVectorize(DepMatrix, OuterLoopId))
return false;

// If inner loop has dependence and outer loop is loop independent then it is
// profitable to interchange to enable inner loop parallelism.
if (!canVectorize(DepMatrix, InnerLoopId))
return true;

// If both the inner and the outer loop can be vectorized, it is necessary to
// check the cost of each vectorized loop for profitability decision. At this
// time we do not have a cost model to estimate them, so return nullopt.
// TODO: Estimate the cost of vectorized loop when both the outer and the
// inner loop can be vectorized.
return std::nullopt;
}

bool LoopInterchangeProfitability::isProfitable(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,13 @@
; }
; }
;
; FIXME: These loops are not exchanged at this time due to the problem in
; profitability heuristic calculation for vectorization.

; CHECK: --- !Missed
; CHECK: --- !Passed
; CHECK-NEXT: Pass: loop-interchange
; CHECK-NEXT: Name: InterchangeNotProfitable
; CHECK-NEXT: Name: Interchanged
; CHECK-NEXT: Function: interchange_necessary_for_vectorization
; CHECK-NEXT: Args:
; CHECK-NEXT: - String: Interchanging loops is not considered to improve cache locality nor vectorization.
; CHECK-NEXT: ...
; CHECK-NEXT: - String: Loop interchanged with enclosing loop.
define void @interchange_necessary_for_vectorization() {
entry:
br label %for.i.header
Expand Down