1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-22 10:42:39 +01:00
llvm-mirror/lib/Transforms/Vectorize
David Sherwood cdd50ed2ff [LoopVectorize] Don't interleave scalar ordered reductions for inner loops
Consider the following loop:

  void foo(float *dst, float *src, int N) {
    for (int i = 0; i < N; i++) {
      dst[i] = 0.0;
      for (int j = 0; j < N; j++) {
        dst[i] += src[(i * N) + j];
      }
    }
  }

When we are not building with -Ofast we may attempt to vectorise the
inner loop using ordered reductions instead. In addition we also try
to select an appropriate interleave count for the inner loop. However,
when choosing a VF=1 the inner loop will be scalar and there is existing
code in selectInterleaveCount that limits the interleave count to 2
for reductions due to concerns about increasing the critical path.
For ordered reductions this problem is even worse due to the additional
data dependency, and so I've added code to simply disable interleaving
for scalar ordered reductions for now.

Test added here:

  Transforms/LoopVectorize/AArch64/strict-fadd-vf1.ll

Differential Revision: https://reviews.llvm.org/D106646
2021-07-27 17:41:01 +01:00
..
CMakeLists.txt
LoadStoreVectorizer.cpp [LoadStoreVectorizer] Support opaque pointers 2021-06-27 15:42:16 +02:00
LoopVectorizationLegality.cpp [LV] Enable vectorization of multiple exit loops w/computable exit counts 2021-07-15 08:53:51 -07:00
LoopVectorizationPlanner.h [LoopVectorize] Fix strict reductions where VF = 1 2021-06-28 11:27:10 +01:00
LoopVectorize.cpp [LoopVectorize] Don't interleave scalar ordered reductions for inner loops 2021-07-27 17:41:01 +01:00
SLPVectorizer.cpp [SLP]Fix costs calculations. 2021-07-26 07:14:03 -07:00
VectorCombine.cpp [IR] Deprecate GetElementPtrInst::CreateInBounds without element type 2021-07-04 16:49:30 +02:00
Vectorize.cpp
VPlan.cpp Recommit "[VPlan] Add recipe for first-order rec phis, make splicing explicit." 2021-07-26 15:50:30 +01:00
VPlan.h Recommit "[VPlan] Add recipe for first-order rec phis, make splicing explicit." 2021-07-26 15:50:30 +01:00
VPlanDominatorTree.h
VPlanHCFGBuilder.cpp
VPlanHCFGBuilder.h
VPlanLoopInfo.h
VPlanPredicator.cpp
VPlanPredicator.h
VPlanSLP.cpp VPlanSLP.cpp - tidy implicit header dependencies. NFCI. 2021-06-13 12:37:17 +01:00
VPlanTransforms.cpp Recommit "[VPlan] Add recipe for first-order rec phis, make splicing explicit." 2021-07-26 15:50:30 +01:00
VPlanTransforms.h [VPlan] Merge predicated-triangle regions, after sinking. 2021-06-28 11:10:38 +01:00
VPlanValue.h Recommit "[VPlan] Add recipe for first-order rec phis, make splicing explicit." 2021-07-26 15:50:30 +01:00
VPlanVerifier.cpp
VPlanVerifier.h
VPRecipeBuilder.h [VPlan] Track both incoming values for first-order recurrence phis. 2021-06-27 14:29:35 +01:00