llvm-project.git/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp, branch main

Revert "[LV] Use ExtractLane(LastActiveLane, V) live outs when tail-folding. (#149042)"

2025-11-13T22:34:55+00:00

This reverts commit 62d1a080e69e3c5e98840e000135afa7c688a77b.

This appears to be causing some runtime failures on RISCV
https://lab.llvm.org/buildbot/#/builders/210/builds/5221

[LV] Use ExtractLane(LastActiveLane, V) live outs when tail-folding. (#149042)

2025-11-12T15:11:00+00:00

Building on top of https://github.com/llvm/llvm-project/pull/148817,
introduce a new abstract LastActiveLane opcode that gets lowered to
Not(Mask) → FirstActiveLane(NotMask) → Sub(result, 1).

When folding the tail, update all extracts for uses outside the loop the
extract the value of the last actice lane.

See also https://github.com/llvm/llvm-project/issues/148603

PR: https://github.com/llvm/llvm-project/pull/149042

[VPlan] Verify incoming values of VPIRPhi matches before checking (NFC)

2025-11-04T18:34:14+00:00

Update the verifier to first check if the number of incoming values
matches the number of predecessors, before using
incoming_values_and_blocks. We unfortunately need also check here, as
this may be called before verifyPhiRecipes runs.

Also update the verifier unit tests, to actually fail for the expected
recipes.

[VPlan] Use switch over opcodes in verifier (NFC).

2025-10-15T19:10:22+00:00

Preparation to make it easier to extend to verify additional opcodes.

[VPlan] Strip VPDT's default constructor (NFC) (#162692)

2025-10-13T10:16:05+00:00

[VPlan] Allow zero-operand m_BranchOn(Cond|Count) (NFC) (#162721)

2025-10-13T07:50:09+00:00

[VPlan] Allow multiple users of (broadcast %evl).

2025-09-27T20:47:54+00:00

CSE may replace multiple redundant broadcasts of EVL with a single
broadcast which may have more than 1 user. Adjust the verifier to allow
this.

Fixes a crash when building llvm-test-suite with EVL:
https://lab.llvm.org/buildbot/#/builders/210/builds/3303

[VPlanPatternMatch] Introduce match functor (NFC) (#159521)

2025-09-18T09:36:12+00:00

Follow up on 7fb3a91 ([PatternMatch] Introduce match functor) to
introduce the VPlanPatternMatch version of the match functor to shorten
some idioms.

Co-authored-by: Luke Lau

[LV][EVL] Support interleaved access with tail folding by EVL (#152070)

2025-09-01T13:20:06+00:00

The InterleavedAccess pass already supports transforming
vector-predicated (vp) load/store intrinsics. With this patch, we start
enabling interleaved access under tail folding by EVL.

This patch introduces a new base class, VPInterleaveBase, and a concrete
class, VPInterleaveEVLRecipe. Both the existing VPInterleaveRecipe and
the new VPInterleaveEVLRecipe inherit from and implement
VPInterleaveBase.

Compared to VPInterleaveRecipe, VPInterleaveEVLRecipe adds an EVL
operand to emit vp.load/vp.store intrinsics.

Currently, tail folding by EVL is only supported for scalable
vectorization. Therefore, VPInterleaveEVLRecipe will only emit
interleave/deinterleave intrinsics. Reverse accesses are not yet
implemented, as masked reverse interleaved access under tail folding is
not yet supported.

Fixed #123201

[VPlan] Add VPBlockBase::hasPredecessors (NFC).

2025-09-01T08:44:49+00:00

Split off from https://github.com/llvm/llvm-project/pull/154510/, add
helper to check if a block has any predecessors.