<feed xmlns='http://www.w3.org/2005/Atom'>
<title>llvm-project.git/llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp, branch users/nico/python-2</title>
<subtitle>Unnamed repository; edit this file 'description' to name the repository.
</subtitle>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/'/>
<entry>
<title>[VPlan] Don't convert VPWidenSelectRecipes to vp.select in EVL transform (#146695)</title>
<updated>2025-07-03T10:50:25+00:00</updated>
<author>
<name>Luke Lau</name>
<email>luke@igalia.com</email>
</author>
<published>2025-07-03T10:50:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=ec25a0568cfbd13a6e375d3a2295b706708d37c5'/>
<id>ec25a0568cfbd13a6e375d3a2295b706708d37c5</id>
<content type='text'>
createEVLRecipe tries to optimise recipes that use the header mask by
replacing them with their VP equivalents and setting the EVL, allowing
the mask to be removed.

However we currently also convert widened selects to vp.select even
though they don't necessarily use the header mask.

Unlike vp.merge a vp.select only makes the "unused" lanes past EVL
poison, so it's not needed for correctness.

In the same vein as #127180, this patch removes the transform for
VPWidenSelectRecipes and keeps them as plain select instructions to
allow for more optimisations.

RISCVVLOptimizer will still be able to optimise away any VL toggles and
we end up with better code generation across llvm-test-suite and SPEC
CPU 2017.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
createEVLRecipe tries to optimise recipes that use the header mask by
replacing them with their VP equivalents and setting the EVL, allowing
the mask to be removed.

However we currently also convert widened selects to vp.select even
though they don't necessarily use the header mask.

Unlike vp.merge a vp.select only makes the "unused" lanes past EVL
poison, so it's not needed for correctness.

In the same vein as #127180, this patch removes the transform for
VPWidenSelectRecipes and keeps them as plain select instructions to
allow for more optimisations.

RISCVVLOptimizer will still be able to optimise away any VL toggles and
we end up with better code generation across llvm-test-suite and SPEC
CPU 2017.
</pre>
</div>
</content>
</entry>
<entry>
<title>[VPlan] Emit VPVectorEndPointerRecipe for reverse interleave pointer adjustment (#144864)</title>
<updated>2025-07-02T10:16:02+00:00</updated>
<author>
<name>Mel Chen</name>
<email>mel.chen@sifive.com</email>
</author>
<published>2025-07-02T10:16:02+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=bc8dad1c7ef02f9a9d8a07c242fc3db470a7579b'/>
<id>bc8dad1c7ef02f9a9d8a07c242fc3db470a7579b</id>
<content type='text'>
A reverse interleave access is essentially composed of multiple
load/store operations with same negative stride, and their addresses are
based on the last lane address of member 0 in the interleaved group.

Currently, we already have VPVectorEndPointerRecipe for computing the
last lane address of consecutive reverse memory accesses. This patch
extends VPVectorEndPointerRecipe to support constant stride and extracts
the reverse interleave group address adjustment from
VPInterleaveRecipe::execute, replacing it with a
VPVectorEndPointerRecipe.

The final goal is to support interleaved accesses with EVL tail folding.
Given that VPInterleaveRecipe is large and tightly coupled — combining
both load and store, and embedding operations like reverse pointer
adjustion (GEP), widen load/store, deinterleave/interleave, and reversal
— breaking it down into smaller, dedicated recipes may allow
VPlanTransforms::tryAddExplicitVectorLength to lower them into EVL-aware
form more effectively.

One foreseeable challenge is that
VPlanTransforms::convertToConcreteRecipes currently runs after
tryAddExplicitVectorLength, so decomposing VPInterleaveRecipe will
likely need to happen earlier in the pipeline to be effective.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A reverse interleave access is essentially composed of multiple
load/store operations with same negative stride, and their addresses are
based on the last lane address of member 0 in the interleaved group.

Currently, we already have VPVectorEndPointerRecipe for computing the
last lane address of consecutive reverse memory accesses. This patch
extends VPVectorEndPointerRecipe to support constant stride and extracts
the reverse interleave group address adjustment from
VPInterleaveRecipe::execute, replacing it with a
VPVectorEndPointerRecipe.

The final goal is to support interleaved accesses with EVL tail folding.
Given that VPInterleaveRecipe is large and tightly coupled — combining
both load and store, and embedding operations like reverse pointer
adjustion (GEP), widen load/store, deinterleave/interleave, and reversal
— breaking it down into smaller, dedicated recipes may allow
VPlanTransforms::tryAddExplicitVectorLength to lower them into EVL-aware
form more effectively.

One foreseeable challenge is that
VPlanTransforms::convertToConcreteRecipes currently runs after
tryAddExplicitVectorLength, so decomposing VPInterleaveRecipe will
likely need to happen earlier in the pipeline to be effective.</pre>
</div>
</content>
</entry>
<entry>
<title>[VPlan] Add VPExpressionRecipe, replacing extended reduction recipes. (#144281)</title>
<updated>2025-07-01T19:44:50+00:00</updated>
<author>
<name>Florian Hahn</name>
<email>flo@fhahn.com</email>
</author>
<published>2025-07-01T19:44:50+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=6b3d2b629cc3805d483ec4522cc97e2540070ae1'/>
<id>6b3d2b629cc3805d483ec4522cc97e2540070ae1</id>
<content type='text'>
This patch adds a new recipe to combine multiple recipes into an
'expression' recipe, which should be considered as single entity for
cost-modeling and transforms. The recipe needs to be 'decomposed', i.e.
replaced by its individual recipes before execute.

This subsumes VPExtendedReductionRecipe and
VPMulAccumulateReductionRecipe and should make it easier to extend to
include more types of bundled patterns, like e.g. extends folded into
loads or various arithmetic instructions, if supported by the target.

It allows avoiding re-creating the original recipes when converting to
concrete recipes, together with removing the need to record various
information. The current version of the patch still retains the original
printing matching VPExtendedReductionRecipe and
VPMulAccumulateReductionRecipe, but this specialized print could be
replaced with printing the bundled recipes directly.

PR: https://github.com/llvm/llvm-project/pull/144281</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This patch adds a new recipe to combine multiple recipes into an
'expression' recipe, which should be considered as single entity for
cost-modeling and transforms. The recipe needs to be 'decomposed', i.e.
replaced by its individual recipes before execute.

This subsumes VPExtendedReductionRecipe and
VPMulAccumulateReductionRecipe and should make it easier to extend to
include more types of bundled patterns, like e.g. extends folded into
loads or various arithmetic instructions, if supported by the target.

It allows avoiding re-creating the original recipes when converting to
concrete recipes, together with removing the need to record various
information. The current version of the patch still retains the original
printing matching VPExtendedReductionRecipe and
VPMulAccumulateReductionRecipe, but this specialized print could be
replaced with printing the bundled recipes directly.

PR: https://github.com/llvm/llvm-project/pull/144281</pre>
</div>
</content>
</entry>
<entry>
<title>[LV] Use vscale for tuning to improve branch weight estimates (#144733)</title>
<updated>2025-07-01T12:23:38+00:00</updated>
<author>
<name>David Sherwood</name>
<email>david.sherwood@arm.com</email>
</author>
<published>2025-07-01T12:23:38+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=9b13dfdfbc6d3ca3ec0e3e6d9f5e9271eee13258'/>
<id>9b13dfdfbc6d3ca3ec0e3e6d9f5e9271eee13258</id>
<content type='text'>
In addBranchWeightToMiddleTerminator we attempt to add branch weights to
the middle block terminator. We pessimistically assume vscale=1, whereas
we can improve the estimate by using the value of vscale used for
tuning.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
In addBranchWeightToMiddleTerminator we attempt to add branch weights to
the middle block terminator. We pessimistically assume vscale=1, whereas
we can improve the estimate by using the value of vscale used for
tuning.</pre>
</div>
</content>
</entry>
<entry>
<title>[VPlan] Support VPWidenIntOrFpInductionRecipes with EVL tail folding (#144666)</title>
<updated>2025-07-01T11:29:24+00:00</updated>
<author>
<name>Luke Lau</name>
<email>luke@igalia.com</email>
</author>
<published>2025-07-01T11:29:24+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=4a2fa0847fb07d181ef71798cb8d7f4c34599fbf'/>
<id>4a2fa0847fb07d181ef71798cb8d7f4c34599fbf</id>
<content type='text'>
Following on from #118638, this handles widened induction variables with
EVL tail folding by setting the VF operand to be EVL, calculated in the
vector body.

We need to do this for correctness since with EVL tail folding the
number of elements processed in the penultimate iteration may not be VF,
but the runtime EVL, and we need take this into account when updating
the backedge value.

- Because the VF may now not be a live-in we need to move the insertion
point to just after the VFs definition
- We also need to avoid truncating it when it's the same size as the
step type, previously this wasn't a problem for live-ins.
- Also because the VF may be smaller than the IV type, since the EVL is
always i32, we may need to zext it.

On -march=rva23u64 -O3 we get 87.1% more loops vectorized on TSVC, and
42.8% more loops vectorized on SPEC CPU 2017</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Following on from #118638, this handles widened induction variables with
EVL tail folding by setting the VF operand to be EVL, calculated in the
vector body.

We need to do this for correctness since with EVL tail folding the
number of elements processed in the penultimate iteration may not be VF,
but the runtime EVL, and we need take this into account when updating
the backedge value.

- Because the VF may now not be a live-in we need to move the insertion
point to just after the VFs definition
- We also need to avoid truncating it when it's the same size as the
step type, previously this wasn't a problem for live-ins.
- Also because the VF may be smaller than the IV type, since the EVL is
always i32, we may need to zext it.

On -march=rva23u64 -O3 we get 87.1% more loops vectorized on TSVC, and
42.8% more loops vectorized on SPEC CPU 2017</pre>
</div>
</content>
</entry>
<entry>
<title>[VPlan] Replace all uses of VF when EVL tail folding. NFCI (#146339)</title>
<updated>2025-06-30T12:47:38+00:00</updated>
<author>
<name>Luke Lau</name>
<email>luke@igalia.com</email>
</author>
<published>2025-06-30T12:47:38+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=f01a7936bea491324e06dcf203749dda08ce7c07'/>
<id>f01a7936bea491324e06dcf203749dda08ce7c07</id>
<content type='text'>
With EVL tail folding, any use of the VF live in should be replaced by
the EVL. Otherwise, it should likely be directly emitted as a constant
via VPTransformState::VF.

This strengthens the EVL transformation by replacing all uses of VF with
EVL and asserting that the only users are VPVectorEndPointerRecipe and
VPScalarIVStepsRecipe, the latter of which is new.

This should be NFC because even though we didn't previously replace the
EVL of VPScalarIVStepsRecipe, it's only used when unrolling which we
don't allow with EVL tail folding yet.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
With EVL tail folding, any use of the VF live in should be replaced by
the EVL. Otherwise, it should likely be directly emitted as a constant
via VPTransformState::VF.

This strengthens the EVL transformation by replacing all uses of VF with
EVL and asserting that the only users are VPVectorEndPointerRecipe and
VPScalarIVStepsRecipe, the latter of which is new.

This should be NFC because even though we didn't previously replace the
EVL of VPScalarIVStepsRecipe, it's only used when unrolling which we
don't allow with EVL tail folding yet.</pre>
</div>
</content>
</entry>
<entry>
<title>[VPlan] Fix crash when trying to narrow interleave group storing const.</title>
<updated>2025-06-29T20:29:12+00:00</updated>
<author>
<name>Florian Hahn</name>
<email>flo@fhahn.com</email>
</author>
<published>2025-06-29T20:29:12+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=b822a32659c3fd5afd75c8f028aaa142fbd68e20'/>
<id>b822a32659c3fd5afd75c8f028aaa142fbd68e20</id>
<content type='text'>
Use dyn_cast_null to handle the case where an interleave groups stores a
constant in any of its lanes.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Use dyn_cast_null to handle the case where an interleave groups stores a
constant in any of its lanes.
</pre>
</div>
</content>
</entry>
<entry>
<title>Revert "[VPlan] Allow derived IVs and scalar-steps in narrowing interleave."</title>
<updated>2025-06-29T13:40:03+00:00</updated>
<author>
<name>Florian Hahn</name>
<email>flo@fhahn.com</email>
</author>
<published>2025-06-29T13:38:42+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=f5ed863176dd286462cd5558723dfe445967fedf'/>
<id>f5ed863176dd286462cd5558723dfe445967fedf</id>
<content type='text'>
This reverts commit 2787759ef2e41b19f8bfde06fe9a26b25d1f5834.

This exposed a crash on some build bots. Revert to investigate.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This reverts commit 2787759ef2e41b19f8bfde06fe9a26b25d1f5834.

This exposed a crash on some build bots. Revert to investigate.
</pre>
</div>
</content>
</entry>
<entry>
<title>[VPlan] Allow derived IVs and scalar-steps in narrowing interleave.</title>
<updated>2025-06-29T12:18:51+00:00</updated>
<author>
<name>Florian Hahn</name>
<email>flo@fhahn.com</email>
</author>
<published>2025-06-29T12:18:50+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=2787759ef2e41b19f8bfde06fe9a26b25d1f5834'/>
<id>2787759ef2e41b19f8bfde06fe9a26b25d1f5834</id>
<content type='text'>
Both VPDerivedIVRecipe and VPScalarIVSteps recipe should be supported in
narrowInterleaveGroups:
 * VPDerivedIVRecipe is based on the canonical IV and independent of VF,
 * VPScalarIVSteps takes the VF as operand, so it will be updated by
   narrowInterleaveGroup.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Both VPDerivedIVRecipe and VPScalarIVSteps recipe should be supported in
narrowInterleaveGroups:
 * VPDerivedIVRecipe is based on the canonical IV and independent of VF,
 * VPScalarIVSteps takes the VF as operand, so it will be updated by
   narrowInterleaveGroup.
</pre>
</div>
</content>
</entry>
<entry>
<title>[VPlan] Simplify code in single scalar transform code (NFC).</title>
<updated>2025-06-28T21:53:14+00:00</updated>
<author>
<name>Florian Hahn</name>
<email>flo@fhahn.com</email>
</author>
<published>2025-06-28T21:52:55+00:00</published>
<link rel='alternate' type='text/html' href='https://git.belthelziquor.com/llvm-project.git/commit/?id=bdb299a67eb863b1b1e6241d7964e7108f1a42aa'/>
<id>bdb299a67eb863b1b1e6241d7964e7108f1a42aa</id>
<content type='text'>
Adjust code as suggested post-commit 3b7b95f78e2.

https://github.com/llvm/llvm-project/commit/3b7b95f78e24aca5175b8f864bfea2953a2dd34f#r160997427
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Adjust code as suggested post-commit 3b7b95f78e2.

https://github.com/llvm/llvm-project/commit/3b7b95f78e24aca5175b8f864bfea2953a2dd34f#r160997427
</pre>
</div>
</content>
</entry>
</feed>
