| Age | Commit message (Collapse) | Author |
|---|
|
other operand is already sext. (#142703)
This prevents us from ending up with (zext (sext X)). The zext will
require an instruction on targets where zext isn't free like RISC-V.
|
|
comparison in eliminateIVComparison. (#138215)
|
|
|
|
When IndVarSimplify simplifies a rem of the induction variable to a cmp
and select, only the select currently receives the rem's source
location; this patch propagates it to the cmp as well.
Found using https://github.com/llvm/llvm-project/pull/107279.
|
|
widening IV (#135207)
`WidenIV::widenWithVariantUse` assumes that exactly one of the binop
operands is the IV to be widened. This miscompilation happens when it
tries to sign-extend the "NonIV" operand while the IV is zero-extended.
Closes https://github.com/llvm/llvm-project/issues/135182.
|
|
Proof: https://alive2.llvm.org/ce/z/NVXaeo
|
|
Relevant parts of ScalarEvolution's API accept a CmpPredicate instead of
a CmpInst::Predicate after 60dc450 (SCEV: migrate to CmpPredicate
(NFC)). After auditing the callers of these APIs, it was found that
IndVarSimplify was dropping samesign information. Fix this.
|
|
With the introduction of CmpPredicate in 51a895a (IR: introduce struct
with CmpInst::Predicate and samesign), PatternMatch is one of the first
key pieces of infrastructure that must be updated to match a CmpInst
respecting samesign information. Implement this change to Cmp-matchers.
This is a preparatory step in migrating the codebase over to
CmpPredicate. Since we no functional changes are desired at this stage,
we have chosen not to migrate CmpPredicate::operator==(CmpPredicate)
calls to use CmpPredicate::getMatching(), as that would have visible
impact on tests that are not yet written: instead, we call
CmpPredicate::operator==(Predicate), preserving the old behavior, while
also inserting a few FIXME comments for follow-ups.
|
|
(#110938)
This macros is always defined: either 0 or 1. The correct pattern is to
use #if.
Re-apply #110185 with more fixes for debug build with the ABI breaking
checks disabled.
|
|
|
|
WideInc/WideIncExpr can be null. Previously this worked out
because the comparison with WideIncExpr would fail. Now we have
accesses to WideInc prior to that. Avoid the issue with an
explicit check.
Fixes https://github.com/llvm/llvm-project/issues/106239.
|
|
Use const SCEV * explicitly in more places to prepare for
https://github.com/llvm/llvm-project/pull/91961. Split off as suggested.
|
|
overflow/saturation instructions (#97987)
Fixes #97986
|
|
instructions (#97284)
Fixes #97282
|
|
Reverts the above commit, as it updates a common header function and
did not update all callsites:
https://lab.llvm.org/buildbot/#/builders/29/builds/382
This reverts commit 6481dc57612671ebe77fe9c34214fba94e1b3b27.
|
|
Uses the new InsertPosition class (added in #94226) to simplify some of
the IRBuilder interface, and removes the need to pass a BasicBlock
alongside a BasicBlock::iterator, using the fact that we can now get the
parent basic block from the iterator even if it points to the sentinel.
This patch removes the BasicBlock argument from each constructor or call
to setInsertPoint.
This has no functional effect, but later on as we look to remove the
`Instruction *InsertBefore` argument from instruction-creation
(discussed
[here](https://discourse.llvm.org/t/psa-instruction-constructors-changing-to-iterator-only-insertion/77845)),
this will simplify the process by allowing us to deprecate the
InsertPosition constructor directly and catch all the cases where we use
instructions rather than iterators.
|
|
Make it easier to access additional state from it.
|
|
Rename has/dropPoisonGeneratingFlagsOrMetadata to
has/dropPoisonGeneratingAnnotations and make it also handle
nonnull, align and range return attributes on calls, similar
to the existing handling for !nonnull, !align and !range metadata.
|
|
When inserting truncs during IV widening, mark the trunc as either nuw
or nsw depending on whether zext or sext widening was used. For
non-negative IVs both nuw and nsw apply.
|
|
IndVars may be able to replace a loop dependent condition with a loop
invariant one, but loop-unswitch runs before IndVars, so the invariant
check remains in the loop.
For an example, consider a read-only loop with a bounds check:
https://godbolt.org/z/8cdj4qhbG
This patch uses a approach similar to the way extra cleanup passes are
run on demand after vectorization (added in acea6e9cfa4c4a0e8678c7).
It introduces a new ShouldRunExtraSimpleLoopUnswitch analysis marker,
which IndVars can use to indicate that extra unswitching is beneficial.
ExtraSimpleLoopUnswitchPassManager uses this analysis to determine
whether to run its passes on a loop.
Compile-time impact (geomean) ranges from +0.0% to 0.02%
https://llvm-compile-time-tracker.com/compare.php?from=138c0beb109ffe47f75a0fe8c4dc2cdabe8a6532&to=19e6e99eeb280d426907ea73a21b139ba7225627&stat=instructions%3Au
Compile-time impact (geomean) of unconditionally running
SimpleLoopUnswitch ranges from +0.05% - +0.16%
https://llvm-compile-time-tracker.com/compare.php?from=138c0beb109ffe47f75a0fe8c4dc2cdabe8a6532&to=2930dfd5accdce2e6f8d5146ae4d626add2065a2&stat=instructions:u
Unconditionally running SimpleLoopUnswitch seems to indicate that there
are multiple other scenarios where we fail to run unswitching when
opportunities remain.
Fixes https://github.com/llvm/llvm-project/issues/85551.
PR: https://github.com/llvm/llvm-project/pull/81271
|
|
getExtendedOperandRecurrence. (#84282)
We can treat a shift by constant as a multiply by a power of 2
and we can treat an or disjoint as a 'add nsw nuw'.
I've added a helper struct similar to a struct used in
ScalarEvolution.cpp
to represent the opcode, operands, and NSW/NUW flags for normal
add/sub/mul
and shl/or that are being treated as mul/add.
I don't think we need to teach cloneIVUser about this. It will continue
to clone them using cloneBitwiseIVUser. After the cloning we will ask
for the SCEV expression for the cloned IV user and verify that it
matches
the AddRec returned by getExtendedOperandRecurrence. Since SCEV also
knows how to convert shl to mul and or disjoint to add nsw nuw, this
should
usually match. If it doesn't match, the cloned IV user will be deleted.
|
|
I'd reverted this in 6c7805d5d1 after a bad stage. Original commit
messsage follows:
[NFC][RemoveDIs] Bulk update utilities to insert with iterators
As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can carry a bit of
debug-info. This commit implements some of that by updating the contents of
llvm/lib/Transforms/Utils to always use iterator-versions of instruction
constructors.
There are two general flavours of update:
* Almost all call-sites just call getIterator on an instruction
* Several make use of an existing iterator (scenarios where the code is
actually significant for debug-info)
The underlying logic is that any call to getFirstInsertionPt or similar
APIs that identify the start of a block need to have that iterator passed
directly to the insertion function, without being converted to a bare
Instruction pointer along the way.
I've also switched DemotePHIToStack to take an optional iterator: it needs
to take an iterator, and having a no-insert-location behaviour appears to
be important. The constructors for ICmpInst and FCmpInst have been updated
too. They're the only instructions that take block _references_ rather than
pointers for certain calls, and a future patch is going to make use of
default-null block insertion locations.
All of this should be NFC.
|
|
This reverts commit 3fda50d3915b2163a54a37b602be7783a89dd808.
Apparently I've missed a hunk while staging this; will back out for now.
Picked up here: https://lab.llvm.org/buildbot/#/builders/139/builds/60429/steps/6/logs/stdio
|
|
As part of the RemoveDIs project we need LLVM to insert instructions using
iterators wherever possible, so that the iterators can carry a bit of
debug-info. This commit implements some of that by updating the contents of
llvm/lib/Transforms/Utils to always use iterator-versions of instruction
constructors.
There are two general flavours of update:
* Almost all call-sites just call getIterator on an instruction
* Several make use of an existing iterator (scenarios where the code is
actually significant for debug-info)
The underlying logic is that any call to getFirstInsertionPt or similar
APIs that identify the start of a block need to have that iterator passed
directly to the insertion function, without being converted to a bare
Instruction pointer along the way.
I've also switched DemotePHIToStack to take an optional iterator: it needs
to take an iterator, and having a no-insert-location behaviour appears to
be important. The constructors for ICmpInst and FCmpInst have been updated
too. They're the only instructions that take block _references_ rather than
pointers for certain calls, and a future patch is going to make use of
default-null block insertion locations.
All of this should be NFC.
|
|
(#78696)
In LoopUnroll, peelLoop is called on the loop. After the loop is peeled
it calls simplifyLoopAfterUnroll on the loop. This call to
simplifyLoopAfterUnroll doesn't preserve the LCSSA form of the parent
loop and thus during the next call to peelLoop the LCSSA form is already
broken.
LoopPeel util takes in the PreserveLCSSA argument and it passes
on the same argument to simplifyLoop which checks if the loop is in a
valid LCSSA form, when (PreserveLCSSA = true).
This causes an assert in simplifyLoop when (PreserveLCSSA = true), as
during the last call LCSSA for the loop wasn't preserved, and thus
crashes at the following assert.
assert(L->isRecursivelyLCSSAForm(*DT, *LI) &&
"Requested to preserve LCSSA, but it's already broken.");
Upon debugging, it is evident that simplifyLoopIVs call inside
simplifyLoopAfterUnroll breaks the LCSSA form. This patch fixes
llvm#77118, it checks if the replacement of IV Users with Loop Invariant
preserves the LCSSA form. If it does not, it emits the required LCSSA
Phi instructions.
|
|
widenIVUse may hoist a wide induction increment and introduce new uses,
but does not recompute the wrap flags. In some cases this can make the
new uses of the wide IV inc more poisonous.
Update the code to recompute flags if needed when hoisting an IV. If
both the narrow and wide IV increment's flags match and we can re-use
the flags from the increments, there's no need to recompute the flags,
as the replacement won't make the new uses of the wide IV's increment
more poisonous.
Note that this also updates a stale comment which claimed that the widen
increment is only used if it dominates the new use.
The helper should also be used to guard the code added in da437330be,
which I am planning on doing separately once the helper lands.
Fixes https://github.com/llvm/llvm-project/issues/82243.
|
|
We are replacing a narrow IV increment with a wider one. If the original
(narrow) increment did not wrap, the wider one should not wrap either.
Set the flags to be the union of both wide increment and original
increment; this ensures we preserve flags SCEV could infer for the wider
increment.
Fixes https://github.com/llvm/llvm-project/issues/71517.
|
|
IndVars may replace an instruction with one of its operands, if they
have the same SCEV expression. However, such a replacement may be more
poisonous.
First, check whether the operand being poison implies that the
instruction is also poison, in which case the replacement is always
safe. If this fails, check whether SCEV can determine that reusing the
instruction is safe, using the same check as SCEVExpander.
Fixes https://github.com/llvm/llvm-project/issues/79861.
|
|
|
|
When performing sext IV widening, if one of the narrow uses is in
a zext nneg, we can treat it like an sext and avoid the insertion
of a trunc.
|
|
|
|
|
|
The same idea as in 34d380e1f63a7e2cdb9ab1e6498f727fcd710a14, but considering
truncation instructions.
Improvement for #59633.
|
|
The current code structure results in cases where if a) we can't clone
the IV user (because it's not in our whitelist) or b) can't prove the
SCEV expressions are identical, we'd sometimes leave both the original
unwiddened IV and the partially widdened IV in code. Instead, just
truncate thw wide IV to the use - same as what we'd do if we couldn't
find an addrec to start with.
Noticed this while playing with changing how we produce addrecs. The
current structure results in a very tight interlock between SCEVs
internal capabilities and indvars code.
|
|
This reverts commit a6c8e27b3a052913a15a13ee0d4ac466c5ab3f92. It appears likely to have caused https://lab.llvm.org/buildbot/#/builders/57/builds/30988.
|
|
As suggested during review of https://github.com/llvm/llvm-project/pull/70990.
|
|
(#70990)
IndVars has the existing notion of a narrow definition which is known to
positive and thus both sign and zero extension kinds are actually the
same operations. There's existing logic for forming a SCEV based on the
extension kind and the no-wrap flags. This change extends that logic to
form the opposite extension kind for a positive def if doing so is
allowed by the flags. Note that we already do something analogous for
the getWideRecurrence case as well.
|
|
zext nneg was recently added to the IR in #67982. This patch teaches
SimplifyIndVars to prefer zext nneg over *both* sext and plain zext,
when a local SCEV query indicates the source is non-negative.
The choice to prefer zext nneg over sext looks slightly aggressive
here, but probably isn't so much in practice. For cases where we'd
"remember" the range fact, instcombine would convert the sext into
a zext nneg anyways. The only cases where this produces a different
result overall are when SCEV knows a non-local fact, and it doesn't
get materialized into the IR. Those are exactly the cases where
using zext nneg are most useful. We do run the risk of e.g. a
missing combine - since we haven't updated most of them yet - but
that seems like a manageable risk.
Note that there are much deeper algorithmic changes we could make
to this code to exploit zext nneg, but this seemed like a reasonable
and low risk starting point.
|
|
Mostly a cleanup so that we don't need to manually emit instructions,
and can eagerly constant fold where relevant.
|
|
|
|
In SimplifyIndvar::replaceFloatIVWithIntegerIV() the return value of
getFPMantissaWidth() was being cast as an unsigned integer and then
compared with the number of bits needed to represent an integer that was
cast to and from a floating-point type. This is a problem because
getFPMantissaWidth() returns -1 if the type does not have a stable
mantissa.
Currently the only type that returns -1 is ppc_fp128, so you'd need a
pretty big induction variable to cause a problem. However, this problem
will be more likely to be exposed when we implement support for decimal
floating-point types.
Strictly speaking, what we want to know here is the size of the biggest
integer that can be represented exactly. We could get that information
even with an unstable mantissa width, but getFPMantissaWidth() won't do
it.
|
|
This patch adds a two-argument SetInsertPoint method to IRBuilder that
takes a block/iterator instead of an instruction, and updates many call
sites to use it. The motivating reason for doing this is given here [0],
we'd like to pass around more information about the position of debug-info
in the iterator object. That necessitates passing iterators around most of
the time.
[0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939
Differential Revision: https://reviews.llvm.org/D152468
|
|
In degenerate cases, the backedge value can be folded to poison.
Fixes https://github.com/llvm/llvm-project/issues/64891.
|
|
We usually only check direct use instruction of IV, while the
bitcast of 'ptrtoint ptr to i64' doesn't affect the result, so go
a step further.
Fix https://github.com/llvm/llvm-project/issues/59633.
Reviewed By: markoshorro
Differential Revision: https://reviews.llvm.org/D151877
|
|
Make sure the invariant expressions are safe to expand. In
particular, we should not speculative a trapping division into
the preheader.
Fixes https://github.com/llvm/llvm-project/issues/62992.
|
|
Extract complex logic of Binary Operator's flag strengthening to a separate
method in order to reuse it.
Patch by Aleksandr Popov!
Differential Revision: https://reviews.llvm.org/D143562
Reviewed By: mkazantsev
|
|
Reapply with checks for instructions in unreachable blocks. A test
case for this was added in 1ee4a93b15bb.
-----
This is a recurring pattern: We want to find the nearest common
dominator (instruction) for two instructions, but currently only
provide an API for the nearest common dominator of two basic blocks.
Add an overload that accepts and return instructions.
|
|
This reverts commit 7f0de9573f758f5f9108795850337a5acbd17eef.
This is missing handling for !isReachableFromEntry() blocks, which
may be relevant for some callers. Revert for now.
|
|
This is a recurring pattern: We want to find the nearest common
dominator (instruction) for two instructions, but currently only
provide an API for the nearest common dominator of two basic blocks.
Add an overload that accepts and return instructions.
|
|
|
