summaryrefslogtreecommitdiff
path: root/llvm/lib/Transforms/Utils/BreakCriticalEdges.cpp
AgeCommit message (Collapse)Author
2025-05-08[KeyInstr] Remap cloned PHIs in BreakCriticalEdges (#133493)Orlando Cazalet-Hyams
RFC: https://discourse.llvm.org/t/rfc-improving-is-stmt-placement-for-better-interactive-debugging/82668
2025-05-08Propagate DebugLocs on phis in BreakCriticalEdges (#133492)Orlando Cazalet-Hyams
The pull request discusses whether this change is needed or not. We leant towards "it can't hurt" on the basis that it's at worst slightly unecessary (but not incorret). The motivation for the patch came from reviewing code duplication sites to update for Key Instructions, finding this, trying to generate a test case and seeing the DebugLocs aren't propagated.
2025-04-03[Utils] Fix incorrect LCSSA PHI nodes when splitting critical edges with ↵Camsyn
MergeIdenticalEdges (#131744) This PR fixes incorrect LCSSA PHI node generation when splitting critical edges with both `PreserveLCSSA` and `MergeIdenticalEdges` enabled. The bug caused PHI nodes in the split block to miss predecessors when multiple identical edges were merged.
2025-03-28[SPIR-V] Add llvm.loop.unroll metadata lowering (#132062)Dmitry Sidorov
.enable lowers to Unroll LoopControl .disable lowers to DontUnroll LoopControl .count lowers to PartialCount LoopControl .full lowers to Unroll LoopControl TODO in future patches: enable structurizer for non-vulkan targets. --------- Signed-off-by: Sidorov, Dmitry <dmitry.sidorov@intel.com>
2025-03-23[Transforms] Use *Set::insert_range (NFC) (#132652)Kazu Hirata
We can use *Set::insert_range to collapse: for (auto Elem : Range) Set.insert(E); down to: Set.insert_range(Range); In some cases, we can further fold that into the set declaration.
2025-01-24[NFC][DebugInfo] Use iterator-flavour getFirstNonPHI at many call-sites ↵Jeremy Morse
(#123737) As part of the "RemoveDIs" project, BasicBlock::iterator now carries a debug-info bit that's needed when getFirstNonPHI and similar feed into instruction insertion positions. Call-sites where that's necessary were updated a year ago; but to ensure some type safety however, we'd like to have all calls to getFirstNonPHI use the iterator-returning version. This patch changes a bunch of call-sites calling getFirstNonPHI to use getFirstNonPHIIt, which returns an iterator. All these call sites are where it's obviously safe to fetch the iterator then dereference it. A follow-up patch will contain less-obviously-safe changes. We'll eventually deprecate and remove the instruction-pointer getFirstNonPHI, but not before adding concise documentation of what considerations are needed (very few). --------- Co-authored-by: Stephen Tozer <Melamoto@gmail.com>
2024-03-04[RemoveDIs] Reapply 3fda50d3915, insert instructions using iteratorsJeremy Morse
I'd reverted this in 6c7805d5d1 after a bad stage. Original commit messsage follows: [NFC][RemoveDIs] Bulk update utilities to insert with iterators As part of the RemoveDIs project we need LLVM to insert instructions using iterators wherever possible, so that the iterators can carry a bit of debug-info. This commit implements some of that by updating the contents of llvm/lib/Transforms/Utils to always use iterator-versions of instruction constructors. There are two general flavours of update: * Almost all call-sites just call getIterator on an instruction * Several make use of an existing iterator (scenarios where the code is actually significant for debug-info) The underlying logic is that any call to getFirstInsertionPt or similar APIs that identify the start of a block need to have that iterator passed directly to the insertion function, without being converted to a bare Instruction pointer along the way. I've also switched DemotePHIToStack to take an optional iterator: it needs to take an iterator, and having a no-insert-location behaviour appears to be important. The constructors for ICmpInst and FCmpInst have been updated too. They're the only instructions that take block _references_ rather than pointers for certain calls, and a future patch is going to make use of default-null block insertion locations. All of this should be NFC.
2024-02-29Revert "[NFC][RemoveDIs] Bulk update utilities to insert with iterators"Jeremy Morse
This reverts commit 3fda50d3915b2163a54a37b602be7783a89dd808. Apparently I've missed a hunk while staging this; will back out for now. Picked up here: https://lab.llvm.org/buildbot/#/builders/139/builds/60429/steps/6/logs/stdio
2024-02-29[NFC][RemoveDIs] Bulk update utilities to insert with iteratorsJeremy Morse
As part of the RemoveDIs project we need LLVM to insert instructions using iterators wherever possible, so that the iterators can carry a bit of debug-info. This commit implements some of that by updating the contents of llvm/lib/Transforms/Utils to always use iterator-versions of instruction constructors. There are two general flavours of update: * Almost all call-sites just call getIterator on an instruction * Several make use of an existing iterator (scenarios where the code is actually significant for debug-info) The underlying logic is that any call to getFirstInsertionPt or similar APIs that identify the start of a block need to have that iterator passed directly to the insertion function, without being converted to a bare Instruction pointer along the way. I've also switched DemotePHIToStack to take an optional iterator: it needs to take an iterator, and having a no-insert-location behaviour appears to be important. The constructors for ICmpInst and FCmpInst have been updated too. They're the only instructions that take block _references_ rather than pointers for certain calls, and a future patch is going to make use of default-null block insertion locations. All of this should be NFC.
2024-01-27[Transforms] Use a range-based for loop (NFC)Kazu Hirata
2023-10-05Use BlockFrequency type in more places (NFC) (#68266)Matthias Braun
The `BlockFrequency` class abstracts `uint64_t` frequency values. Use it more consistently in various APIs and disable implicit conversion to make usage more consistent and explicit. - Use `BlockFrequency Freq` parameter for `setBlockFreq`, `getProfileCountFromFreq` and `setBlockFreqAndScale` functions. - Return `BlockFrequency` in `getEntryFreq()` functions. - While on it change some `const BlockFrequency& Freq` parameters to plain `BlockFreqency Freq`. - Mark `BlockFrequency(uint64_t)` constructor as explicit. - Add missing `BlockFrequency::operator!=`. - Remove `uint64_t BlockFreqency::getMaxFrequency()`. - Add `BlockFrequency BlockFrequency::max()` function.
2023-09-11[NFC][RemoveDIs] Prefer iterator-insertion over instructionsJeremy Morse
Continuing the patch series to get rid of debug intrinsics [0], instruction insertion needs to be done with iterators rather than instruction pointers, so that we can communicate information in the iterator class. This patch adds an iterator-taking insertBefore method and converts various call sites to take iterators. These are all sites where such debug-info needs to be preserved so that a stage2 clang can be built identically; it's likely that many more will need to be changed in the future. At this stage, this is just changing the spelling of a few operations, which will eventually become signifiant once the debug-info bearing iterator is used. [0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939 Differential Revision: https://reviews.llvm.org/D152537
2022-12-15[NFC] Rename Function::insertBasicBlockAt() to Function::insert().Vasileios Porpodas
I think this is a better name because it is what STL uses. Differential Revision: https://reviews.llvm.org/D140068
2022-12-12[IR][NFC] Adds Function::insertBasicBlockAt() to replace things like ↵Vasileios Porpodas
F->getBasicBlockList().insert() This is part of a series of patches that aim at making Function::getBasicBlockList() private. Differential Revision: https://reviews.llvm.org/D139906
2022-07-08[BasicBlockUtils] Allow critical edge splitting with callbr terminatorsNikita Popov
After D129205, we support SplitBlockPredecessors() for predecessors with callbr terminators. This means that it is now also safe to invoke critical edge splitting for an edge coming from a callbr terminator. Remove checks in various passes that were protecting against that. Differential Revision: https://reviews.llvm.org/D129256
2022-03-01Cleanup includes: TransformsUtilsserge-sans-paille
Estimation on the impact on preprocessor output: before: 1065307662 after: 1064800684 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D120741
2022-02-23PGOInstrumentation, GCOVProfiling: Split indirectbr critical edges ↵Matthias Braun
regardless of PHIs The `SplitIndirectBrCriticalEdges` function was originally designed for `CodeGenPrepare` and skipped splitting of edges when the destination block didn't contain any `PHI` instructions. This only makes sense when reducing COPYs like `CodeGenPrepare`. In the case of `PGOInstrumentation` or `GCOVProfiling` it would result in missed counters and wrong result in functions with computed goto. Differential Revision: https://reviews.llvm.org/D120096
2021-04-06[SplitEdge] Update SplitCriticalEdge to return a nullptr only when the edge ↵Sidharth Baveja
is not critical Summary: The function SplitCriticalEdge (called by SplitEdge) can return a nullptr in cases where the edge is a critical. SplitEdge uses SplitCriticalEdge assuming it can always split all critical edges, which is an incorrect assumption. The three cases where the function SplitCriticalEdge will return a nullptr is: 1. DestBB is an exception block 2. Options.IgnoreUnreachableDests is set to true and isa(DestBB->getFirstNonPHIOrDbgOrLifetime()) is not equal to a nullptr 3. LoopSimplify form must be preserved (Options.PreserveLoopSimplify is true) and it cannot be maintained for a loop due to indirect branches For each of these situations they are handled in the following way: 1. Modified the function ehAwareSplitEdge originally from llvm/lib/Transforms/Coroutines/CoroFrame.cpp to handle the cases when the DestBB is an exception block. This function is called directly in SplitEdge. SplitEdge does not call SplitCriticalEdge in this case 2. Options.IgnoreUnreachableDests is set to false by default, so this situation does not apply. 3. Return a nullptr in this situation since the SplitCriticalEdge also returned nullptr. Nothing we can do in this case. Reviewed By: asbirlea Differential Revision:https://reviews.llvm.org/D94619
2021-01-15BreakCriticalEdges: do not split the critical edge from a CallBr indirect ↵Nick Desaulniers
successor Otherwise we'll fail the assertion in SplitBlockPredecessors() related to splitting the edges from CallBr's. Fixes: https://github.com/ClangBuiltLinux/linux/issues/1161 Fixes: https://github.com/ClangBuiltLinux/linux/issues/1252 Reviewed By: void, MaskRay, jyknight Differential Revision: https://reviews.llvm.org/D88438
2021-01-07[SplitEdge] Add new parameter to SplitEdge to name the newly created basic blockSidharth Baveja
Summary: Currently SplitEdge does not support passing in parameter which allows you to name the newly created BasicBlock. This patch updates the function such that the name of the block can be passed in, if users of this utility decide to do so. Reviewed By: Whitney, bmahjour, asbirlea, jamieschmeiser Differential Revision: https://reviews.llvm.org/D94176
2020-11-18[Transforms] Use llvm::is_contained (NFC)Kazu Hirata
2020-06-17BreakCriticalEdges for callbr indirect destsNick Desaulniers
Summary: llvm::SplitEdge was failing an assertion that the BasicBlock only had one successor (for BasicBlocks terminated by CallBrInst, we typically have multiple successors). It was surprising that the earlier call to SplitCriticalEdge did not handle the critical edge (there was an early return). Removing that triggered another assertion relating to creating a BlockAddress for a BasicBlock that did not (yet) have a parent, which is a simple order of operations issue in llvm::SplitCriticalEdge (a freshly constructed BasicBlock must be inserted into a Function's basic block list to have a parent). Thanks to @nathanchance for the report. Fixes: https://github.com/ClangBuiltLinux/linux/issues/1018 Reviewers: craig.topper, jyknight, void, fhahn, efriedma Reviewed By: efriedma Subscribers: eli.friedman, rnk, efriedma, fhahn, hiraditya, llvm-commits, nathanchance, srhines Tags: #llvm Differential Revision: https://reviews.llvm.org/D81607
2020-06-12[BreakCritEdges] Add option to opt-out of perserving loop-simplify.Florian Hahn
This patch adds a new option to CriticalEdgeSplittingOptions to control whether loop-simplify form must be preserved. It is them used by GVN to indicate that loop-simplify form does not have to be preserved. This fixes a crash exposed by 189efe295b6e. If the critical edge we are splitting goes from a block inside a loop to a block outside the loop, splitting the edge will create a new exit block. As a result, the new block will branch to the original exit block, which will add a non-loop predecessor, breaking loop-simplify form. To preserve loop-simplify form, the predecessor blocks of the original exit are split, but that does not work for blocks with indirectbr terminators. If preserving loop-simplify form is requested, bail out , before making any changes. Reviewers: reames, hfinkel, davide, efriedma Reviewed By: efriedma Differential Revision: https://reviews.llvm.org/D81582
2020-05-21[BrachProbablityInfo] Set edge probabilities at once and fix ↵Yevgeny Rouban
calcMetadataWeights() Hide the method that allows setting probability for particular edge and introduce a public method that sets probabilities for all outgoing edges at once. Setting individual edge probability is error prone. More over it is difficult to check that the total probability is 1.0 because there is no easy way to know when the user finished setting all the probabilities. Related bug is fixed in BranchProbabilityInfo::calcMetadataWeights(). Changing unreachable branch probabilities to raw(1) and distributing the rest (oldProbability - raw(1)) over the reachable branches could introduce total probability inaccuracy bigger than 1/numOfBranches. Reviewers: yamauchi, ebrevnov Tags: #llvm Differential Revision: https://reviews.llvm.org/D79396
2020-05-13Revert "[BrachProbablityInfo] Set edge probabilities at once. NFC."Reid Kleckner
This reverts commit eef95f2746c3347b8dad19091ffb82a88d73acd3. The new assertion about branch propability sums does not hold.
2020-05-13[BrachProbablityInfo] Set edge probabilities at once. NFC.Yevgeny Rouban
Hide the method that allows setting probability for particular edge and introduce a public method that sets probabilities for all outgoing edges at once. Setting individual edge probability is error prone. More over it is difficult to check that the total probability is 1.0 because there is no easy way to know when the user finished setting all the probabilities. Reviewers: yamauchi, ebrevnov Tags: #llvm Differential Revision: https://reviews.llvm.org/D79396
2020-05-07SplitIndirectBrCriticalEdges: Fix Branch Probability updateYevgeny Rouban
Splitting critical edges for indirect branches the SplitIndirectBrCriticalEdges() function may break branch probabilities if target basic block happens to have unset a probability for any of its successors. That is because in such cases the getEdgeProbability(Target) function returns probability 1/NumOfSuccessors and it is called after Target was split (thus Target has a single successor). As the result the correspondent successor of the split block gets probability 100% but 1/NumOfSuccessors is expected (or better be left unset). Reviewers: yamauchi Differential Revision: https://reviews.llvm.org/D78806
2019-11-13Sink all InitializePasses.h includesReid Kleckner
This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
2019-03-12[SanitizerCoverage] Avoid splitting critical edges when destination is a ↵Craig Topper
basic block containing unreachable This patch adds a new option to SplitAllCriticalEdges and uses it to avoid splitting critical edges when the destination basic block ends with unreachable. Otherwise if we split the critical edge, sanitizer coverage will instrument the new block that gets inserted for the split. But since this block itself shouldn't be reachable this is pointless. These basic blocks will stick around and generate assembly, but they don't end in sane control flow and might get placed at the end of the function. This makes it look like one function has code that flows into the next function. This showed up while compiling the linux kernel with clang. The kernel has a tool called objtool that detected the code that appeared to flow from one function to the next. https://github.com/ClangBuiltLinux/linux/issues/351#issuecomment-461698884 Differential Revision: https://reviews.llvm.org/D57982 llvm-svn: 355947
2019-02-22BreakCriticalEdges: Update PostDominatorTreeMatt Arsenault
llvm-svn: 354673
2019-02-12[NFC] Rename DontDeleteUselessPHIs --> KeepOneInputPHIsMax Kazantsev
llvm-svn: 353801
2019-02-08Implementation of asm-goto support in LLVMCraig Topper
This patch accompanies the RFC posted here: http://lists.llvm.org/pipermail/llvm-dev/2018-October/127239.html This patch adds a new CallBr IR instruction to support asm-goto inline assembly like gcc as used by the linux kernel. This instruction is both a call instruction and a terminator instruction with multiple successors. Only inline assembly usage is supported today. This also adds a new INLINEASM_BR opcode to SelectionDAG and MachineIR to represent an INLINEASM block that is also considered a terminator instruction. There will likely be more bug fixes and optimizations to follow this, but we felt it had reached a point where we would like to switch to an incremental development model. Patch by Craig Topper, Alexander Ivchenko, Mikhail Dvoretckii Differential Revision: https://reviews.llvm.org/D53765 llvm-svn: 353563
2019-01-19Update the file headers across all of the LLVM projects in the monorepoChandler Carruth
to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2018-10-15[TI removal] Make variables declared as `TerminatorInst` and initializedChandler Carruth
by `getTerminator()` calls instead be declared as `Instruction`. This is the biggest remaining chunk of the usage of `getTerminator()` that insists on the narrow type and so is an easy batch of updates. Several files saw more extensive updates where this would cascade to requiring API updates within the file to use `Instruction` instead of `TerminatorInst`. All of these were trivial in nature (pervasively using `Instruction` instead just worked). llvm-svn: 344502
2018-10-15[TI removal] Remove TerminatorInst as an input parameter from all publicChandler Carruth
LLVM APIs. There weren't very many. We still have the instruction visitor, and APIs with TerminatorInst as a return type or an output parameter. llvm-svn: 344494
2018-09-07[MemorySSA] Update MemoryPhi wiring for block splitting to consider if ↵Alina Sbirlea
identical edges were merged. Summary: Block splitting is done with either identical edges being merged, or not. Only critical edges can be split without merging identical edges based on an option. Teach the memoryssa updater to take this into account: for the same edge between two blocks only move one entry from the Phi in Old to the new Phi in New. Reviewers: george.burgess.iv Subscribers: sanjoy, jlebar, Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D51563 llvm-svn: 341709
2018-08-21Update MemorySSA in BasicBlockUtils.Alina Sbirlea
Summary: Extend BasicBlocksUtils to update MemorySSA. Subscribers: sanjoy, arsenm, nhaehnle, jlebar, Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D45300 llvm-svn: 340365
2018-03-28Transforms: Introduce Transforms/Utils.h rather than spreading the ↵David Blaikie
declarations amongst Scalar.h and IPO.h Fixes layering - Transforms/Utils shouldn't depend on including a Scalar or IPO header, because Scalar and IPO depend on Utils. llvm-svn: 328717
2017-12-30Use phi ranges to simplify code. No functionality change intended.Benjamin Kramer
llvm-svn: 321585
2017-12-13Remove redundant includes from lib/Transforms.Michael Zolotukhin
llvm-svn: 320628
2017-12-12Split IndirectBr critical edges before PGO gen/use passes.Hiroshi Yamauchi
Summary: The PGO gen/use passes currently fail with an assert failure if there's a critical edge whose source is an IndirectBr instruction and that edge needs to be instrumented. To avoid this in certain cases, split IndirectBr critical edges in the PGO gen/use passes. This works for blocks with single indirectbr predecessors, but not for those with multiple indirectbr predecessors (splitting an IndirectBr critical edge isn't always possible.) Reviewers: davidxl, xur Reviewed By: davidxl Subscribers: efriedma, llvm-commits, mehdi_amini Differential Revision: https://reviews.llvm.org/D40699 llvm-svn: 320511
2017-12-04Move splitIndirectCriticalEdges() to BasicBlockUtils.h.Hiroshi Yamauchi
Summary: Move splitIndirectCriticalEdges() from CodeGenPrepare to BasicBlockUtils.h so that it can be called from other places. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40750 llvm-svn: 319689
2017-08-17[Dominators] Teach LoopUnswitch to use the incremental APIJakub Kuderski
Summary: This patch makes LoopUnswitch use new incremental API for updating dominators. It also updates SplitCriticalEdge, as it is called in LoopUnswitch. There doesn't seem to be any noticeable performance difference when bootstrapping clang with this patch. Reviewers: dberlin, davide, sanjoy, grosser, chandlerc Reviewed By: davide, grosser Subscribers: mzolotukhin, llvm-commits Differential Revision: https://reviews.llvm.org/D35528 llvm-svn: 311093
2016-07-22[PM] Port BreakCriticalEdges to the new PM.Wei Mi
Differential Revision: https://reviews.llvm.org/D22688 llvm-svn: 276449
2016-04-23improve documentation comments; NFCSanjay Patel
llvm-svn: 267292
2015-10-13TransformUtils: Remove implicit ilist iterator conversions, NFCDuncan P. N. Exon Smith
Continuing the work from last week to remove implicit ilist iterator conversions. First related commit was probably r249767, with some more motivation in r249925. This edition gets LLVMTransformUtils compiling without the implicit conversions. No functional change intended. llvm-svn: 250142
2015-08-04Drive-by fixes for LandingPad -> EHPadDavid Majnemer
This change was done as an audit and is by inspection. The new EH system is still very much a work in progress. NFC for the landingpad case. llvm-svn: 243965
2015-07-22[PM/AA] Remove all of the dead AliasAnalysis pointers being threadedChandler Carruth
through APIs that are no longer necessary now that the update API has been removed. This will make changes to the AA interfaces significantly less disruptive (I hope). Either way, it seems like a really nice cleanup. llvm-svn: 242882
2015-06-23Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)Alexander Kornienko
Apparently, the style needs to be agreed upon first. llvm-svn: 240390
2015-06-19Fixed/added namespace ending comments using clang-tidy. NFCAlexander Kornienko
The patch is generated using this command: tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \ -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \ llvm/lib/ Thanks to Eugene Kosov for the original patch! llvm-svn: 240137