summaryrefslogtreecommitdiff
path: root/llvm/lib/Transforms/Scalar/MergedLoadStoreMotion.cpp
AgeCommit message (Collapse)Author
2025-01-27[NFC][DebugInfo] Switch more call-sites to using iterator-insertion (#124283)Jeremy Morse
To finalise the "RemoveDIs" work removing debug intrinsics, we're updating call sites that insert instructions to use iterators instead. This set of changes are those where it's not immediately obvious that just calling getIterator to fetch an iterator is correct, and one or two places where more than one line needs to change. Overall the same rule holds though: iterators generated for the start of a block such as getFirstNonPHIIt need to be passed into insert/move methods without being unwrapped/rewrapped, everything else can use getIterator.
2024-11-15[MergedLoadStore] Preserve common metadata when sinking stores. (#116382)Florian Hahn
When sinking a store, preserve common metadata present on stores on both sides of the diamond. PR: https://github.com/llvm/llvm-project/pull/116382
2024-06-27[IR] Add getDataLayout() helpers to BasicBlock and Instruction (#96902)Nikita Popov
This is a helper to avoid writing `getModule()->getDataLayout()`. I regularly try to use this method only to remember it doesn't exist... `getModule()->getDataLayout()` is also a common (the most common?) reason why code has to include the Module.h header.
2023-11-20[NewPM] Remove MergedLoadStoreMotionLegacyPass (#72813)Aiden Grossman
This pass isn't used anywhere and thus isn't tested anywhere. Because of these reasons, remove it.
2023-09-11[NFC][RemoveDIs] Prefer iterator-insertion over instructionsJeremy Morse
Continuing the patch series to get rid of debug intrinsics [0], instruction insertion needs to be done with iterators rather than instruction pointers, so that we can communicate information in the iterator class. This patch adds an iterator-taking insertBefore method and converts various call sites to take iterators. These are all sites where such debug-info needs to be preserved so that a stage2 clang can be built identically; it's likely that many more will need to be changed in the future. At this stage, this is just changing the spelling of a few operations, which will eventually become signifiant once the debug-info bearing iterator is used. [0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939 Differential Revision: https://reviews.llvm.org/D152537
2023-04-04[MergedLoadStoreMotion] Merge stores with conflicting value typesJeff Byrnes
Since memory does not have an intrinsic type, we do not need to require value type matching on stores in order to sink them. To facilitate that, this patch finds stores which are sinkable, but have conflicting types, and bitcasts the ValueOperand so they are easily sinkable into a PHINode. Rather than doing fancy analysis to optimally insert the bitcast, we always insert right before the relevant store in the diamond branch. The assumption is that later passes (e.g. GVN, SimplifyCFG) will clean up bitcasts as needed. Differential Revision: https://reviews.llvm.org/D147348
2023-02-22[NFC] Use single quotes for single char output during `printPipline`Liren Peng
Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D144365
2022-12-27Reapply [MergeLoadStoreMotion] Don't require GEP for sinkingNikita Popov
Reapply with a fix for a failing debuginfo assignment tracking test. ----- Allow sinking stores where both operands are the same, don't require them to have an identical GEP in each block. This came up when migrating tests to opaque pointers, where zero-index GEPs are omitted.
2022-12-27Revert "[MergeLoadStoreMotion] Don't require GEP for sinking"Nikita Popov
I missed a test failure in the DebugInfo directory. This reverts commit 2c15b9d9e1a898cfd849db81b36d278eac3ef24e. This reverts commit fb435e1cb5842e1437436e9e7378dfc4106fdad8.
2022-12-27[MergeLoadStoreMotion] Don't require GEP for sinkingNikita Popov
Allow sinking stores where both operands are the same, don't require them to have an identical GEP in each block. This came up when migrating tests to opaque pointers, where zero-index GEPs are omitted.
2022-11-15[Assignment Tracking][16/*] Account for assignment tracking in mldst-motionOCHyams
The Assignment Tracking debug-info feature is outlined in this RFC: https://discourse.llvm.org/t/ rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir mldst-motion will merge and sink the stores in if-diamond branches into the common successor. Attach a merged DIAssignID to the merged store. Reviewed By: jmorse Differential Revision: https://reviews.llvm.org/D133311
2022-08-30[DebugInfo] Fix line number attribution in mldst-motionOCHyams
Taking the example from the test included in this patch: $ cat test.cpp -n 1 void fun(int *a, int cond) { 2 if (cond) 3 a[1] = 1; 4 else 5 a[1] = 2; 6 } mldst-motion will merge and sink the stores in if.then and if.else into if.end. The resultant PHI, gep and store should be attributed line zero with the innermost common scope rather than picking a debug location from one of the original stores. Reviewed By: djtodoro Differential Revision: https://reviews.llvm.org/D132741
2022-03-03Cleanup includes: Transform/Scalarserge-sans-paille
Estimated impact on preprocessor output line: before: 1062981579 after: 1062494547 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D120817
2022-03-01Cleanup includes: TransformsUtilsserge-sans-paille
Estimation on the impact on preprocessor output: before: 1065307662 after: 1064800684 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D120741
2021-09-15[NPM] Added -print-pipeline-passes print params for a few passes.Markus Lavin
Added '-print-pipeline-passes' printing of parameters for those passes declared with *_WITH_PARAMS macro in PassRegistry.def. Note that it only prints the parameters declared inside *_WITH_PARAMS as in a few cases there appear to be additional parameters not parsable. The following passes are now covered (i.e. all of those with *_WITH_PARAMS in PassRegistry.def). LoopExtractorPass - loop-extract HWAddressSanitizerPass - hwsan EarlyCSEPass - early-cse EntryExitInstrumenterPass - ee-instrument LowerMatrixIntrinsicsPass - lower-matrix-intrinsics LoopUnrollPass - loop-unroll AddressSanitizerPass - asan MemorySanitizerPass - msan SimplifyCFGPass - simplifycfg LoopVectorizePass - loop-vectorize MergedLoadStoreMotionPass - mldst-motion GVN - gvn StackLifetimePrinterPass - print<stack-lifetime> SimpleLoopUnswitchPass - simple-loop-unswitch Differential Revision: https://reviews.llvm.org/D109310
2021-05-18[NewPM] Don't mark AA analyses as preservedArthur Eubanks
Currently all AA analyses marked as preserved are stateless, not taking into account their dependent analyses. So there's no need to mark them as preserved, they won't be invalidated unless their analyses are. SCEVAAResults was the one exception to this, it was treated like a typical analysis result. Make it like the others and don't invalidate unless SCEV is invalidated. Reviewed By: asbirlea Differential Revision: https://reviews.llvm.org/D102032
2020-02-02[Transforms] Simplify with make_early_inc_rangeFangrui Song
2019-11-13Sink all InitializePasses.h includesReid Kleckner
This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
2019-09-05[MergedLoadStoreMotion] Sink stores to BB with more than 2 predecessorsDenis Bakhvalov
If we have: bb5: br i1 %arg3, label %bb6, label %bb7 bb6: %tmp = getelementptr inbounds i32, i32* %arg1, i64 2 store i32 3, i32* %tmp, align 4 br label %bb9 bb7: %tmp8 = getelementptr inbounds i32, i32* %arg1, i64 2 store i32 3, i32* %tmp8, align 4 br label %bb9 bb9: ; preds = %bb4, %bb6, %bb7 ... We can't sink stores directly into bb9. This patch creates new BB that is successor of %bb6 and %bb7 and sinks stores into that block. SplitFooterBB is the parameter to the pass that controls that behavior. Change-Id: I7fdf50a772b84633e4b1b860e905bf7e3e29940f Differential: https://reviews.llvm.org/D66234 llvm-svn: 371089
2019-01-19Update the file headers across all of the LLVM projects in the monorepoChandler Carruth
to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2018-11-02[DebugInfo][InstMerge] Fix -debugify for phi node created by -mldst-motionJordan Rupprecht
Summary: -mldst-motion creates a new phi node without any debug info. Use the merged debug location from the incoming stores to fix this. Fixes PR38177. The test case here is (somewhat) simplified from: ``` struct S { int foo; void fn(int bar); }; void S::fn(int bar) { if (bar) foo = 1; else foo = 0; } ``` Reviewers: dblaikie, gbedwell, aprantl, vsk Reviewed By: vsk Subscribers: vsk, JDevlieghere, llvm-commits Tags: #debug-info Differential Revision: https://reviews.llvm.org/D54019 llvm-svn: 346027
2018-05-16[STLExtras] Add size() for ranges, and remove distance()Vedant Kumar
r332057 introduced distance() for ranges. Based on post-commit feedback, this renames distance() to size(). The new size() is also only enabled when the operation is O(1). Differential Revision: https://reviews.llvm.org/D46976 llvm-svn: 332551
2018-05-14Rename DEBUG macro to LLVM_DEBUG.Nicola Zaghen
The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-10[STLExtras] Add distance() for ranges, pred_size(), and succ_size()Vedant Kumar
This commit adds a wrapper for std::distance() which works with ranges. As it would be a common case to write `distance(predecessors(BB))`, this also introduces `pred_size()` and `succ_size()` helpers to make that easier to write. Differential Revision: https://reviews.llvm.org/D46668 llvm-svn: 332057
2018-05-09[MergedLoadStoreMotion] Fix a debug invariant bug in mergeStoresBjorn Pettersson
Summary: MergedLoadStoreMotion::mergeStores is using some heuristics to limit the amount of stores that it tries to sink (see MagicCompileTimeControl in MergedLoadStoreMotion.cpp). The heuristic involves counting the number of instructions in one of the basic blocks that is part of the transformation. We now ignore dbg intrinsics when counting instruction for the MagicCompileTimeControl heuristic. This to make sure that the amount of stores that are sunk doesn't depend on the amount of debug information (if -g is used or not). Reviewers: Gerolf, davide, majnemer Reviewed By: davide Subscribers: dberlin, bjope, aprantl, JDevlieghere, llvm-commits Differential Revision: https://reviews.llvm.org/D46600 llvm-svn: 331852
2018-05-01Remove \brief commands from doxygen comments.Adrian Prantl
We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46290 llvm-svn: 331272
2018-02-23Mark MergedLoadStoreMotion as not preserving MemDep resultsBjorn Steinbrink
Summary: MemDep caches results that signify that a dependence is non-local, and there is currently no way to invalidate such cache entries. Unfortunately, when MLSM sinks a store that can result in a non-local dependence becoming a local one, and then MemDep gives wrong answers. The easiest way out here is to just say that MLSM does indeed not preserve MemDep results. Reviewers: davide, Gerolf Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D43177 llvm-svn: 325880
2017-12-13Remove redundant includes from lib/Transforms.Michael Zolotukhin
llvm-svn: 320628
2017-12-07[ModRefInfo] Make enum ModRefInfo an enum class [NFC].Alina Sbirlea
Summary: Make enum ModRefInfo an enum class. Changes to ModRefInfo values should be done using inline wrappers. This should prevent future bit-wise opearations from being added, which can be more error-prone. Reviewers: sanjoy, dberlin, hfinkel, george.burgess.iv Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40933 llvm-svn: 320107
2017-07-09[IR] Make use of ↵Craig Topper
Type::isPtrOrPtrVectorTy/isIntOrIntVectorTy/isFPOrFPVectorTy to shorten code. NFC llvm-svn: 307491
2017-01-29Remove inclusion of SSAUpdater from several passes.Davide Italiano
It is, in fact, unused. Found while reviewing Danny's new SSAUpdater and porting passes to it to see how the new API looked like. llvm-svn: 293407
2017-01-24Remove the load hoisting code of MLSM, it is completely subsumed by GVNHoistDaniel Berlin
Summary: GVNHoist performs all the optimizations that MLSM does to loads, in a more general way, and in a faster time bound (MLSM is N^3 in most cases, N^4 in a few edge cases). This disables the load portion. Note that the way ld_hoist_st_sink.ll is written makes one think that the loads should be moved to the while.preheader block, but 1. Neither MLSM nor GVNHoist do it (they both move them to identical places). 2. MLSM couldn't possibly do it anyway, as the while.preheader block is not the head of the diamond, while.body is. (GVNHoist could do it if it was legal). 3. At a glance, it's not legal anyway because the in-loop load conflict with the in-loop store, so the loads must stay in-loop. I am happy to update the test to use update_test_checks so that checking is tighter, just was going to do it as a followup. Note that i can find no particular benefit to the store portion on any real testcase/benchmark i have (even size-wise). If we really still want it, i am happy to commit to writing a targeted store sinker, just taking the code from the MemorySSA port of MergedLoadStoreMotion (which is N^2 worst case, and N most of the time). We can do what it does in a much better time bound. We also should be both hoisting and sinking stores, not just sinking them, anyway, since whether we should hoist or sink to merge depends basically on luck of the draw of where the blockers are placed. Nonetheless, i have left it alone for now. Reviewers: chandlerc, davide Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D29079 llvm-svn: 292971
2017-01-15[PM] Introduce an analysis set used to preserve all analyses overChandler Carruth
a function's CFG when that CFG is unchanged. This allows transformation passes to simply claim they preserve the CFG and analysis passes to check for the CFG being preserved to remove the fanout of all analyses being listed in all passes. I've gone through and removed or cleaned up as many of the comments reminding us to do this as I could. Differential Revision: https://reviews.llvm.org/D28627 llvm-svn: 292054
2016-09-07IR: Remove Value::intersectOptionalDataWith, replace all calls with calls to ↵Peter Collingbourne
Instruction::andIRFlags. The two functions are functionally equivalent. Differential Revision: https://reviews.llvm.org/D22830 llvm-svn: 280884
2016-08-09Consistently use FunctionAnalysisManagerSean Silva
Besides a general consistently benefit, the extra layer of indirection allows the mechanical part of https://reviews.llvm.org/D23256 that requires touching every transformation and analysis to be factored out cleanly. Thanks to David for the suggestion. llvm-svn: 278077
2016-07-21[MergedLoadStoreMotion] Remove out of date commentDavid Majnemer
llvm-svn: 276239
2016-07-10Give helper classes/functions internal linkage. NFC.Benjamin Kramer
llvm-svn: 275014
2016-06-28[PM] Normalize FIXMEs for missing PreserveCFG to have the same wording.Michael Kuperstein
llvm-svn: 273974
2016-06-24Switch more loops to be range-basedDavid Majnemer
This makes the code a little more concise, no functional change is intended. llvm-svn: 273644
2016-06-17[PM] Port MergedLoadStoreMotion to the new pass manager, take two.Davide Italiano
This is indeed a much cleaner approach (thanks to Daniel Berlin for pointing out), and also David/Sean for review. Differential Revision: http://reviews.llvm.org/D21454 llvm-svn: 273032
2016-06-16[PM] Revert the port of MergeLoadStoreMotion to the new pass manager.Davide Italiano
Daniel Berlin expressed some real concerns about the port and proposed and alternative approach. I'll revert this for now while working on a new patch, which I hope to put up for review shortly. Sorry for the churn. llvm-svn: 272925
2016-06-14[MergedLoadStoreMotion] Before quering AA verify the loads are the same.Chad Rosier
Basicaa stats show the number of queries in Spec2k6 are reduced by 4540 or ~.67% overall. llvm-svn: 272661
2016-06-14[PM/MergedLoadStoreMotion] Preserve analyses more aggressively.Davide Italiano
llvm-svn: 272611
2016-06-14[PM] Port MergedLoadStoreMotion to the new pass manager.Davide Italiano
llvm-svn: 272606
2016-06-13[PM/MergedLoadStoreMotion] Remove unneeded pass dependency.Davide Italiano
llvm-svn: 272598
2016-06-13[PM/MergeLoadStoreMotion] Convert the logic to static functions.Davide Italiano
Pass AliasAnalyis and MemoryDepResult around. This is in preparation for porting this pass to the new PM. llvm-svn: 272595
2016-06-12[MergedLoadStoreMotion] Use correct helper for load hoist safety.Eli Friedman
It isn't legal to hoist a load past a call which might not return; even if it doesn't throw, it could, for example, call exit(). Fixes http://llvm.org/PR27953. llvm-svn: 272495
2016-05-26[MergedLoadStoreMotion] Don't transform across may-throw callsDavid Majnemer
It is unsafe to hoist a load before a function call which may throw, the throw might prevent a pointer dereference. Likewise, it is unsafe to sink a store after a call which may throw. The caller might be able to observe the difference. This fixes PR27858. llvm-svn: 270828
2016-05-26[MergedLoadStoreMotion] Small cleanupDavid Majnemer
No functional change is intended. llvm-svn: 270824
2016-04-22Re-commit optimization bisect support (r267022) without new pass manager ↵Andrew Kaylor
support. The original commit was reverted because of a buildbot problem with LazyCallGraph::SCC handling (not related to the OptBisect handling). Differential Revision: http://reviews.llvm.org/D19172 llvm-svn: 267231