summaryrefslogtreecommitdiff
path: root/llvm/lib/Transforms/Scalar/LoopDataPrefetch.cpp
AgeCommit message (Collapse)Author
2025-04-23[CostModel] Remove optional from InstructionCost::getValue() (#135596)David Green
InstructionCost is already an optional value, containing an Invalid state that can be checked with isValid(). There is little point in returning another optional from getValue(). Most uses do not make use of it being a std::optional, dereferencing the value directly (either isValid has been checked previously or the Cost is assumed to be valid). The one case that does in AMDGPU used value_or which has been replaced by a isValid() check.
2024-11-02[Scalar] Remove unused includes (NFC) (#114645)Kazu Hirata
Identified with misc-include-cleaner.
2024-10-17[LLVM] Make more use of IRBuilder::CreateIntrinsic. NFC. (#112706)Jay Foad
Convert many instances of: Fn = Intrinsic::getOrInsertDeclaration(...); CreateCall(Fn, ...) to the equivalent CreateIntrinsic call.
2024-10-11[NFC] Rename `Intrinsic::getDeclaration` to `getOrInsertDeclaration` (#111752)Rahul Joshi
Rename the function to reflect its correct behavior and to be consistent with `Module::getOrInsertFunction`. This is also in preparation of adding a new `Intrinsic::getDeclaration` that will have behavior similar to `Module::getFunction` (i.e, just lookup, no creation).
2024-06-27[IR] Add getDataLayout() helpers to BasicBlock and Instruction (#96902)Nikita Popov
This is a helper to avoid writing `getModule()->getDataLayout()`. I regularly try to use this method only to remember it doesn't exist... `getModule()->getDataLayout()` is also a common (the most common?) reason why code has to include the Module.h header.
2023-11-07[NFC] Remove Type::getInt8PtrTy (#71029)Paulo Matos
Replace this with PointerType::getUnqual(). Followup to the opaque pointer transition. Fixes an in-code TODO item.
2022-07-27Allow data prefetch into non-default address spaceStanislav Mekhanoshin
I am playing with the LoopDataPrefetch pass and found out that it bails to work with a pointer in a non-zero address space. This patch adds the target callback to check if an address space is to be considered for prefetching. Default implementation still only allows address space 0, so this is NFCI. This does not currently affect any known targets, but seems to be generally useful for the future. Differential Revision: https://reviews.llvm.org/D129795
2022-07-26[LoopDataPrefetch] Fix crash when TTI doesn't set CacheLineSizezhongyunde
Fix https://github.com/llvm/llvm-project/issues/56681 Reviewed By: RKSimon Differential Revision: https://reviews.llvm.org/D130418
2022-07-14[SCEVExpander] Make CanonicalMode handing in isSafeToExpand() more robust ↵Nikita Popov
(PR50506) isSafeToExpand() for addrecs depends on whether the SCEVExpander will be used in CanonicalMode. At least one caller currently gets this wrong, resulting in PR50506. Fix this by a) making the CanonicalMode argument on the freestanding functions required and b) adding member functions on SCEVExpander that automatically take the SCEVExpander mode into account. We can use the latter variant nearly everywhere, and thus make sure that there is no chance of CanonicalMode mismatch. Fixes https://github.com/llvm/llvm-project/issues/50506. Differential Revision: https://reviews.llvm.org/D129630
2022-06-09Pipe potentially invalid InstructionCost through CodeMetricsPhilip Reames
Per the documentation in Support/InstructionCost.h, the purpose of an invalid cost is so that clients can change behavior on impossible to cost inputs. CodeMetrics was instead asserting that invalid costs never occurred. On a target with an incomplete cost model - e.g. RISCV - this means that transformations would crash on (falsely) invalid constructs - e.g. scalable vectors. While we certainly should improve the cost model - and I plan to do so in the near future - we also shouldn't be crashing. This violates the explicitly stated purpose of an invalid InstructionCost. I updated all of the "easy" consumers where bailouts were locally obvious. I plan to follow up with loop unroll in a following change. Differential Revision: https://reviews.llvm.org/D127131
2022-03-03Cleanup includes: Transform/Scalarserge-sans-paille
Estimated impact on preprocessor output line: before: 1062981579 after: 1062494547 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D120817
2022-02-13[Transforms] Use default member initialization in Prefetch (NFC)Kazu Hirata
2021-12-21[llvm] Use depth_first (NFC)Kazu Hirata
2021-08-26[LoopDataPrefetch] Add missed LoopSimplify dependence for prefetch passAndrew Wei
SCEVExpander::expandCodeFor may expand add recurrences for loop with a preheader, so we should make LoopDataPrefetch dependent on LoopSimplify. This patch will try to fix : https://bugs.llvm.org/show_bug.cgi?id=43784 Reviewed By: Meinersbur Differential Revision: https://reviews.llvm.org/D108448
2021-05-30[NFCI] Move DEBUG_TYPE definition below #includesMindong Chen
When you try to define a new DEBUG_TYPE in a header file, DEBUG_TYPE definition defined around the #includes in files include it could result in redefinition warnings even compile errors. Reviewed By: tejohnson Differential Revision: https://reviews.llvm.org/D102594
2020-09-22[LoopInfo] empty() -> isInnermost(), add isOutermost()Stefanos Baziotis
Differential Revision: https://reviews.llvm.org/D82895
2020-05-20[SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC).Florian Hahn
SCEVExpander modifies the underlying function so it is more suitable in Transforms/Utils, rather than Analysis. This allows using other transform utils in SCEVExpander. This patch was originally committed as b8a3c34eee06, but broke the modules build, as LoopAccessAnalysis was using the Expander. The code-gen part of LAA was moved to lib/Transforms recently, so this patch can be landed again. Reviewers: sanjoy.google, efriedma, reames Reviewed By: sanjoy.google Differential Revision: https://reviews.llvm.org/D71537
2020-05-06Quiet some -Wdocumentation warnings.Benjamin Kramer
2020-04-23[llvm][NFC][CallSite] Remove remaining {Immutable}CallSite usesMircea Trofin
Reviewers: dblaikie, craig.topper Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D78789
2020-04-02[LoopDataPrefetch] Remove unused include that's a layering violationBenjamin Kramer
2020-04-02[LoopDataPrefetch + SystemZ] Let target decide on prefetching for each loop.Jonas Paulsson
This patch adds - New arguments to getMinPrefetchStride() to let the target decide on a per-loop basis if software prefetching should be done even with a stride within the limit of the hw prefetcher. - New TTI hook enableWritePrefetching() to let a target do write prefetching by default (defaults to false). - In LoopDataPrefetch: - A search through the whole loop to gather information before emitting any prefetches. This way the target can get information via new arguments to getMinPrefetchStride() and emit prefetches more selectively. Collected information includes: Does the loop have a call, how many memory accesses, how many of them are strided, how many prefetches will cover them. This is NFC to before as long as the target does not change its definition of getMinPrefetchStride(). - If a previous access to the same exact address was 'read', and the current one is 'write', make it a 'write' prefetch. - If two accesses that are covered by the same prefetch do not dominate each other, put the prefetch in a block that dominates both of them. - If a ConstantMaxTripCount is less than ItersAhead, then skip the loop. - A SystemZ implementation of getMinPrefetchStride(). Review: Ulrich Weigand, Michael Kruse Differential Revision: https://reviews.llvm.org/D70228
2020-01-04Revert "[SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC)."Florian Hahn
This reverts commit 51ef53f3bd23559203fe9af82ff2facbfedc1db3, as it breaks some bots.
2020-01-04[SCEV] Move ScalarEvolutionExpander.cpp to Transforms/Utils (NFC).Florian Hahn
SCEVExpander modifies the underlying function so it is more suitable in Transforms/Utils, rather than Analysis. This allows using other transform utils in SCEVExpander. Reviewers: sanjoy.google, efriedma, reames Reviewed By: sanjoy.google Differential Revision: https://reviews.llvm.org/D71537
2019-11-13Sink all InitializePasses.h includesReid Kleckner
This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
2019-07-25Allow prefetching from non-zero address spacesJF Bastien
Summary: This is useful for targets which have prefetch instructions for non-default address spaces. <rdar://problem/42662136> Subscribers: nemanjai, javed.absar, hiraditya, kbarton, jkorous, dexonsmith, cfe-commits, llvm-commits, RKSimon, hfinkel, t.p.northover, craig.topper, anemet Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D65254 llvm-svn: 367032
2019-02-01[opaque pointer types] Pass function types to CallInst creation.James Y Knight
This cleans up all CallInst creation in LLVM to explicitly pass a function type rather than deriving it from the pointer's element-type. Differential Revision: https://reviews.llvm.org/D57170 llvm-svn: 352909
2019-01-19Update the file headers across all of the LLVM projects in the monorepoChandler Carruth
to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2018-05-14Rename DEBUG macro to LLVM_DEBUG.Nicola Zaghen
The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240
2018-05-01Remove \brief commands from doxygen comments.Adrian Prantl
We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46290 llvm-svn: 331272
2018-01-19[NFC] fix trivial typos in commentsHiroshi Inoue
"the the" -> "the" llvm-svn: 322934
2017-12-13Remove redundant includes from lib/Transforms.Michael Zolotukhin
llvm-svn: 320628
2017-10-11[NFC] Convert OptimizationRemarkEmitter old emit() calls to new closureVivek Pandya
parameterized emit() calls Summary: This is not functional change to adopt new emit() API added in r313691. Reviewed By: anemet Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D38285 llvm-svn: 315476
2017-10-09Rename OptimizationDiagnosticInfo.* to OptimizationRemarkEmitter.*Adam Nemet
Sync it up with the name of the class actually defined here. This has been bothering me for a while... llvm-svn: 315249
2017-08-16[LoopDataPrefetch][AArch64FalkorHWPFFix] Preserve ScalarEvolutionGeoff Berry
Summary: Mark LoopDataPrefetch and AArch64FalkorHWPFFix passes as preserving ScalarEvolution since they do not alter loop structure and should not alter any SCEV values (though LoopDataPrefetch may introduce new instructions that won't have cached SCEV values yet). This can result in slight code differences, mainly w.r.t. nsw/nuw flags on SCEVs, since these are computed somewhat lazily when a zext/sext instruction is encountered. As a result, passes after the modified passes may see SCEVs with more nsw/nuw flags present. Reviewers: sanjoy, anemet Subscribers: aemerson, rengolin, mzolotukhin, javed.absar, kristof.beyls, mcrosier, llvm-commits Differential Revision: https://reviews.llvm.org/D36716 llvm-svn: 311032
2016-12-19Revert @llvm.assume with operator bundles (r289755-r289757)Daniel Jasper
This creates non-linear behavior in the inliner (see more details in r289755's commit thread). llvm-svn: 290086
2016-12-15Remove the AssumptionCacheHal Finkel
After r289755, the AssumptionCache is no longer needed. Variables affected by assumptions are now found by using the new operand-bundle-based scheme. This new scheme is more computationally efficient, and also we need much less code... llvm-svn: 289756
2016-09-30[LoopDataPrefetch] Port to new streaming API for opt remarksAdam Nemet
llvm-svn: 282826
2016-09-08[LoopDataPrefetch] Use range based for loop; NFCIBalaram Makam
Switch to range based for loop. No functional change, but more readable code. llvm-svn: 280966
2016-08-13[PM] Port LoopDataPrefetch to new pass managerTeresa Johnson
Summary: Refactor the existing support into a LoopDataPrefetch implementation class and a LoopDataPrefetchLegacyPass class that invokes it. Add a new LoopDataPrefetchPass for the new pass manager that utilizes the LoopDataPrefetch implementation class. Reviewers: mehdi_amini Subscribers: sanjoy, mzolotukhin, nemanjai, llvm-commits Differential Revision: https://reviews.llvm.org/D23483 llvm-svn: 278591
2016-07-22[LoopDataPrefetch] Fix unused variable in release buildAdam Nemet
llvm-svn: 276491
2016-07-22[LoopDataPrefetch] Include hotness of region in opt remarkAdam Nemet
llvm-svn: 276488
2016-07-22[LoopDataPrefetch] Sort headersAdam Nemet
llvm-svn: 276487
2016-06-26Apply clang-tidy's modernize-loop-convert to most of lib/Transforms.Benjamin Kramer
Only minor manual fixes. No functionality change intended. llvm-svn: 273808
2016-06-10Remove a few gendered pronouns.Nico Weber
llvm-svn: 272422
2016-05-05[LoopDataPrefetch] Add optimization remarkAdam Nemet
With -Rpass=loop-data-prefetch, show the memory access that got prefetched. llvm-svn: 268578
2016-05-03Add opt-bisect support to additional passes that can be skippedAndrew Kaylor
Differential Revision: http://reviews.llvm.org/D19882 llvm-svn: 268457
2016-03-29[LoopDataPrefetch] Centralize the tuning cl::opts under the passAdam Nemet
This is effectively NFC, minus the renaming of the options (-cyclone-prefetch-distance -> -prefetch-distance). The change was requested by Tim in D17943. llvm-svn: 264806
2016-03-29[LoopDataPrefetch] Make more member functions private, NFC.Adam Nemet
llvm-svn: 264798
2016-03-18[LoopDataPrefetch] Add TTI to limit the number of iterations to prefetch aheadAdam Nemet
Summary: It can hurt performance to prefetch ahead too much. Be conservative for now and don't prefetch ahead more than 3 iterations on Cyclone. Reviewers: hfinkel Subscribers: llvm-commits, mzolotukhin Differential Revision: http://reviews.llvm.org/D17949 llvm-svn: 263772
2016-03-18[LoopDataPrefetch/Aarch64] Allow selective prefetching of large-strided accessesAdam Nemet
Summary: And use this TTI for Cyclone. As it was explained in the original RFC (http://thread.gmane.org/gmane.comp.compilers.llvm.devel/92758), the HW prefetcher work up to 2KB strides. I am also adding tests for this and the previous change (D17943): * Cyclone prefetching accesses with a large stride * Cyclone not prefetching accesses with a small stride * Generic Aarch64 subtarget not prefetching either Reviewers: hfinkel Subscribers: aemerson, rengolin, llvm-commits, mzolotukhin Differential Revision: http://reviews.llvm.org/D17945 llvm-svn: 263771