summaryrefslogtreecommitdiff
path: root/mlir/lib/Dialect/GPU/Transforms/MemoryPromotion.cpp
AgeCommit message (Collapse)Author
2025-07-24[mlir][NFC] Use `hasOneBlock` instead of `llvm::hasSingleElement(region)` ↵Longsheng Mou
(#149809)
2025-07-23[mlir] Remove unused includes (NFC) (#150266)Kazu Hirata
These are identified by misc-include-cleaner. I've filtered out those that break builds. Also, I'm staying away from llvm-config.h, config.h, and Compiler.h, which likely cause platform- or compiler-specific build failures.
2025-07-21[mlir][NFC] update `mlir/Dialect` create APIs (16/n) (#149922)Maksim Levental
See https://github.com/llvm/llvm-project/pull/147168 for more info.
2023-06-30[mlir][NFC] Clean up builder usage around constants/non-foldable opsMatthias Springer
* Use `create` instead of `createOrFold` for constant ops. Constants cannot be folded any further. * Use `create` instead of `createOrFold` for ops that do not have a folder. * Use C++ op builders that take an `int` instead of creating a `ConstantIndexOp`. * Create `tensor::DimOp` instead of `linalg::createOrFoldDimOp` when it is certain that the operand is a tensor. Differential Revision: https://reviews.llvm.org/D154196
2023-05-12[mlir] Move casting calls from methods to function callsTres Popp
The MLIR classes Type/Attribute/Operation/Op/Value support cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast functionality in addition to defining methods with the same name. This change begins the migration of uses of the method to the corresponding function call as has been decided as more consistent. Note that there still exist classes that only define methods directly, such as AffineExpr, and this does not include work currently to support a functional cast/isa call. Caveats include: - This clang-tidy script probably has more problems. - This only touches C++ code, so nothing that is being generated. Context: - https://mlir.llvm.org/deprecation/ at "Use the free function variants for dyn_cast/cast/isa/…" - Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443 Implementation: This first patch was created with the following steps. The intention is to only do automated changes at first, so I waste less time if it's reverted, and so the first mass change is more clear as an example to other teams that will need to follow similar steps. Steps are described per line, as comments are removed by git: 0. Retrieve the change from the following to build clang-tidy with an additional check: https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check 1. Build clang-tidy 2. Run clang-tidy over your entire codebase while disabling all checks and enabling the one relevant one. Run on all header files also. 3. Delete .inc files that were also modified, so the next build rebuilds them to a pure state. 4. Some changes have been deleted for the following reasons: - Some files had a variable also named cast - Some files had not included a header file that defines the cast functions - Some files are definitions of the classes that have the casting methods, so the code still refers to the method instead of the function without adding a prefix or removing the method declaration at the same time. ``` ninja -C $BUILD_DIR clang-tidy run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\ -header-filter=mlir/ mlir/* -fix rm -rf $BUILD_DIR/tools/mlir/**/*.inc git restore mlir/lib/IR mlir/lib/Dialect/DLTI/DLTI.cpp\ mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp\ mlir/lib/**/IR/\ mlir/lib/Dialect/SparseTensor/Transforms/SparseVectorization.cpp\ mlir/lib/Dialect/Vector/Transforms/LowerVectorMultiReduction.cpp\ mlir/test/lib/Dialect/Test/TestTypes.cpp\ mlir/test/lib/Dialect/Transform/TestTransformDialectExtension.cpp\ mlir/test/lib/Dialect/Test/TestAttributes.cpp\ mlir/unittests/TableGen/EnumsGenTest.cpp\ mlir/test/python/lib/PythonTestCAPI.cpp\ mlir/include/mlir/IR/ ``` Differential Revision: https://reviews.llvm.org/D150123
2023-04-20[mlir][Affine][NFC] Wrap dialect in "affine" namespaceMatthias Springer
This cleanup aligns the affine dialect with all the other dialects. Differential Revision: https://reviews.llvm.org/D148687
2023-01-13[mlir][gpu] Migrate hard-coded address space integers to an enum attribute ↵Christopher Bate
(gpu::AddressSpaceAttr) This is a purely mechanical change that introduces an enum attribute in the GPU dialect to represent the various memref memory spaces as opposed to the hard-coded integer attributes that are currently used. The following steps were taken to make the transition across the codebase: 1. Introduce a pass "gpu-lower-memory-space-attributes": The pass updates all memref types that have a memory space attribute that is a `gpu::AddressSpaceAttr`. These attributes are changed to `IntegerAttr`'s using a mapping that is given by the caller. This pass is based on the "map-memref-spirv-storage-class" pass and the common functions can probably be refactored into a set of utilities under the MemRef dialect. 2. Update the verifiers of GPU/NVGPU dialect operations. If a verifier currently checks the address space of an operand using e.g.`getWorkspaceAddressSpace`, then it can continue to do so. However, the checks are changed to only fail if the memory space is either missing or a wrong value of type `gpu::AddressSpaceAttr`. Otherwise, it just assumes the address space is correct because it was specifically lowered to something other than a `gpu::AddressSpaceAttr`. 3. Update existing gpu-to-llvm conversion infrastructure. In the existing gpu-to-X passes, we add a full conversion equivalent to `gpu-lower-memory-space-attributes` just before doing the conversion to the LLVMDialect. This is done because currently both the gpu-to-llvm passes (rocdl,nvvm) run gpu-to-gpu rewrites within the pass, which introduce `AddressSpaceAttr` memory space annotations. Therefore, I inserted the memory space conversion between the gpu-to-gpu rewrites and the LLVM conversion. For more context see the below discourse discussion: https://discourse.llvm.org/t/gpu-workgroup-shared-memory-address-space-is-hard-coded/ Reviewed By: ftynse Differential Revision: https://reviews.llvm.org/D140644
2023-01-10Move from llvm::makeArrayRef to ArrayRef deduction guides - last partserge-sans-paille
This is a follow-up to https://reviews.llvm.org/D140896, split into several parts as it touches a lot of files. Differential Revision: https://reviews.llvm.org/D141298
2022-09-29[mlir][arith] Change dialect name from Arithmetic to ArithJakub Kuderski
Suggested by @lattner in https://discourse.llvm.org/t/rfc-define-precise-arith-semantics/65507/22. Tested with: `ninja check-mlir check-mlir-integration check-mlir-mlir-spirv-cpu-runner check-mlir-mlir-vulkan-runner check-mlir-examples` and `bazel build --config=generic_clang @llvm-project//mlir:all`. Reviewed By: lattner, Mogball, rriddle, jpienaar, mehdi_amini Differential Revision: https://reviews.llvm.org/D134762
2022-06-20[mlir] move SCF headers to SCF/{IR,Transforms} respectivelyAlex Zinenko
This aligns the SCF dialect file layout with the majority of the dialects. Reviewed By: jpienaar Differential Revision: https://reviews.llvm.org/D128049
2022-06-09[mlir][gpu] Move GPU headers into IR/ and Transforms/Mogball
Depends on D127350 Reviewed By: rriddle Differential Revision: https://reviews.llvm.org/D127352
2022-01-24[mlir] Remove a bunch of unnecessary dialect dependenciesRiver Riddle
A lot of dialects have dependencies that are unnecessary, either because of copy/paste of files when creating things or some other means. This commit cleans up a bunch of the simple ones: * Copy/Paste or missed during refactoring Most of the dependencies cleaned up here look like copy/paste errors when creating new dialects/transformations, or because the dependency wasn't removed during a refactoring (e.g. when splitting the standard dialect). * Unnecessary hard coding of constant operations in matchers There are a few instances where a dialect had a dependency because it was hardcoding checks for constant operations instead of using the better m_Constant approach. Differential Revision: https://reviews.llvm.org/D118062
2022-01-24[mlir:Transforms] Move out the remaining non-dialect independent transforms ↵River Riddle
and utilities This has been a major TODO for a very long time, and is necessary for establishing a proper dialect-free dependency layering for the Transforms library. Code was moved to effectively two main locations: * Affine/ There was quite a bit of affine dialect related code in Transforms/ do to historical reasons (of a time way into MLIR's past). The following headers were moved to: Transforms/LoopFusionUtils.h -> Dialect/Affine/LoopFusionUtils.h Transforms/LoopUtils.h -> Dialect/Affine/LoopUtils.h Transforms/Utils.h -> Dialect/Affine/Utils.h The following transforms were also moved: AffineLoopFusion, AffinePipelineDataTransfer, LoopCoalescing * SCF/ Only one SCF pass was in Transforms/ (likely accidentally placed here): ParallelLoopCollapsing The SCF specific utilities in LoopUtils have been moved to SCF/Utils.h * Misc: mlir::moveLoopInvariantCode was also moved to LoopLikeInterface.h given that it is a simple utility defined in terms of LoopLikeOpInterface. Differential Revision: https://reviews.llvm.org/D117848
2022-01-19[mlir] Make locations required when adding/creating block argumentsRiver Riddle
BlockArguments gained the ability to have locations attached a while ago, but they have always been optional. This goes against the core tenant of MLIR where location information is a requirement, so this commit updates the API to require locations. Fixes #53279 Differential Revision: https://reviews.llvm.org/D117633
2022-01-18[mlir] Replace StrEnumAttr -> EnumAttr in core dialectsMogball
Removes uses of `StrEnumAttr` in core dialects Reviewed By: mehdi_amini, rriddle Differential Revision: https://reviews.llvm.org/D117514
2022-01-02Apply clang-tidy fixes for performance-for-range-copy to MLIR (NFC)Mehdi Amini
2021-10-13[MLIR] Replace std ops with arith dialect opsMogball
Precursor: https://reviews.llvm.org/D110200 Removed redundant ops from the standard dialect that were moved to the `arith` or `math` dialects. Renamed all instances of operations in the codebase and in tests. Reviewed By: rriddle, jpienaar Differential Revision: https://reviews.llvm.org/D110797
2021-05-21[mlir][Standard] NFC - Drop remaining EDSC usageNicolas Vasilache
Drop the remaining EDSC subdirectories and update all uses. Differential Revision: https://reviews.llvm.org/D102911
2021-05-20[mlir][MemRef] NFC - Drop MemRef EDSC usageNicolas Vasilache
Drop the MemRef dialect EDSC subdirectory and update all uses. Differential Revision: https://reviews.llvm.org/D102868
2021-05-19[mlir][SCF] NFC - Drop SCF EDSC usageNicolas Vasilache
Drop the SCF dialect EDSC subdirectory and update all uses. Differential Revision: https://reviews.llvm.org/D102780
2021-03-15[MLIR] Create memref dialect and move dialect-specific ops from std.Julian Gross
Create the memref dialect and move dialect-specific ops from std dialect to this dialect. Moved ops: AllocOp -> MemRef_AllocOp AllocaOp -> MemRef_AllocaOp AssumeAlignmentOp -> MemRef_AssumeAlignmentOp DeallocOp -> MemRef_DeallocOp DimOp -> MemRef_DimOp MemRefCastOp -> MemRef_CastOp MemRefReinterpretCastOp -> MemRef_ReinterpretCastOp GetGlobalMemRefOp -> MemRef_GetGlobalOp GlobalMemRefOp -> MemRef_GlobalOp LoadOp -> MemRef_LoadOp PrefetchOp -> MemRef_PrefetchOp ReshapeOp -> MemRef_ReshapeOp StoreOp -> MemRef_StoreOp SubViewOp -> MemRef_SubViewOp TransposeOp -> MemRef_TransposeOp TensorLoadOp -> MemRef_TensorLoadOp TensorStoreOp -> MemRef_TensorStoreOp TensorToMemRefOp -> MemRef_BufferCastOp ViewOp -> MemRef_ViewOp The roadmap to split the memref dialect from std is discussed here: https://llvm.discourse.group/t/rfc-split-the-memref-dialect-from-std/2667 Differential Revision: https://reviews.llvm.org/D98041
2020-06-02[mlir] post-commit review fixesAlex Zinenko
This fixes several post-commit nits from D79688 and D80135, namely typos, debug output and control flow inversion.
2020-05-20[mlir][gpu] Refactor functions for workgroup and private buffer attributions.Wen-Heng (Jack) Chung
Summary: Consolidate interfaces adding workgroup and private buffer attributions in GPU dialect. Note all private buffer attributions must follow workgroup buffer attributions. Reviewers: herhut Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, csigg, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, llvm-commits Tags: #llvm, #mlir Differential Revision: https://reviews.llvm.org/D79508
2020-05-19[mlir] scf::ForOp: provide builders with callbacks for loop bodyAlex Zinenko
Thanks to a recent change that made `::build` functions take an instance of `OpBuilder`, it is now possible to build operations within a region attached to the operation about to be created. Exercise this on `scf::ForOp` by taking a callback that populates the loop body while the loop is being created. Additionally, provide helper functions to build perfect nests of `ForOp`s, with support for iteration arguments. These functions provide the same functionality as EDSC LoopNestBuilder with simpler implementation, without relying on edsc::ScopedContext, and using `OpBuilder` in an unambiguous way. Compatibility functions for EDSC are provided, but may be removed in the future. Differential Revision: https://reviews.llvm.org/D79688
2020-05-11[mlir] NFC: Rename LoopOps dialect to SCF (Structured Control Flow)Alex Zinenko
This dialect contains various structured control flow operaitons, not only loops, reflect this in the name. Drop the Ops suffix for consistency with other dialects. Note that this only moves the files and changes the C++ namespace from 'loop' to 'scf'. The visible IR prefix remains the same and will be updated separately. The conversions will also be updated separately. Differential Revision: https://reviews.llvm.org/D79578
2020-04-23[mlir][EDSC] Retire ValueHandleNicolas Vasilache
The EDSC discussion [thread](https://llvm.discourse.group/t/evolving-builder-apis-based-on-lessons-learned-from-edsc/879) points out that ValueHandle has become an unnecessary level of abstraction since MLIR switch from `Value *` to `Value` everywhere. This revision removes this level of indirection.
2020-04-14[llvm][STLExtras] Move various iterator/range utilities from MLIR to LLVMRiver Riddle
This revision moves the various range utilities present in MLIR to LLVM to enable greater reuse. This revision moves the following utilities: * indexed_accessor_* This is set of utility iterator/range base classes that allow for building a range class where the iterators are represented by an object+index pair. * make_second_range Given a range of pairs, returns a range iterating over the `second` elements. * hasSingleElement Returns if the given range has 1 element. size() == 1 checks end up being very common, but size() is not always O(1) (e.g., ilist). This method provides O(1) checks for those cases. Differential Revision: https://reviews.llvm.org/D78064
2020-04-13[mlir][NFC] Replace mlir/Support/Functional.h with llvm equivalents.River Riddle
Summary: Functional.h contains many different methods that have a direct, and more efficient, equivalent in LLVM. This revision replaces all usages with the LLVM equivalent, and removes the header. This is part of larger cleanup, pr45513, merging MLIR support facilities into LLVM. Differential Revision: https://reviews.llvm.org/D78053
2020-02-10[mlir][EDSC] Almost NFC - Refactor and untangle EDSC dependenciesNicolas Vasilache
This CL refactors EDSCs to layer them better and break unnecessary dependencies. After this refactoring, the top-level EDSC target only depends on IR but not on Dialects anymore and each dialect has its own EDSC directory. This simplifies the layering and breaks cyclic dependencies. In particular, the declarative builder + folder are made explicit and are now confined to Linalg. As the refactoring occurred, certain classes and abstractions that were not paying for themselves have been removed. Differential Revision: https://reviews.llvm.org/D74302
2020-01-16[mlir] NFC: Fix trivial typosKazuaki Ishizaki
Summary: Fix trivial typos Differential Revision: https://reviews.llvm.org/D72672
2020-01-09[mlir][GPU] introduce utilities for promotion to workgroup memoryAlex Zinenko
Introduce a set of function that promote a memref argument of a `gpu.func` to workgroup memory using memory attribution. The promotion boils down to additional loops performing the copy from the original argument to the attributed memory in the beginning of the function, and back at the end of the function using all available threads. The loop bounds are specified so as to adapt to any size of the workgroup. These utilities are intended to compose with other existing utilities (loop coalescing and tiling) in cases where the distribution of work across threads is uneven, e.g. copying a 2D memref with only the threads along the "x" dimension. Similarly, specialization of the kernel to specific launch sizes should be implemented as a separate pass combining constant propagation and canonicalization. Introduce a simple attribute-driven pass to test the promotion transformation since we don't have a heuristic at the moment. Differential revision: https://reviews.llvm.org/D71904