summaryrefslogtreecommitdiff
path: root/mlir/lib/Dialect/GPU/Transforms/AsyncRegionRewriter.cpp
AgeCommit message (Collapse)Author
2025-11-13[MLIR] Apply clang-tidy fixes for misc-use-internal-linkage in ↵Mehdi Amini
AsyncRegionRewriter.cpp (NFC)
2025-07-21[mlir][NFC] update `mlir/Dialect` create APIs (16/n) (#149922)Maksim Levental
See https://github.com/llvm/llvm-project/pull/147168 for more info.
2025-07-07[mlir] Remove unused includes (NFC) (#147455)Kazu Hirata
These are identified by misc-include-cleaner. I've filtered out those that break builds. Also, I'm staying away from llvm-config.h, config.h, and Compiler.h, which likely cause platform- or compiler-specific build failures.
2025-03-06 [MLIR][NFC] Retire let constructor for GPU (#129849)lorenzo chelini
`let constructor` is legacy (do not use in tree!) since the table gen backend emits most of the glue logic to build a pass.
2024-12-13[MLIR] Create GPU utils library & move distribution utils (#119264)Petr Kurapov
Continue the move of `warp_execute_on_lane_0` op to the gpu dialect (#116994). This patch creates a utils library in GPU and moves generic helper functions there.
2024-04-01[mlir][NFC] Simplify type checks with isa predicates (#87183)Jakub Kuderski
For more context on isa predicates, see: https://github.com/llvm/llvm-project/pull/83753.
2023-05-15Cleanup uses of getAttrDictionary() in MLIR to use ↵Mehdi Amini
getDiscardableAttrDictionary() when possible This also speeds up some benchmarks in compiling simple fortan file by 2x! Fixes #62687 Differential Revision: https://reviews.llvm.org/D150540
2023-05-12[mlir] Move casting calls from methods to function callsTres Popp
The MLIR classes Type/Attribute/Operation/Op/Value support cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast functionality in addition to defining methods with the same name. This change begins the migration of uses of the method to the corresponding function call as has been decided as more consistent. Note that there still exist classes that only define methods directly, such as AffineExpr, and this does not include work currently to support a functional cast/isa call. Caveats include: - This clang-tidy script probably has more problems. - This only touches C++ code, so nothing that is being generated. Context: - https://mlir.llvm.org/deprecation/ at "Use the free function variants for dyn_cast/cast/isa/…" - Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443 Implementation: This first patch was created with the following steps. The intention is to only do automated changes at first, so I waste less time if it's reverted, and so the first mass change is more clear as an example to other teams that will need to follow similar steps. Steps are described per line, as comments are removed by git: 0. Retrieve the change from the following to build clang-tidy with an additional check: https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check 1. Build clang-tidy 2. Run clang-tidy over your entire codebase while disabling all checks and enabling the one relevant one. Run on all header files also. 3. Delete .inc files that were also modified, so the next build rebuilds them to a pure state. 4. Some changes have been deleted for the following reasons: - Some files had a variable also named cast - Some files had not included a header file that defines the cast functions - Some files are definitions of the classes that have the casting methods, so the code still refers to the method instead of the function without adding a prefix or removing the method declaration at the same time. ``` ninja -C $BUILD_DIR clang-tidy run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\ -header-filter=mlir/ mlir/* -fix rm -rf $BUILD_DIR/tools/mlir/**/*.inc git restore mlir/lib/IR mlir/lib/Dialect/DLTI/DLTI.cpp\ mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp\ mlir/lib/**/IR/\ mlir/lib/Dialect/SparseTensor/Transforms/SparseVectorization.cpp\ mlir/lib/Dialect/Vector/Transforms/LowerVectorMultiReduction.cpp\ mlir/test/lib/Dialect/Test/TestTypes.cpp\ mlir/test/lib/Dialect/Transform/TestTransformDialectExtension.cpp\ mlir/test/lib/Dialect/Test/TestAttributes.cpp\ mlir/unittests/TableGen/EnumsGenTest.cpp\ mlir/test/python/lib/PythonTestCAPI.cpp\ mlir/include/mlir/IR/ ``` Differential Revision: https://reviews.llvm.org/D150123
2023-05-01Introduce MLIR Op PropertiesMehdi Amini
This new features enabled to dedicate custom storage inline within operations. This storage can be used as an alternative to attributes to store data that is specific to an operation. Attribute can also be stored inside the properties storage if desired, but any kind of data can be present as well. This offers a way to store and mutate data without uniquing in the Context like Attribute. See the OpPropertiesTest.cpp for an example where a struct with a std::vector<> is attached to an operation and mutated in-place: struct TestProperties { int a = -1; float b = -1.; std::vector<int64_t> array = {-33}; }; More complex scheme (including reference-counting) are also possible. The only constraint to enable storing a C++ object as "properties" on an operation is to implement three functions: - convert from the candidate object to an Attribute - convert from the Attribute to the candidate object - hash the object Optional the parsing and printing can also be customized with 2 extra functions. A new options is introduced to ODS to allow dialects to specify: let usePropertiesForAttributes = 1; When set to true, the inherent attributes for all the ops in this dialect will be using properties instead of being stored alongside discardable attributes. The TestDialect showcases this feature. Another change is that we introduce new APIs on the Operation class to access separately the inherent attributes from the discardable ones. We envision deprecating and removing the `getAttr()`, `getAttrsDictionary()`, and other similar method which don't make the distinction explicit, leading to an entirely separate namespace for discardable attributes. Recommit d572cd1b067f after fixing python bindings build. Differential Revision: https://reviews.llvm.org/D141742
2023-05-01Revert "Introduce MLIR Op Properties"Mehdi Amini
This reverts commit d572cd1b067f1177a981a4711bf2e501eaa8117b. Some bots are broken and investigation is needed before relanding.
2023-05-01Introduce MLIR Op PropertiesMehdi Amini
This new features enabled to dedicate custom storage inline within operations. This storage can be used as an alternative to attributes to store data that is specific to an operation. Attribute can also be stored inside the properties storage if desired, but any kind of data can be present as well. This offers a way to store and mutate data without uniquing in the Context like Attribute. See the OpPropertiesTest.cpp for an example where a struct with a std::vector<> is attached to an operation and mutated in-place: struct TestProperties { int a = -1; float b = -1.; std::vector<int64_t> array = {-33}; }; More complex scheme (including reference-counting) are also possible. The only constraint to enable storing a C++ object as "properties" on an operation is to implement three functions: - convert from the candidate object to an Attribute - convert from the Attribute to the candidate object - hash the object Optional the parsing and printing can also be customized with 2 extra functions. A new options is introduced to ODS to allow dialects to specify: let usePropertiesForAttributes = 1; When set to true, the inherent attributes for all the ops in this dialect will be using properties instead of being stored alongside discardable attributes. The TestDialect showcases this feature. Another change is that we introduce new APIs on the Operation class to access separately the inherent attributes from the discardable ones. We envision deprecating and removing the `getAttr()`, `getAttrsDictionary()`, and other similar method which don't make the distinction explicit, leading to an entirely separate namespace for discardable attributes. Differential Revision: https://reviews.llvm.org/D141742
2023-01-12[mlir] Add operations to BlockAndValueMapping and rename it to IRMappingJeff Niu
The patch adds operations to `BlockAndValueMapping` and renames it to `IRMapping`. When operations are cloned, old operations are mapped to the cloned operations. This allows mapping from an operation to a cloned operation. Example: ``` Operation *opWithRegion = ... Operation *opInsideRegion = &opWithRegion->front().front(); IRMapping map Operation *newOpWithRegion = opWithRegion->clone(map); Operation *newOpInsideRegion = map.lookupOrNull(opInsideRegion); ``` Migration instructions: All includes to `mlir/IR/BlockAndValueMapping.h` should be replaced with `mlir/IR/IRMapping.h`. All uses of `BlockAndValueMapping` need to be renamed to `IRMapping`. Reviewed By: rriddle, mehdi_amini Differential Revision: https://reviews.llvm.org/D139665
2022-11-15[mlir] Remove `Transforms/SideEffectUtils.h` and move the methods into ↵Mahesh Ravishankar
`Interface/SideEffectInterfaces.h`. The methods in `SideEffectUtils.h` (and their implementations in `SideEffectUtils.cpp`) seem to have similar intent to methods already existing in `SideEffectInterfaces.h`. Move the decleration (and implementation) from `SideEffectUtils.h` (and `SideEffectUtils.cpp`) into `SideEffectInterfaces.h` (and `SideEffectInterface.cpp`). Also drop the `SideEffectInterface::hasNoEffect` method in favor of `mlir::isMemoryEffectFree` which actually recurses into the operation instead of just relying on the `hasRecursiveMemoryEffectTrait` exclusively. Differential Revision: https://reviews.llvm.org/D137857
2022-09-30[mlir:GPU][NFC] Update GPU API to use prefixed accessorsRiver Riddle
This doesn't flip the switch for prefix generation yet, that'll be done in a followup.
2022-09-30[mlir:Async][NFC] Update Async API to use prefixed accessorsRiver Riddle
This doesn't flip the switch for prefix generation yet, that'll be done in a followup.
2022-09-21[mlir] Flip Async/GPU/OpenACC/OpenMP to use Both accessorsRiver Riddle
This allows for incrementally updating the old API usages without needing to update everything at once. These will be left on Both for a little bit and then flipped to prefixed when all APIs have been updated. Differential Revision: https://reviews.llvm.org/D134386
2022-08-31[MLIR] Update pass declarations to new autogenerated filesMichele Scuttari
The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow dropping the old infrastructure. Reviewed By: mehdi_amini, rriddle Differential Review: https://reviews.llvm.org/D132838
2022-08-30Revert "[MLIR] Update pass declarations to new autogenerated files"Michele Scuttari
This reverts commit 2be8af8f0e0780901213b6fd3013a5268ddc3359.
2022-08-30[MLIR] Update pass declarations to new autogenerated filesMichele Scuttari
The patch introduces the required changes to update the pass declarations and definitions to use the new autogenerated files and allow dropping the old infrastructure. Reviewed By: mehdi_amini, rriddle Differential Review: https://reviews.llvm.org/D132838
2022-06-09[mlir][gpu] Move GPU headers into IR/ and Transforms/Mogball
Depends on D127350 Reviewed By: rriddle Differential Revision: https://reviews.llvm.org/D127352
2022-04-28[mlir] Don't iterate mutable user listVitaly Buka
executeOp.operandsMutable().append(asyncTokens) in addAsyncDependencyAfter can resize and invalidate iterators. Fixes reports like https://reviews.llvm.org/P8286 Reviewed By: herhut Differential Revision: https://reviews.llvm.org/D124577
2022-04-18[mlir:NFC] Remove the forward declaration of FuncOp in the mlir namespaceRiver Riddle
FuncOp has been moved to the `func` namespace for a little over a month, the using directive can be dropped now.
2022-01-24[mlir] Remove a bunch of unnecessary dialect dependenciesRiver Riddle
A lot of dialects have dependencies that are unnecessary, either because of copy/paste of files when creating things or some other means. This commit cleans up a bunch of the simple ones: * Copy/Paste or missed during refactoring Most of the dependencies cleaned up here look like copy/paste errors when creating new dialects/transformations, or because the dependency wasn't removed during a refactoring (e.g. when splitting the standard dialect). * Unnecessary hard coding of constant operations in matchers There are a few instances where a dialect had a dependency because it was hardcoding checks for constant operations instead of using the better m_Constant approach. Differential Revision: https://reviews.llvm.org/D118062
2022-01-19[mlir] Make locations required when adding/creating block argumentsRiver Riddle
BlockArguments gained the ability to have locations attached a while ago, but they have always been optional. This goes against the core tenant of MLIR where location information is a requirement, so this commit updates the API to require locations. Fixes #53279 Differential Revision: https://reviews.llvm.org/D117633
2022-01-18[mlir][Pass] Deprecate FunctionPass in favor of OperationPass<FuncOp>River Riddle
The only benefit of FunctionPass is that it filters out function declarations. This isn't enough to justify carrying it around, as we can simplify filter out declarations when necessary within the pass. We can also explore with better scheduling primitives to filter out declarations at the pipeline level in the future. The definition of FunctionPass is left intact for now to allow time for downstream users to migrate. Differential Revision: https://reviews.llvm.org/D117182
2021-06-10[mlir] Support pre-existing tokens in 'gpu-async-region'Christian Sigg
Allow gpu ops implementing the async interface to already be async when running the GpuAsyncRegionPass. That pass threads a 'current token' through a block with ops implementing the gpu async interface. After this change, existing async ops (returning a !gpu.async.token) set the current token. Existing synchronous `gpu.wait` ops reset the current token. Reviewed By: herhut Differential Revision: https://reviews.llvm.org/D103396
2021-05-06[mlir] Add support for ops with regions in 'gpu-async-region' rewriter.Christian Sigg
Reviewed By: herhut Differential Revision: https://reviews.llvm.org/D101757
2021-02-25Pass GPU events instead of streams across async regions.Christian Sigg
Lower !gpu.async.tokens returned from async.execute regions to events instead of streams. Make !gpu.async.token returned from !async.execute single-use. This allows creating one event per use and destroying them without leaking or ref-counting. Technically we only need this for stream/event-based lowering. I kept the code separate from the rest of the gpu-async-region pass so that we can make this optional or move to a separate pass as needed. Reviewed By: herhut Differential Revision: https://reviews.llvm.org/D96965
2021-02-09[mlir][IR] Remove the concept of `OperationProperties`River Riddle
These properties were useful for a few things before traits had a better integration story, but don't really carry their weight well these days. Most of these properties are already checked via traits in most of the code. It is better to align the system around traits, and improve the performance/cost of traits in general. Differential Revision: https://reviews.llvm.org/D96088
2021-01-11[mlir] Make GpuAsyncRegion pass depend on async dialect.Christian Sigg
Do not cache gpu.async.token type so that the pass can be created before the GPU dialect is registered. Reviewed By: ftynse Differential Revision: https://reviews.llvm.org/D94397
2020-12-17[mlir] Remove the MutableDictionaryAttr classRiver Riddle
This class used to serve a few useful purposes: * Allowed containing a null DictionaryAttr * Provided some simple mutable API around a DictionaryAttr The first of which is no longer an issue now that there is much better caching support for attributes in general, and a cache in the context for empty dictionaries. The second results in more trouble than it's worth because it mutates the internal dictionary on every action, leading to a potentially large number of dictionary copies. NamedAttrList is a much better alternative for the second use case, and should be modified as needed to better fit it's usage as a DictionaryAttrBuilder. Differential Revision: https://reviews.llvm.org/D93442
2020-12-16[mlir] Fix for gpu-async-region pass.Christian Sigg
- the !gpu.async.token is the second result of 'gpu.alloc async', not the first. - async.execute construction takes operand types not yet wrapped in !async.value. - fix typo Reviewed By: herhut Differential Revision: https://reviews.llvm.org/D93156
2020-12-09[mlir] Use mlir::OpState::operator->() to get to methods of mlir::Operation. ↵Christian Sigg
This is a preparation step to remove the corresponding methods from OpState. Reviewed By: silvas, rriddle Differential Revision: https://reviews.llvm.org/D92878
2020-12-03[mlir][gpu] Move gpu.wait ops from async.execute regions to its dependencies.Christian Sigg
This can prevent unnecessary host synchronization. Reviewed By: herhut Differential Revision: https://reviews.llvm.org/D90346
2020-11-02[MLIR] Work around an ICE in GCC 7.Benjamin Kramer
Looks like we have a blind spot in the testing matrix. AsyncRegionRewriter.cpp: In member function ‘virtual void {anonymous}::GpuAsyncRegionPass::runOnFunction()’: AsyncRegionRewriter.cpp:113:16: internal compiler error: in replace_placeholders_r, at cp/tree.c:2804 if (getFunction() ~~~~~~~~~~~~~ .getRegion() ~~~~~~~~~~~~ .walk(Callback{OpBuilder{&getContext()}}) ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-10-29[mlir][gpu] Add pass to make GPU ops within a region execute asynchronously.Christian Sigg
Do not use the pass yet, except in a test. Reviewed By: herhut Differential Revision: https://reviews.llvm.org/D89937