summaryrefslogtreecommitdiff
path: root/mlir/lib/Dialect/Vector/Transforms/VectorTransferSplitRewritePatterns.cpp
AgeCommit message (Collapse)Author
2025-07-25[mlir][NFC] update `mlir/Dialect` create APIs (32/n) (#150657)Maksim Levental
See https://github.com/llvm/llvm-project/pull/147168 for more info.
2025-07-22[mlir][NFC] update `mlir/Dialect` create APIs (24/n) (#149931)Maksim Levental
See https://github.com/llvm/llvm-project/pull/147168 for more info.
2025-07-14[mlir] Remove unused includes (NFC) (#148769)Kazu Hirata
These are identified by misc-include-cleaner. I've filtered out those that break builds. Also, I'm staying away from llvm-config.h, config.h, and Compiler.h, which likely cause platform- or compiler-specific build failures.
2025-05-12[mlir][vector] Standardize `base` Naming Across Vector Ops (NFC) (#137859)Andrzej Warzyński
[mlir][vector] Standardize base Naming Across Vector Ops (NFC) This change standardizes the naming convention for the argument representing the value to read from or write to in Vector ops that interface with Tensors or MemRefs. Specifically, it ensures that all such ops use the name `base` (i.e., the base address or location to which offsets are applied). Updated operations: * `vector.transfer_read`, * `vector.transfer_write`. For reference, these ops already use `base`: * `vector.load`, `vector.store`, `vector.scatter`, `vector.gather`, `vector.expandload`, `vector.compressstore`, `vector.maskedstore`, `vector.maskedload`. This is a non-functional change (NFC) and does not alter the semantics of these operations. However, it does require users of the XFer ops to switch from `op.getSource()` to `op.getBase()`. To ease the transition, this PR temporarily adds a `getSource()` interface method for compatibility. This is intended for downstream use only and should not be relied on upstream. The method will be removed prior to the LLVM 21 release. Implements #131602
2025-04-14[mlir] Use llvm::append_range (NFC) (#135722)Kazu Hirata
2025-01-21[mlir][IR][NFC] Move free-standing functions to `MemRefType` (#123465)Matthias Springer
Turn free-standing `MemRefType`-related helper functions in `BuiltinTypes.h` into member functions.
2024-04-19Switch member calls to `isa/dyn_cast/cast/...` to free function calls. (#89356)Christian Sigg
This change cleans up call sites. Next step is to mark the member functions deprecated. See https://mlir.llvm.org/deprecation and https://discourse.llvm.org/t/preferred-casting-style-going-forward.
2024-02-08[MLIR] Fix crash in AffineMap::replace for zero result maps (#80930)Uday Bondhugula
Fix obvious bug in AffineMap::replace for the case of zero result maps. Extend/complete inferExprsFromList to work with empty expression lists.
2024-01-17[mlir][IR] Rename "update root" to "modify op" in rewriter API (#78260)Matthias Springer
This commit renames 4 pattern rewriter API functions: * `updateRootInPlace` -> `modifyOpInPlace` * `startRootUpdate` -> `startOpModification` * `finalizeRootUpdate` -> `finalizeOpModification` * `cancelRootUpdate` -> `cancelOpModification` The term "root" is a misnomer. The root is the op that a rewrite pattern matches against (https://mlir.llvm.org/docs/PatternRewriter/#root-operation-name-optional). A rewriter must be notified of all in-place op modifications, not just in-place modifications of the root (https://mlir.llvm.org/docs/PatternRewriter/#pattern-rewriter). The old function names were confusing and have contributed to various broken rewrite patterns. Note: The new function names use the term "modify" instead of "update" for consistency with the `RewriterBase::Listener` terminology (`notifyOperationModified`).
2023-11-16[mlir][vector] Clean up VectorTransferOpInterface (#72353)Matthias Springer
- Better documentation. - Rename interface methods: `source` -> `getSource`, `indices` -> `getIndices`, etc. to conform with MLIR naming conventions. A default implementation is not needed. - Turn many interface methods into helper functions. Most of the previous interface methods were not meant to be overridden, and if some were overridden without others, the op would be have been broken.
2023-07-14[mlir][vector][NFC] Minor VectorTransferOpInterface cleanupMatthias Springer
* Rename functions with underscore to camel case. * Return C++ bools of "in_bounds" values instead of an `ArrayAttr`. Differential Revision: https://reviews.llvm.org/D155277
2023-07-11[mlir][vector] Handle memory space conflicts in VectorTransferSplit patternsQuinn Dawkins
Currently the transfer splitting patterns will generate an invalid cast when the source memref for a transfer op has a non-default memory space. This is handled by first introducing a `memref.memory_space_cast` in such cases. Differential Revision: https://reviews.llvm.org/D154515
2023-07-03[mlir][vector] Clean up some dimension size checksMatthias Springer
* Add `memref::getMixedSize` (same as in the tensor dialect). * Simplify in-bounds check in `VectorTransferSplitRewritePatterns.cpp` and fix off-by-one error in the static in-bounds check. * Use "memref::DimOp" instead of `createOrFoldDimOp` when possible. Differential Revision: https://reviews.llvm.org/D154218
2023-06-22[mlir][affine] More efficient `makeComposedFolded...` helpersMatthias Springer
The old code used to materialize constants as ops, immediately folded them into the resulting affine map and then deleted the constant ops again. Instead, directly fold the attributes into the affine map. Furthermore, all helpers accept `OpFoldResult` instead of `Value` now. This makes the code at call sites more efficient, because it is no longer necessary to materialize a `Value`, just to be able to use these helper functions. Note: The API has changed (accepts OpFoldResult instead of Value), otherwise this change is NFC. Differential Revision: https://reviews.llvm.org/D153324
2023-05-12[mlir] Move casting calls from methods to function callsTres Popp
The MLIR classes Type/Attribute/Operation/Op/Value support cast/dyn_cast/isa/dyn_cast_or_null functionality through llvm's doCast functionality in addition to defining methods with the same name. This change begins the migration of uses of the method to the corresponding function call as has been decided as more consistent. Note that there still exist classes that only define methods directly, such as AffineExpr, and this does not include work currently to support a functional cast/isa call. Caveats include: - This clang-tidy script probably has more problems. - This only touches C++ code, so nothing that is being generated. Context: - https://mlir.llvm.org/deprecation/ at "Use the free function variants for dyn_cast/cast/isa/…" - Original discussion at https://discourse.llvm.org/t/preferred-casting-style-going-forward/68443 Implementation: This first patch was created with the following steps. The intention is to only do automated changes at first, so I waste less time if it's reverted, and so the first mass change is more clear as an example to other teams that will need to follow similar steps. Steps are described per line, as comments are removed by git: 0. Retrieve the change from the following to build clang-tidy with an additional check: https://github.com/llvm/llvm-project/compare/main...tpopp:llvm-project:tidy-cast-check 1. Build clang-tidy 2. Run clang-tidy over your entire codebase while disabling all checks and enabling the one relevant one. Run on all header files also. 3. Delete .inc files that were also modified, so the next build rebuilds them to a pure state. 4. Some changes have been deleted for the following reasons: - Some files had a variable also named cast - Some files had not included a header file that defines the cast functions - Some files are definitions of the classes that have the casting methods, so the code still refers to the method instead of the function without adding a prefix or removing the method declaration at the same time. ``` ninja -C $BUILD_DIR clang-tidy run-clang-tidy -clang-tidy-binary=$BUILD_DIR/bin/clang-tidy -checks='-*,misc-cast-functions'\ -header-filter=mlir/ mlir/* -fix rm -rf $BUILD_DIR/tools/mlir/**/*.inc git restore mlir/lib/IR mlir/lib/Dialect/DLTI/DLTI.cpp\ mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp\ mlir/lib/**/IR/\ mlir/lib/Dialect/SparseTensor/Transforms/SparseVectorization.cpp\ mlir/lib/Dialect/Vector/Transforms/LowerVectorMultiReduction.cpp\ mlir/test/lib/Dialect/Test/TestTypes.cpp\ mlir/test/lib/Dialect/Transform/TestTransformDialectExtension.cpp\ mlir/test/lib/Dialect/Test/TestAttributes.cpp\ mlir/unittests/TableGen/EnumsGenTest.cpp\ mlir/test/python/lib/PythonTestCAPI.cpp\ mlir/include/mlir/IR/ ``` Differential Revision: https://reviews.llvm.org/D150123
2023-04-20[mlir][Affine][NFC] Wrap dialect in "affine" namespaceMatthias Springer
This cleanup aligns the affine dialect with all the other dialects. Differential Revision: https://reviews.llvm.org/D148687
2023-03-23[mlir][Vector] NFC - Reorganize vector patternsNicolas Vasilache
Vector dialect patterns have grown enormously in the past year to a point where they are now impenetrable. Start reorganizing them towards finer-grained control. Differential Revision: https://reviews.llvm.org/D146736
2023-01-20[MLIR] Remove scf.if builder with explicit result types and callbacksFrederik Gossen
Instead, use the builder and infer the return type based on the inner `yield` ops. Also, fix uses that do not create the terminator as required for the callback builders. Differential Revision: https://reviews.llvm.org/D142056
2023-01-14[mlir] Use std::optional instead of llvm::Optional (NFC)Kazu Hirata
This patch replaces (llvm::|)Optional< with std::optional<. I'll post a separate patch to remove #include "llvm/ADT/Optional.h". This is part of an effort to migrate from llvm::Optional to std::optional: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-01-13[mlir] Add #include <optional> (NFC)Kazu Hirata
This patch adds #include <optional> to those files containing llvm::Optional<...> or Optional<...>. I'll post a separate patch to actually replace llvm::Optional with std::optional. This is part of an effort to migrate from llvm::Optional to std::optional: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-01-12[mlir][ods] Generate inferReturnTypes for ops with TypesMatchWithJeff Niu
Ops that use TypesMatchWith to constrain result types for verification and to infer result types during parser generation should also be able to have the `inferReturnTypes` method auto generated. This patch upgrades the logic for generating `inferReturnTypes` to handle the TypesMatchWith trait by building a type inference graph where each edge corresponds to "type of A can be inferred from type of B", supporting transformers other than `"$_self"`. Reviewed By: lattner, rriddle Differential Revision: https://reviews.llvm.org/D141231
2023-01-12[mlir] Add operations to BlockAndValueMapping and rename it to IRMappingJeff Niu
The patch adds operations to `BlockAndValueMapping` and renames it to `IRMapping`. When operations are cloned, old operations are mapped to the cloned operations. This allows mapping from an operation to a cloned operation. Example: ``` Operation *opWithRegion = ... Operation *opInsideRegion = &opWithRegion->front().front(); IRMapping map Operation *newOpWithRegion = opWithRegion->clone(map); Operation *newOpInsideRegion = map.lookupOrNull(opInsideRegion); ``` Migration instructions: All includes to `mlir/IR/BlockAndValueMapping.h` should be replaced with `mlir/IR/IRMapping.h`. All uses of `BlockAndValueMapping` need to be renamed to `IRMapping`. Reviewed By: rriddle, mehdi_amini Differential Revision: https://reviews.llvm.org/D139665
2022-12-03[mlir] Use std::nullopt instead of None (NFC)Kazu Hirata
This patch mechanically replaces None with std::nullopt where the compiler would warn if None were deprecated. The intent is to reduce the amount of manual work required in migrating from Optional to std::optional. This is part of an effort to migrate from llvm::Optional to std::optional: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-11-21Merge kDynamicSize and kDynamicSentinel into one constant.Aliia Khasanova
resolve conflicts Differential Revision: https://reviews.llvm.org/D138282
2022-11-18[mlir] GreedyPatternRewriter: Reprocess modified opsMatthias Springer
Ops that were modifed in-place (`finalizeRootUpdate` was called) should be reprocessed by the GreedyPatternRewriter. This is currently not happening with `GreedyRewriteConfig::maxIterations = 1`. Note: If your project goes into an infinite loop because of this change, you likely have one or multiple faulty patterns that modify the same operations in-place (`updateRootInplace`) indefinitely. Differential Revision: https://reviews.llvm.org/D138038
2022-09-29[mlir][arith] Change dialect name from Arithmetic to ArithJakub Kuderski
Suggested by @lattner in https://discourse.llvm.org/t/rfc-define-precise-arith-semantics/65507/22. Tested with: `ninja check-mlir check-mlir-integration check-mlir-mlir-spirv-cpu-runner check-mlir-mlir-vulkan-runner check-mlir-examples` and `bazel build --config=generic_clang @llvm-project//mlir:all`. Reviewed By: lattner, Mogball, rriddle, jpienaar, mehdi_amini Differential Revision: https://reviews.llvm.org/D134762
2022-09-16[mlir] switch bufferization to use strided layout attributeAlex Zinenko
Bufferization already makes the assumption that buffers pass function boundaries in the strided form and uses the corresponding affine map layouts. Switch it to use the recently introduced strided layout instead to avoid unnecessary casts when bufferizing further operations to the memref dialect counterparts that now largely rely on the strided layout attribute. Depends On D133947 Reviewed By: nicolasvasilache Differential Revision: https://reviews.llvm.org/D133951
2022-06-20[mlir] move SCF headers to SCF/{IR,Transforms} respectivelyAlex Zinenko
This aligns the SCF dialect file layout with the majority of the dialects. Reviewed By: jpienaar Differential Revision: https://reviews.llvm.org/D128049
2022-04-25[mlir][vector] insert `alloca`s outside of loopsAlex Zinenko
After https://reviews.llvm.org/D119743 added the `AutomaticAllocationScope` trait to loop-like constructs, the vector transfer full/partial splitting pass started inserting allocations for temporaries within the closest loop rather than the closest function (or other allocation scope such as `async.execute`). While this is correct as long as the lowered code takes care of automatic deallocation at the end of each iteration of the loop, this interferes with downstream optimizations that expect `alloca`s to be at the function level. Step over loops when looking for the closest allocation scope in vector transfer full/partial splitting pass thus restoring the original behavior. Reviewed By: hanchung Differential Revision: https://reviews.llvm.org/D124366
2022-03-28[mlir] Flip Vector dialect accessors used to prefixed form.Jacques Pienaar
This has been on _Both for a couple of weeks. Flip usages in core with intention to flip flag to _Prefixed in follow up. Needed to add a couple of helper methods in AffineOps and Linalg to facilitate a pure flag flip in follow up as some of these classes are used in templates and so sensitive to Vector dialect changes. Differential Revision: https://reviews.llvm.org/D122151
2022-03-14[mlir][linalg] Replace linalg.fill by OpDSL variant.gysit
The revision removes the linalg.fill operation and renames the OpDSL generated linalg.fill_tensor operation to replace it. After the change, all named structured operations are defined via OpDSL and there are no handwritten operations left. A side-effect of the change is that the pretty printed form changes from: ``` %1 = linalg.fill(%cst, %0) : f32, tensor<?x?xf32> -> tensor<?x?xf32> ``` changes to ``` %1 = linalg.fill ins(%cst : f32) outs(%0 : tensor<?x?xf32>) -> tensor<?x?xf32> ``` Additionally, the builder signature now takes input and output value ranges as it is the case for all other OpDSL operations: ``` rewriter.create<linalg::FillOp>(loc, val, output) ``` changes to ``` rewriter.create<linalg::FillOp>(loc, ValueRange{val}, ValueRange{output}) ``` All other changes remain minimal. In particular, the canonicalization patterns are the same and the `value()`, `output()`, and `result()` methods are now implemented by the FillOpInterface. Depends On D120726 Reviewed By: nicolasvasilache Differential Revision: https://reviews.llvm.org/D120728
2022-02-15[mlir] Flipping vector dialect to both prefixed form.Jacques Pienaar
Following https://discourse.llvm.org/t/psa-ods-generated-accessors-will-change-to-have-a-get-prefix-update-you-apis/4476 Mostly mechanical, avoiding function name conflicts. Differential Revision: https://reviews.llvm.org/D119607
2022-02-07[mlir][NFC] Remove a few op builders that simply swap parameter orderRiver Riddle
Differential Revision: https://reviews.llvm.org/D119093
2022-02-02[mlir] Move StandardOps/Utils to Arithmetic and sever a bunch of ↵River Riddle
dependencies on Standard The Utils.cpp file in StandardOps essentially just contains utilities for interacting with arithmetic operations, and at this point makes more sense as a utility file for the arithemtic dialect. Differential Revision: https://reviews.llvm.org/D118280
2022-02-02[mlir][vector] Avoid hoisting alloca'ed temporary buffers across ↵Nicolas Vasilache
AutomaticAllocationScope This revision avoids incorrect hoisting of alloca'd buffers across an AutomaticAllocationScope boundary. In the more general case, we will probably need a ParallelScope-like interface. Differential Revision: https://reviews.llvm.org/D118768
2022-02-01Revert "Revert "[mlir] Purge `linalg.copy` and use `memref.copy` instead.""Alexander Belyaev
This reverts commit 25bf6a2a9bc6ecb3792199490c70c4ce50a94aea.
2022-01-31Revert "[mlir] Purge `linalg.copy` and use `memref.copy` instead."Alexander Belyaev
This reverts commit 016956b68081705ffee511c334e31e414fa1ddbf. Reverting it to fix NVidia build without being in a hurry.
2022-01-31[mlir] Purge `linalg.copy` and use `memref.copy` instead.Alexander Belyaev
Differential Revision: https://reviews.llvm.org/D118028
2022-01-31[mlir][vector][NFC] Split into IR, Transforms and UtilsMatthias Springer
This reduces the dependencies of the MLIRVector target and makes the dialect consistent with other dialects. Differential Revision: https://reviews.llvm.org/D118533