summaryrefslogtreecommitdiff
path: root/llvm/lib/Target/ARM/ARMLoadStoreOptimizer.cpp
AgeCommit message (Collapse)Author
2025-05-23[NFC][CodeGen] Adopt MachineFunctionProperties convenience accessors (#141101)Rahul Joshi
2025-04-13[llvm] Call *Map::erase directly (NFC) (#135545)Kazu Hirata
2025-03-13[ARM] Pass ArrayRef by value instead of const reference. NFCCraig Topper
2024-11-12[ARM] Remove unused includes (NFC) (#115995)Kazu Hirata
Identified with misc-include-cleaner.
2024-10-11[ARM] Avoid repeated hash lookups (NFC) (#111935)Kazu Hirata
2024-10-10[ARM] Fix formatting (NFC)Kazu Hirata
I'm about to post a PR in this area.
2024-09-22[ARM] Avoid repeated hash lookups (NFC) (#109569)Kazu Hirata
2024-08-24[Target] Use llvm::replace (NFC) (#105942)Kazu Hirata
2024-07-17Rapply "[Target] Use range-based for loops (NFC) (#98844)"Kazu Hirata
This iteration drops hunks where the loop body adds more elements.
2024-07-15Revert "[Target] Use range-based for loops (NFC) (#98844)"Kazu Hirata
This reverts commit 3614f65a7ba9d925010e3316a1d93bcebc632178. fixupImmediateBr seems to resize ImmBranches.
2024-07-15[Target] Use range-based for loops (NFC) (#98844)Kazu Hirata
2024-06-11[CodeGen][NewPM] Split `MachineDominatorTree` into a concrete analysis ↵paperchalice
result (#94571) Prepare for new pass manager version of `MachineDominatorTreeAnalysis`. We may need a machine dominator tree version of `DomTreeUpdater` to handle `SplitCriticalEdge` in some CodeGen passes.
2024-04-24[CodeGen] Make the parameter TRI required in some functions. (#85968)Xu Zhang
Fixes #82659 There are some functions, such as `findRegisterDefOperandIdx` and `findRegisterDefOperand`, that have too many default parameters. As a result, we have encountered some issues due to the lack of TRI parameters, as shown in issue #82411. Following @RKSimon 's suggestion, this patch refactors 9 functions, including `{reads, kills, defines, modifies}Register`, `registerDefIsDead`, and `findRegister{UseOperandIdx, UseOperand, DefOperandIdx, DefOperand}`, adjusting the order of the TRI parameter and making it required. In addition, all the places that call these functions have also been updated correctly to ensure no additional impact. After this, the caller of these functions should explicitly know whether to pass the `TargetRegisterInfo` or just a `nullptr`.
2024-03-08Reapply "Convert many LivePhysRegs uses to LiveRegUnits" (#84338)AtariDreams
This only converts the instances where all that is needed is to change the variable type name. Basically, anything that involves a function that LiveRegUnits does not directly have was skipped to play it safe. Reverts https://github.com/llvm/llvm-project/commit/7a0e222a17058a311b69153d0b6f1b4459414778
2024-03-07Revert "Convert many LivePhysRegs uses to LiveRegUnits (#83905)"Jay Foad
This reverts commit 2a13422b8bcee449405e3ebff957b4020805f91c. It was causing test failures on the expensive check builders.
2024-03-06Convert many LivePhysRegs uses to LiveRegUnits (#83905)AtariDreams
2024-02-26[ARM] Update IsRestored for LR based on all returns (#82745)ostannard
PR #75527 fixed ARMFrameLowering to set the IsRestored flag for LR based on all of the return instructions in the function, not just one. However, there is also code in ARMLoadStoreOptimizer which changes return instructions, but it set IsRestored based on the one instruction it changed, not the whole function. The fix is to factor out the code added in #75527, and also call it from ARMLoadStoreOptimizer if it made a change to return instructions. Fixes #80287.
2023-12-24[Target] Use range-based for loops (NFC)Kazu Hirata
2023-10-12[llvm] Use llvm::erase_if (NFC)Kazu Hirata
2023-07-14[ARM] Adjust strd/ldrd codegen alignment requirementsMaurice Heumann
In change https://reviews.llvm.org/D152790, it was discovered that the alignment requirement calculation for LDRD/STRD codegen was suboptimal and the calculation for volatile loads and stores was adjusted. This change here adopts the calculation for the remaining non-volatile occurances. Recommitting after undefined behavior fix in D155093. Differential Revision: https://reviews.llvm.org/D153800
2023-07-03Revert "[ARM] Adjust strd/ldrd codegen alignment requirements"David Spickett
This reverts commit 92a9c30c61da7f973d55cd84fade424159b9cac9. This has caused a test failure in the 2nd stage of Linaro's Arm 32 bit buildbots. LLVM::simplified-template-names.s 7: error: Simplified template DW_AT_name could not be reconstituted: check:10'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 8: original: f3<unsigned char, (unsigned char)'\x00'> check:10'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 9: reconstituted: f3<unsigned char, (unsigned char)'\x7f'> check:10'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I suspect a load/store is slightly off.
2023-07-02[ARM] Adjust strd/ldrd codegen alignment requirementsMaurice Heumann
In change https://reviews.llvm.org/D152790, it was discovered that the alignment requirement calculation for LDRD/STRD codegen was suboptimal and the calculation for volatile loads and stores was adjusted. This change here adopts the calculation for the remaining non-volatile occurances. Differential Revision: https://reviews.llvm.org/D153800
2023-05-04Fix a warning in D149762 [-Wunused-variable]NAKAMURA Takumi
2023-05-03Change if() continue; to an assert if a DBG_VALUE or DBG_VALUE_LIST returns ↵Shubham Sandeep Rastogi
a null DILocalVariable A DBG_VALUE or DBG_VALUE_LIST must always return a non-null DILocalVariable, the ARMLoadStoreOptimizer code that move’s DBG_VALUE and DBG_VALUE_LIST instructions if their corresponding loads have been moved, currently just continues if it finds a DBG_VALUE or DBG_VALUE_LIST with a null DILocalVariable, change that to an assert. Differential revision: https://reviews.llvm.org/D149762
2023-04-24Move DBG_VALUE's that depend on loads to after aShubham Sandeep Rastogi
load if the load is moved due to the pre register allocation ld/st optimization pass The issue here is that there can be a scenario where debug information is lost because of the pre register allocation load store optimization pass, where a load who's result describes the debug infomation for a local variable gets moved below the load and that causes the debug information for that load to get lost. Example: Before the Pre Register Allocation Load Store Pass inst_a %2 = ld ... inst_b DBG_VALUE %2, "x", ... %3 = ld ... After the Pass: inst_a inst_b DBG_VALUE %2, "x", ... %2 = ld ... %3 = ld ... The load has now been moved to after the DBG_VAL that uses its result and the debug info for "x" has been lost. What we want is: inst_a inst_b %2 = ld ... DBG_VALUE %2, "x", ... %3 = ld ... Which is what this patch addresses Differential Revision: https://reviews.llvm.org/D145168
2023-04-12Revert "Move DBG_VALUE's that depend on loads to after a"Shubham Sandeep Rastogi
This reverts commit 0aaf634152f25a805563d552e72d89e8202d84f2. Reverted this because of build failure https://lab.llvm.org/buildbot#builders/245/builds/7035 /home/tcwg-buildbot/worker/clang-armv8-quick/llvm/llvm/test/DebugInfo/Generic/incorrect-variable-debugloc1.ll:28:12: error: DWARF23: expected string not found in input ; DWARF23: DW_OP_lit13{{$}} ^ <stdin>:1:1: note: scanning from here -: file format elf32-littlearm ^ <stdin>:19:20: note: possible intended match here DW_AT_frame_base (DW_OP_reg13 SP) ^
2023-04-12Move DBG_VALUE's that depend on loads to after aShubham Sandeep Rastogi
load if the load is moved due to the pre register allocation ld/st optimization pass The issue here is that there can be a scenario where debug information is lost because of the pre register allocation load store optimization pass, where a load who's result describes the debug infomation for a local variable gets moved below the load and that causes the debug information for that load to get lost. Example: Before the Pre Register Allocation Load Store Pass inst_a %2 = ld ... inst_b DBG_VALUE %2, "x", ... %3 = ld ... After the Pass: inst_a inst_b DBG_VALUE %2, "x", ... %2 = ld ... %3 = ld ... The load has now been moved to after the DBG_VAL that uses its result and the debug info for "x" has been lost. What we want is: inst_a inst_b %2 = ld ... DBG_VALUE %2, "x", ... %3 = ld ... Which is what this patch addresses Differential Revision: https://reviews.llvm.org/D145168
2022-06-20[ARM] Allow distributing postinc with PHI usesDavid Green
Although this doesn't usually come up, we can have uses of the BaseAccess of a distributed postinc being a PHI. This doesn't need the usual dominance check as we will dominate along the phi edge, allowing us to still create a postinc load/store. Differential Revision: https://reviews.llvm.org/D127676
2022-05-26[Target] use getSubtarget<> instead of static_cast<>(getSubtarget())Zongwei Lan
Differential Revision: https://reviews.llvm.org/D125391
2022-03-16Cleanup codegen includesserge-sans-paille
This is a (fixed) recommit of https://reviews.llvm.org/D121169 after: 1061034926 before: 1063332844 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D121681
2022-03-12Cleanup includes: DebugInfo & CodeGenserge-sans-paille
Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D121332
2022-03-10Revert "Cleanup codegen includes"Nico Weber
This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20. Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang, and many LLVM tests, see comments on https://reviews.llvm.org/D121169
2022-03-10Cleanup codegen includesserge-sans-paille
after: 1061034926 before: 1063332844 Differential Revision: https://reviews.llvm.org/D121169
2021-12-12[Target] Use llvm::reverse (NFC)Kazu Hirata
2021-12-07[ARM] Implement PAC return address signing mechanism for PACBTI-MTies Stuij
This patch implements PAC return address signing for armv8-m. This patch roughly accomplishes the following things: - PAC and AUT instructions are generated. - They're part of the stack frame setup, so that shrink-wrapping can move them inwards to cover only part of a function - The auth code generated by PAC is saved across subroutine calls so that AUT can find it again to check - PAC is emitted before stacking registers (so that the SP it signs is the one on function entry). - The new pseudo-register ra_auth_code is mentioned in the DWARF frame data - With CMSE also in use: PAC is emitted before stacking FPCXTNS, and AUT validates the corresponding value of SP - Emit correct unwind information when PAC is replaced by PACBTI - Handle tail calls correctly Some notes: We make the assembler accept the `.save {ra_auth_code}` directive that is emitted by the compiler when it saves a register that contains a return address authentication code. For EHABI we need to have the `FrameSetup` flag on the instruction and handle the `t2PACBTI` opcode (identically to `t2PAC`), so we can emit `.save {ra_auth_code}`, instead of `.save {r12}`. For PACBTI-M, the instruction which computes return address PAC should use SP value before adjustment for the argument registers save are (used for variadic functions and when a parameter is is split between stack and register), but at the same it should be after the instruction that saves FPCXT when compiling a CMSE entry function. This patch moves the varargs SP adjustment after the FPCXT save (they are never enabled at the same time), so in a following patch handling of the `PAC` instruction can be placed between them. Epilogue emission code adjusted in a similar manner. PACBTI-M code generation should not emit any instructions for architectures v6-m, v8-m.base, and for A- and R-class cores. Diagnostic message for such cases is handled separately by a future ticket. note on tail calls: If the called function has four arguments that occupy registers `r0`-`r3`, the only option for holding the function pointer itself is `r12`, but this register is used to keep the PAC during function/prologue epilogue and clobbers the function pointer. When we do the tail call we need the five registers (`r0`-`r3` and `r12`) to keep six values - the four function arguments, the function pointer and the PAC, which is obviously impossible. One option would be to authenticate the return address before all callee-saved registers are restored, so we have a scratch register to temporarily keep the value of `r12`. The issue with this approach is that it violates a fundamental invariant that PAC is computed using CFA as a modifier. It would also mean using separate instructions to pop `lr` and the rest of the callee-saved registers, which would offset the advantages of doing a tail call. Instead, this patch disables indirect tail calls when the called function take four or more arguments and the return address sign and authentication is enabled for the caller function, conservatively assuming the caller function would spill LR. This patch is part of a series that adds support for the PACBTI-M extension of the Armv8.1-M architecture, as detailed here: https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/armv8-1-m-pointer-authentication-and-branch-target-identification-extension The PACBTI-M specification can be found in the Armv8-M Architecture Reference Manual: https://developer.arm.com/documentation/ddi0553/latest The following people contributed to this patch: - Momchil Velikov - Ties Stuij Reviewed By: danielkiss Differential Revision: https://reviews.llvm.org/D112429
2021-12-02[ARM] Introduce i8neg and i8pos addressing modesDavid Green
Some instructions with i8 immediate ranges can only hold negative values (like t2LDRHi8), only hold positive values (like t2STRT) or hold +/- depending on the U bit (like the pre/post inc instructions. e.g t2LDRH_POST). This patch splits the AddrModeT2_i8 into AddrModeT2_i8, AddrModeT2_i8pos and AddrModeT2_i8neg to make this clear. This allows us to get the offset ranges of t2LDRHi8 correct in the load/store optimizer, fixing issues where we could end up creating instructions with positive offsets (which may then be encoded as ldrht). Differential Revision: https://reviews.llvm.org/D114638
2021-11-26[Target] Use range-based for loops (NFC)Kazu Hirata
2021-11-26[Target] Use range-based for loops (NFC)Kazu Hirata
2021-11-23[llvm] Use range-based for loops (NFC)Kazu Hirata
2021-09-19[llvm] Use pop_back_val (NFC)Kazu Hirata
2021-09-10[CodeGen, Target] Use pred_empty and succ_empty (NFC)Kazu Hirata
2021-09-02[ARM] Add a tail-predication loop predicate registerDavid Green
The semantics of tail predication loops means that the value of LR as an instruction is executed determines the predicate. In other words: mov r3, #3 DLSTP lr, r3 // Start tail predication, lr==3 VADD.s32 q0, q1, q2 // Lanes 0,1 and 2 are updated in q0. mov lr, #1 VADD.s32 q0, q1, q2 // Only first lane is updated. This means that the value of lr cannot be spilled and re-used in tail predication regions without potentially altering the behaviour of the program. More lanes than required could be stored, for example, and in the case of a gather those lanes might not have been setup, leading to alignment exceptions. This patch adds a new lr predicate operand to MVE instructions in order to keep a reference to the lr that they use as a tail predicate. It will usually hold the zeroreg meaning not predicated, being set to the LR phi value in the MVETPAndVPTOptimisationsPass. This will prevent it from being spilled anywhere that it needs to be used. A lot of tests needed updating. Differential Revision: https://reviews.llvm.org/D107638
2021-08-10[ARM] Change a couple of instances of LiveRegs.contains to !LiveRegs.availableDavid Green
This changes a couple of calls to LiveRegs.contains to !LiveRegs.available, one in Thumb1FrameLoweringInfo (which modifies a test to look more correct to me, given r7 should be the frame pointer so is not available), and another in the ARMLoadStoreOptimizer, that I don't have a test for, it was just found by inspection. Differential Revision: https://reviews.llvm.org/D107454
2021-07-26[ARM] Ensure correct regclass in distributing postincDavid Green
The register class required for some MVE loads/stores is more constrained than the register we use when creating postinc. Make sure we constrain the register class to keep the code correct.
2021-02-24[ARM] Expand the range of allowed post-incs in load/store optimizerDavid Green
Currently the load/store optimizer will only fold in increments of the same size as the load/store. This patch expands that to any legal immediate for the post-inc instruction. This is a recommit of 3b34b06fc5908b with correctness fixes and extra tests. Differential Revision: https://reviews.llvm.org/D95885
2021-02-19Revert "[ARM] Expand the range of allowed post-incs in load/store optimizer"David Green
This reverts commit 3b34b06fc5908b4f7dc720c0655d5756bd8e2a28 as runtime errors were reported.
2021-02-18[ARM] Expand the range of allowed post-incs in load/store optimizerDavid Green
Currently the load/store optimizer will only fold in increments of the same size as the load/store. This patch expands that to any legal immediate for the post-inc instruction. Differential Revision: https://reviews.llvm.org/D95885
2021-02-15[ARM] Extend search for increment in load/store optimizerDavid Green
Currently the findIncDecAfter will only look at the next instruction for post-inc candidates in the load/store optimizer. This extends that to a search through the current BB, until an instruction that modifies or uses the increment reg is found. This allows more post-inc load/stores and ldm/stm's to be created, especially in cases where a schedule might move instructions further apart. We make sure not to look any further for an SP, as that might invalidate stack slots that are still in use. Differential Revision: https://reviews.llvm.org/D95881
2021-01-11[ARM] Add uses for locals introduced for debug messages. NFC.Stephan Herhut
This adds uses for locals introduced for new debug messages for the load store optimizer. Those locals are only used on debug statements and otherwise create unused variable warnings. Differential Revision: https://reviews.llvm.org/D94398
2021-01-11[ARM] Add debug messages for the load store optimizer. NFCDavid Green