summaryrefslogtreecommitdiff
path: root/llvm/lib/Target/AMDGPU/SIOptimizeExecMaskingPreRA.cpp
AgeCommit message (Collapse)Author
2025-02-20[AMDGPU][NewPM] Port SIOptimizeExecMaskingPreRA to NPM (#125351)Akshat Oke
2024-10-03[AMDGPU] Qualify auto. NFC. (#110878)Jay Foad
Generated automatically with: $ clang-tidy -fix -checks=-*,llvm-qualified-auto $(find lib/Target/AMDGPU/ -type f)
2024-07-10[CodeGen][NewPM] Port `LiveIntervals` to new pass manager (#98118)paperchalice
- Add `LiveIntervalsAnalysis`. - Add `LiveIntervalsPrinterPass`. - Use `LiveIntervalsWrapperPass` in legacy pass manager. - Use `std::unique_ptr` instead of raw pointer for `LICalc`, so destructor and default move constructor can handle it correctly. This would be the last analysis required by `PHIElimination`.
2024-04-24[CodeGen] Make the parameter TRI required in some functions. (#85968)Xu Zhang
Fixes #82659 There are some functions, such as `findRegisterDefOperandIdx` and `findRegisterDefOperand`, that have too many default parameters. As a result, we have encountered some issues due to the lack of TRI parameters, as shown in issue #82411. Following @RKSimon 's suggestion, this patch refactors 9 functions, including `{reads, kills, defines, modifies}Register`, `registerDefIsDead`, and `findRegister{UseOperandIdx, UseOperand, DefOperandIdx, DefOperand}`, adjusting the order of the TRI parameter and making it required. In addition, all the places that call these functions have also been updated correctly to ensure no additional impact. After this, the caller of these functions should explicitly know whether to pass the `TargetRegisterInfo` or just a `nullptr`.
2023-06-16[MC] Add MCRegisterInfo::regunits for iteration over register unitsSergei Barannikov
Reviewed By: foad Differential Revision: https://reviews.llvm.org/D152098
2023-05-09[AMDGPU] Recompute liveness in SIOptimizeExecMaskingPreRApvanhout
Instead of ad-hoc updating liveness, recompute it completely for the affected register. This does not affect any existing test and fixes an edge case that caused a "Non-empty but used interval" error in the register allocator due to how the pass updated liveranges. It created a "isolated" live-through segment. Overall this change just seems to be a net positive with no side effect observed. There may be a compile time impact but it's expected to be minimal. Fixes SWDEV-388279 Reviewed By: critson Differential Revision: https://reviews.llvm.org/D150105
2023-04-08[AMDGPU] Fix a case of updating LiveIntervals in SIOptimizeExecMaskingPreRAJay Foad
This was causing two test failures when I applied D129208 to enable extra verification of LiveIntervals: LLVM :: CodeGen/AMDGPU/optimize-negated-cond-exec-masking-wave32.mir LLVM :: CodeGen/AMDGPU/optimize-negated-cond-exec-masking.mir Differential Revision: https://reviews.llvm.org/D147721
2023-04-07[AMDGPU] Fix comment in SIOptimizeExecMaskingPreRAJay Foad
2023-02-05[AMDGPU] Fix liveness in the SIOptimizeExecMaskingPreRA.cppStanislav Mekhanoshin
If a condition register def happens past the newly created use we do not properly update LIS. It has two problems: 1) We do not extend defining segment to the end of its block marking it a live-out (this is regression after https://reviews.llvm.org/rG09d38dd7704a52e8ad2d5f8f39aaeccf107f4c56) 2) We do not extend use segment to the beginning of the use block marking it a live-in. Fixes: SWDEV-379563 Differential Revision: https://reviews.llvm.org/D143302
2022-11-04AMDGPU: Fix assert when trying to overextend liverangeMatt Arsenault
This was trying to add segments beyond the new and use, so skip additional segments. This would hit (S < E && "Cannot create empty or backwards segment").
2022-07-17[AMDGPU] Improve liveness copying in si-optimize-exec-masking-pre-raCarl Ritson
Further improve liveness copying for CC register post optimization by mirroring live internal splits. The fixes a bug in register allocation when CC register liveness is extended across a branches instead of split. Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D129557
2022-07-06[AMDGPU] Additional liveness tests for si-optimize-exec-masking-pre-raCarl Ritson
Merge tests and fixes from D128110 and D128315 on top of already committed D128800. Original author: arsenm Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D128882
2022-06-30[AMDGPU] Fix liveness for loops in si-optimize-exec-masking-pre-raCarl Ritson
Follow up to D127894, new liveness update code needs to handle the case where S_ANDN2 input must be extended through loops when V_CNDMASK_B32 has been hoisted. Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D128800
2022-06-22AMDGPU: Fix invalid liveness after si-optimize-exec-masking-pre-raMatt Arsenault
This was leaving behind a use at the deleted instruction which the verifier would fail during allocation.
2022-02-18[AMDGPU][NFC] Fix typosSebastian Neubauer
Fix some typos in the amdgpu backend. Differential Revision: https://reviews.llvm.org/D119235
2021-03-24[AMDGPU] SIOptimizeExecMaskingPreRA should check constant bus constraint ↵alex-t
when folds EXEC copy Folding EXEC copy into it's single use may lead to constant bus constraint violation as it adds one more SGPR operand. This change makes it validate the user instruction with the new SGPR operand and only fold it if it is legal. Reviewed By: rampitec, arsenm Differential Revision: https://reviews.llvm.org/D98888
2021-01-20[NFC][AMDGPU] Split AMDGPUSubtarget.h to R600 and GCN subtargetsdfukalov
... to reduce headers dependency. Reviewed By: rampitec, arsenm Differential Revision: https://reviews.llvm.org/D95036
2021-01-07[NFC][AMDGPU] Reduce include files dependency.dfukalov
Reviewed By: rampitec Differential Revision: https://reviews.llvm.org/D93813
2020-11-04[NFC] Use Register/MCRegisterMircea Trofin
Differential Revision: https://reviews.llvm.org/D90724
2020-10-20[AMDGPU] Remove fix up operand from SI_ELSECarl Ritson
Remove immediate operand from SI_ELSE which indicates if EXEC has been modified. Instead always emit code that handles EXEC and remove unnecessary instructions during pre-RA optimisation. This facilitates passes (i.e. SIWholeQuadMode) adding exec mask manipulation post control flow lowering, and pre control flow lower passes do not need to be aware of SI_ELSE handling. Reviewed By: nhaehnle Differential Revision: https://reviews.llvm.org/D89644
2020-10-20[AMDGPU][NFC] Tidy SIOptimizeExecMaskingPreRA for extensibilityCarl Ritson
Remove duplicate code and move things around to make it easier to add additional optimisations to the pass. Reviewed By: rampitec Differential Revision: https://reviews.llvm.org/D89619
2020-08-20[AMDGPU] Remove uses of Register::isPhysicalRegister/isVirtualRegisterJay Foad
... in favour of the isPhysical/isVirtual methods.
2020-07-28AMDGPU: Check for other defs when folding conditions into s_andn2_b64Matt Arsenault
We can't fold the masked compare value through the select if the select condition is re-defed after the and instruction. Fixes a verifier error and trying to use the outgoing value defined in the block. I'm not sure why this pass is bothering to handle physregs. It's making this more complex and forces extra liveness computation.
2020-07-14[AMDGPU] Propagate dead flag during pre-RA exec mask optimizationsCarl Ritson
Preserve SCC dead flags in SIOptimizeExecMaskingPreRA. This helps with removing redundant s_andn2 instructions later. Reviewed By: rampitec Differential Revision: https://reviews.llvm.org/D83637
2020-04-12[AMDGPU] Don't assert on partial exec copyAustin Kerbow
After Machine CSE and coalescing we can end up with copies of exec to subregister SGPRs. Differential Revision: https://reviews.llvm.org/D77992
2020-03-12[AMDGPU] Simplify exec copiesStanislav Mekhanoshin
The patch removes late endcf handling and only leaves the related portion with redundant exec mask copy elimination. Differential Revision: https://reviews.llvm.org/D76095
2020-03-11[AMDGPU] Disable nested endcf collapseStanislav Mekhanoshin
The assumption is that conditional regions are perfectly nested and a mask restored at the exit from the inner block will be completely covered by a mask restored in the outer. It turns out with our current structurizer this is not always the case. Disable the optimization for now, but I want to keep it around for a while to either try after further structurizer changes or to move it into control flow lowering where we have more info and reuse the test. Differential Revision: https://reviews.llvm.org/D75958
2019-11-13Sink all InitializePasses.h includesReid Kleckner
This file lists every pass in LLVM, and is included by Pass.h, which is very popular. Every time we add, remove, or rename a pass in LLVM, it caused lots of recompilation. I found this fact by looking at this table, which is sorted by the number of times a file was changed over the last 100,000 git commits multiplied by the number of object files that depend on it in the current checkout: recompiles touches affected_files header 342380 95 3604 llvm/include/llvm/ADT/STLExtras.h 314730 234 1345 llvm/include/llvm/InitializePasses.h 307036 118 2602 llvm/include/llvm/ADT/APInt.h 213049 59 3611 llvm/include/llvm/Support/MathExtras.h 170422 47 3626 llvm/include/llvm/Support/Compiler.h 162225 45 3605 llvm/include/llvm/ADT/Optional.h 158319 63 2513 llvm/include/llvm/ADT/Triple.h 140322 39 3598 llvm/include/llvm/ADT/StringRef.h 137647 59 2333 llvm/include/llvm/Support/Error.h 131619 73 1803 llvm/include/llvm/Support/FileSystem.h Before this change, touching InitializePasses.h would cause 1345 files to recompile. After this change, touching it only causes 550 compiles in an incremental rebuild. Reviewers: bkramer, asbirlea, bollu, jdoerfert Differential Revision: https://reviews.llvm.org/D70211
2019-10-08AMDGPU: Propagate undef flag during pre-RA exec mask optimizationsNicolai Haehnle
Summary: Issue: https://github.com/GPUOpen-Drivers/llpc/issues/204 Reviewers: arsenm, rampitec Subscribers: kzhuravl, jvesely, wdng, yaxunl, dstuttard, tpr, t-tye, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D68184 llvm-svn: 374041
2019-08-20Revert "AMDGPU: Fix iterator error when lowering SI_END_CF"Matt Arsenault
This reverts r367500 and r369203. This is causing various test failures. llvm-svn: 369417
2019-08-15Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVMDaniel Sanders
Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-01Finish moving TargetRegisterInfo::isVirtualRegister() and friends to ↵Daniel Sanders
llvm::Register as started by r367614. NFC llvm-svn: 367633
2019-08-01Reapply "AMDGPU: Split block for si_end_cf"Matt Arsenault
This reverts commit r359363, reapplying r357634 llvm-svn: 367500
2019-06-16[AMDGPU] gfx10 conditional registers handlingStanislav Mekhanoshin
This is cpp source part of wave32 support, excluding overriden getRegClass(). Differential Revision: https://reviews.llvm.org/D63351 llvm-svn: 363513
2019-04-27Revert "AMDGPU: Split block for si_end_cf"Mark Searles
This reverts commit 7a6ef3004655dd86d722199c471ae78c28e31bb4. We discovered some internal test failures, so reverting for now. Differential Revision: https://reviews.llvm.org/D61213 llvm-svn: 359363
2019-04-23[AMDGPU] Fixed addReg() in SIOptimizeExecMaskingPreRA.cppStanislav Mekhanoshin
The second argument is flags, not subreg. Differential Revision: https://reviews.llvm.org/D61031 llvm-svn: 359017
2019-04-22AMDGPU: Fix not checking for copy when looking at copy srcMatt Arsenault
Effectively reverts r356956. The check for isFullCopy was excessive, but there still needs to be a check that this is a copy. llvm-svn: 358890
2019-04-15[AMDGPU] Fixed incorrect test in vcnd/vcmp optimizationTim Renouf
This fixes a test I introduced in change D59191 (that added src0 and src1 modifiers to the v_cndmask instruction for disassembly purposes). Spotted by David Binderman in bug 41488. Differential Revision: https://reviews.llvm.org/D60652 Change-Id: I6ac95e66cd84e812ed3359ad57bcd0e13198ba0c llvm-svn: 358392
2019-04-03AMDGPU: Split block for si_end_cfMatt Arsenault
Relying on no spill or other code being inserted before this was precarious. It relied on code diligently checking isBasicBlockPrologue which is likely to be forgotten. Ideally this could be done earlier, but this doesn't work because of phis. Any other instruction can't be placed before them, so we have to accept the position being incorrect during SSA. This avoids regressions in the fast register allocator rewrite from inverting the direction. llvm-svn: 357634
2019-03-28AMDGPU: Make exec mask optimzations more resistant to block splitsMatt Arsenault
Also improve the check for SALU instructions to also ignore implicit_def and other fake instructions. llvm-svn: 357170
2019-03-27AMDGPU: Skip debug_instr when collapsing end_cfMatt Arsenault
Based on how these are inserted, I doubt this was causing a problem in practice. llvm-svn: 357090
2019-03-25AMDGPU: Remove unnecessary check for isFullCopyMatt Arsenault
Subregister indexes are not used for physical register operands, so isFullCopy is implied by the physical register check. llvm-svn: 356956
2019-03-18[AMDGPU] Asm/disasm v_cndmask_b32_e64 with abs/neg source modifiersTim Renouf
This commit allows v_cndmask_b32_e64 with abs, neg source modifiers on src0, src1 to be assembled and disassembled. This does appear to be allowed, even though they are floating point modifiers and the operand type is b32. To do this, I added src0_modifiers and src1_modifiers to the MachineInstr, which involved fixing up several places in codegen and mir tests. Differential Revision: https://reviews.llvm.org/D59191 Change-Id: I69bf4a8c73ebc65744f6110bb8fc4e937d79fbea llvm-svn: 356398
2019-03-12[AMDGPU] Add support for immediate operand for S_ENDPGMDavid Stuttard
Summary: Add support for immediate operand in S_ENDPGM Change-Id: I0c56a076a10980f719fb2a8f16407e9c301013f6 Reviewers: alexshap Subscribers: qcolombet, arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, tpr, t-tye, eraman, arphaman, Petar.Avramovic, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D59213 llvm-svn: 355902
2019-02-22AMDGPU: Use removeAllRegUnitsForPhysRegMatt Arsenault
llvm-svn: 354686
2019-01-19Update the file headers across all of the LLVM projects in the monorepoChandler Carruth
to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2018-12-13[AMDGPU] Fix build failure, second attemptStanislav Mekhanoshin
Some compilers complain that variable is captured and some complain when it is not. Switch to [&]. llvm-svn: 349006
2018-12-13[AMDGPU] Fix build failureStanislav Mekhanoshin
Fixed error 'lambda capture 'CondReg' is not required to be captured for this use'. llvm-svn: 349005
2018-12-13[AMDGPU] Simplify negated conditionStanislav Mekhanoshin
Optimize sequence: %sel = V_CNDMASK_B32_e64 0, 1, %cc %cmp = V_CMP_NE_U32 1, %1 $vcc = S_AND_B64 $exec, %cmp S_CBRANCH_VCC[N]Z => $vcc = S_ANDN2_B64 $exec, %cc S_CBRANCH_VCC[N]Z It is the negation pattern inserted by DAGCombiner::visitBRCOND() in the rebuildSetCC(). Differential Revision: https://reviews.llvm.org/D55402 llvm-svn: 349003
2018-08-28AMDGPU: Don't delete instructions if S_ENDPGM has implicit usesMatt Arsenault
This can leave behind the uses with the defs removed. Since this should only really happen in tests, it's not worth the effort of trying to handle this. llvm-svn: 340866
generated by cgit v1.2.3 (git 2.25.1) at 2026-10-24 09:48:51 +0000