summaryrefslogtreecommitdiff
path: root/llvm/test/CodeGen/AMDGPU/llvm.mulo.ll
AgeCommit message (Collapse)Author
2025-07-23[RFC][NFC][AMDGPU] Remove `-verify-machineinstrs` from ↵Shilei Tian
`llvm/test/CodeGen/AMDGPU/*.ll` (#150024) Recent upstream trends have moved away from explicitly using `-verify-machineinstrs`, as it's already covered by the expensive checks. This PR removes almost all `-verify-machineinstrs` from tests in `llvm/test/CodeGen/AMDGPU/*.ll`, leaving only those tests where its removal currently causes failures.
2025-06-27[AMDGPU] Fix bad removal of s_delay_alu (#145728)Ana Mihajlovic
instructionWaitsForSGPRWrites function covers ALL SALU instructions, including those like s_waitcnt that don't read from sgpr. This results in removing delay_alu instructions in cases like VALU->SGPR->VALU, which results in performance regression. Change modifies the function so that it checks if instruction also reads a sgpr.
2025-04-10Reapply [AMDGPU] SIFixSgprCopies should not process twice VGPR to SGPR ↵alex-t
copies inserted by PHI preprocessing. (#135243) LIT tests which were incorrectly merged are corrected.
2025-04-10Revert "[AMDGPU] SIFixSgprCopies should not process twice VGPR to SGPR ↵Nico Weber
copies inserted by PHI preprocessing. (#134153)" This reverts commit 0563569978fee1e780a560494c89869074933f58. Breaks tests, see comments on https://github.com/llvm/llvm-project/pull/134153
2025-04-10[AMDGPU] SIFixSgprCopies should not process twice VGPR to SGPR copies ↵alex-t
inserted by PHI preprocessing. (#134153) PHI operands and results must belong to the same register class. If a PHI node produces an SGPR, but one of its operands is a VGPR, we insert a VGPR-to-SGPR copy in the operand’s source block. The PHI operand is then updated to use the destination register of the inserted copy. These inserted copies are processed immediately when they are created. Therefore, we should avoid reprocessing them when handling their parent block later. --------- Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
2025-03-28[AMDGPU] Unused sdst writing to null (#133229)Ana Mihajlovic
Unused sdst writing to null to avoid a false VALU->SALU dependency stall. This requires using the VOP3 encoding.
2025-03-18AMDGPU: Move insertion into V2SCopies map (#130776)Matt Arsenault
Insert the start instruction directly into the map before the uses. This prevents improperly re-visting sgpr->vgpr phi inputs multiple times which would trigger a use after free. I don't particularly trust the iteration scheme here. This is also unnecessarily revisting transitive users of a phi or reg_sequence for every input operand, but I will address that separately. Fixes #130646. I also believe it fixes #130119, although that test fails less consistently for me.
2025-03-13Reland "[AMDGPU] Remove s_delay_alu for VALU->SGPR->SALU (#127212)" (#131111)Ana Mihajlovic
We have a VALU->SGPR->SALU (VALU writing to SGPR and SALU reading from it). When VALU is issued, it increments internal counter VA_SDST used to track use of this SGPR. SALU will not issue until VA_SDST is zero, that is when VALU is finished writing. Therefore, delays added by s_delay_alu are not needed in this situation.
2025-03-13AMDGPU: Replace ptr addrspace(1) undefs with poison (#130900)Matt Arsenault
Many tests use store to undef as a placeholder use, so just replace all of these with poison.
2025-03-12Revert "[AMDGPU] Remove s_delay_alu for VALU->SGPR->SALU (#127212)"Kazu Hirata
This reverts commit 71582c6667a6334c688734cae628e906b3c1ac1d. Multiple buildbot failures have been reported: https://github.com/llvm/llvm-project/pull/127212
2025-03-12[AMDGPU] Remove s_delay_alu for VALU->SGPR->SALU (#127212)Ana Mihajlovic
We have a VALU->SGPR->SALU (VALU writing to SGPR and SALU reading from it). When VALU is issued, it increments internal counter VA_SDST used to track use of this SGPR. SALU will not issue until VA_SDST is zero, that is when VALU is finished writing. Therefore, delays added by s_delay_alu are not needed in this situation.
2025-01-30PeepholeOpt: Do not add subregister indexes to reg_sequence operands (#124111)Matt Arsenault
Given the rest of the pass just gives up when it needs to compose subregisters, folding a subregister extract directly into a reg_sequence is counterproductive. Later fold attempts in the function will give up on the subregister operand, preventing looking up through the reg_sequence. It may still be profitable to do these folds if we start handling the composes. There are some test regressions, but this mostly looks better.
2025-01-30[AMDGPU] Rewrite GFX12 SGPR hazard handling to dedicated pass (#118750)Carl Ritson
- Algorithm operates over whole IR to attempt to minimize waits. - Add support for VALU->VALU SGPR hazards via VA_SDST/VA_VCC.
2024-11-08Reapply "[AMDGPU] Still set up the two SGPRs for queue ptr even it is COV5 ↵Shilei Tian
(#112403)" This reverts commit ca33649abe5fad93c57afef54e43ed9b3249cd86.
2024-11-08Revert "[AMDGPU] Still set up the two SGPRs for queue ptr even it is COV5 ↵Shilei Tian
(#112403)" This reverts commit e215a1e27d84adad2635a52393621eb4fa439dc9 as it broke both hip and openmp buildbots.
2024-11-08[AMDGPU] Still set up the two SGPRs for queue ptr even it is COV5 (#112403)Shilei Tian
2024-10-21[AMDGPU] Skip VGPR deallocation for waveslot limited kernels (#112765)Stanislav Mekhanoshin
MSG_DEALLOC_VGPRS slows down very small waveslot limited kernels. It's been identified this message is only really needed for VGPR limited kernels. A kernel becomes VGPR limited if a total number of VGPRs per SIMD / number of used VGPRs is more than a number of wave slots.
2024-07-23[AMDGPU] Codegen support for constrained multi-dword sloads (#96163)Christudasan Devadasan
For targets that support xnack replay feature (gfx8+), the multi-dword scalar loads shouldn't clobber any register that holds the src address. The constrained version of the scalar loads have the early clobber flag attached to the dst operand to restrict RA from re-allocating any of the src regs for its dst operand.
2024-07-15Reapply "AMDGPU: Move attributor into optimization pipeline (#83131)" and ↵Matt Arsenault
follow up commit "clang/AMDGPU: Defeat attribute optimization in attribute test" (#98851) This reverts commit adaff46d087799072438dd744b038e6fd50a2d78. Drop the -O3 checks from default-attributes.hip. I don't know why they are different on some bots but reverting this is far too disruptive.
2024-07-14Revert "AMDGPU: Move attributor into optimization pipeline (#83131)" and ↵dyung
follow up commit "clang/AMDGPU: Defeat attribute optimization in attribute test" (#98851) This reverts commits 677cc15e0ff2e0e6aa30538eb187990a6a8f53c0 and 78bc1b64a6dc3fb6191355a5e1b502be8b3668e7. The test CodeGenHIP/default-attributes.hip is failing on multiple bots even after the attempted fix including the following: - https://lab.llvm.org/buildbot/#/builders/3/builds/1473 - https://lab.llvm.org/buildbot/#/builders/65/builds/1380 - https://lab.llvm.org/buildbot/#/builders/161/builds/595 - https://lab.llvm.org/buildbot/#/builders/154/builds/1372 - https://lab.llvm.org/buildbot/#/builders/133/builds/1547 - https://lab.llvm.org/buildbot/#/builders/81/builds/755 - https://lab.llvm.org/buildbot/#/builders/40/builds/570 - https://lab.llvm.org/buildbot/#/builders/13/builds/748 - https://lab.llvm.org/buildbot/#/builders/12/builds/1845 - https://lab.llvm.org/buildbot/#/builders/11/builds/1695 - https://lab.llvm.org/buildbot/#/builders/190/builds/1829 - https://lab.llvm.org/buildbot/#/builders/193/builds/962 - https://lab.llvm.org/buildbot/#/builders/23/builds/991 - https://lab.llvm.org/buildbot/#/builders/144/builds/2256 - https://lab.llvm.org/buildbot/#/builders/46/builds/1614 These bots have been broken for a day, so reverting to get everything back to green.
2024-07-14AMDGPU: Move attributor into optimization pipeline (#83131)Matt Arsenault
Removing it from the codegen pipeline induces a lot of test churn because llc is no longer optimizing out implicit arguments to kernels. Mostly mechanical, but there are some creative test updates. I preferred to take the changes as-is in tests where the ABI isn't relevant. In cases where it's more relevant, or the optimize out logic was too ingrained in the test, I pre-run the optimization. Some cases manually add attributes to disable inputs.
2024-01-18[AMDGPU] CodeGen for GFX12 S_WAIT_* instructions (#77438)Jay Foad
Update SIMemoryLegalizer and SIInsertWaitcnts to use separate wait instructions per counter (e.g. S_WAIT_LOADCNT) and split VMCNT into separate LOADCNT, SAMPLECNT and BVHCNT counters.
2024-01-17[AMDGPU] Disable V_MAD_U64_U32/V_MAD_I64_I32 workaround for GFX12 (#77927)Jay Foad
2024-01-16[AMDGPU,test] Change llc -march= to -mtriple= (#75982)Fangrui Song
Similar to 806761a7629df268c8aed49657aeccffa6bca449. For IR files without a target triple, -mtriple= specifies the full target triple while -march= merely sets the architecture part of the default target triple, leaving a target triple which may not make sense, e.g. amdgpu-apple-darwin. Therefore, -march= is error-prone and not recommended for tests without a target triple. The issue has been benign as we recognize $unknown-apple-darwin as ELF instead of rejecting it outrightly. This patch changes AMDGPU tests to not rely on the default OS/environment components. Tests that need fixes are not changed: ``` LLVM :: CodeGen/AMDGPU/fabs.f64.ll LLVM :: CodeGen/AMDGPU/fabs.ll LLVM :: CodeGen/AMDGPU/floor.ll LLVM :: CodeGen/AMDGPU/fneg-fabs.f64.ll LLVM :: CodeGen/AMDGPU/fneg-fabs.ll LLVM :: CodeGen/AMDGPU/r600-infinite-loop-bug-while-reorganizing-vector.ll LLVM :: CodeGen/AMDGPU/schedule-if-2.ll ```
2023-12-14[AMDGPU] Enable GCNRewritePartialRegUses pass by default. (#72975)Valery Pykhtin
Let's try once again after #69957 has landed.
2023-07-19[AMDGPU] Insert s_nop before s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)Jay Foad
Differential Revision: https://reviews.llvm.org/D155681
2023-07-04[AMDGPU] Do not wait for vscnt on function entry and returnJay Foad
SIInsertWaitcnts inserts waitcnt instructions to resolve data dependencies. The GFX10+ vscnt (VMEM store count) counter is never used in this way. It is only used to resolve memory dependencies, and that is handled by SIMemoryLegalizer. Hence there is no need to conservatively wait for vscnt to be 0 on function entry and before returns. Differential Revision: https://reviews.llvm.org/D153537
2023-06-07[AMDGPU] Turn off pass to rewrite partially used virtual superregisters ↵Valery Pykhtin
after RenameIndependentSubregs pass with registers of minimal size. There is a failure with this pass in the case when target register class for a subregister isn't known from instruction description (for ex. COPY). Currently in this situation the RC is obtained using TargetRegisterInfo::getSubRegisterClass but in general it's not working. In order to fix this two things should be done: 1. Stop processing a subregister if the target register class is unknown (conservative approach) 2. Improve deduction of subregister' target register class (i.e by processing COPY chain) I was going to implement point 1 but my tests use implicit operands for S_NOP and they don't have associated target register class and all tests fail. Therefore I decided to turn off the pass now, implement point 1 and fix my tests. Reviewed By: arsenm, #amdgpu Differential Revision: https://reviews.llvm.org/D152291
2023-05-26[AMDGPU] Add pass to rewrite partially used virtual superregisters after ↵Valery Pykhtin
RenameIndependentSubregs pass with registers of minimal size. The main purpose of this is to simplify register pressure tracking as after the pass there is no need to track subreg liveness anymore. On the other hand this pass creates more possibilites for the subreg unaware code, as many of the subregs becomes ordinary registers. Intersting sideeffect: spill-vgpr.ll has lost a lot of spills. Reviewed By: #amdgpu, arsenm Differential Revision: https://reviews.llvm.org/D139732
2022-11-29AMDGPU: Bulk update some generic intrinsic tests to opaque pointersMatt Arsenault
Done purely with the script.
2022-11-29[DAG] Attempt to replace a mul node with an existing umul_lohi/smul_lohi ↵Simon Pilgrim
node (PR59217) As discussed on Issue #59217, under certain circumstances the DAG can generate duplicate MUL and MUL_LOHI nodes, often during MULO legalization. This patch attempts to replace MUL nodes with additional uses of the LO result from the MUL_LOHI node Differential Revision: https://reviews.llvm.org/D138790
2022-09-15[AMDGPU] Always select s_cselect_b32 for uniform 'select' SDNodeAlexander Timofeev
This patch contains changes necessary to carry physical condition register (SCC) dependencies through the SDNode scheduler. It adds the edge in the SDNodeScheduler dependency graph instead of inserting the SCC copy between each definition and use. This approach lets the scheduler place instructions in an optimal way placing the copy only when the dependency cannot be resolved. Reviewed By: rampitec Differential Revision: https://reviews.llvm.org/D133593
2022-07-05[AMDGPU] gfx11 Generate VOPD InstructionsJoe Nash
We form VOPD instructions in the GCNCreateVOPD pass by combining back-to-back component instructions. There are strict register constraints for creating a legal VOPD, namely that the matching operands (e.g. src0x and src0y, src1x and src1y) must be in different register banks. We add a PostRA scheduler mutation to put possible VOPD components back-to-back. Depends on D128442, D128270 Reviewed By: #amdgpu, rampitec Differential Revision: https://reviews.llvm.org/D128656
2022-06-30[AMDGPU] GFX11: automatically release VGPRs at the end of the shaderJay Foad
GFX11 has a new message type MSG_DEALLOC_VGPRS which can be used to release a shader's VGPRs. Sending this at the end of a shader (just before the s_endpgm) can help overall system performance in cases where the s_endpgm would have to wait for outstanding VMEM stores to complete before releasing the VGPRs. Differential Revision: https://reviews.llvm.org/D128442
2022-06-29[AMDGPU] New AMDGPUInsertDelayAlu passJay Foad
Differential Revision: https://reviews.llvm.org/D128270
2022-06-16[AMDGPU] Change use null for dead sdst to be gfx1030+David Stuttard
Pre gfx1030 null for sdst is different. c97436f8b6e2 [AMDGPU] Use null for dead sdst operand - requires a change to make it not apply to pre gfx1030 Differential Revision: https://reviews.llvm.org/D127869
2022-06-13[AMDGPU] Use null for dead sdst operandStanislav Mekhanoshin
Differential Revision: https://reviews.llvm.org/D127542
2022-06-13[AMDGPU] Fix GFX11 codegen for V_MAD_U64_U32 and V_MAD_I64_I32Jay Foad
GFX11 uses different pseudos for these because of a new constraint on which operands' registers can overlap. Differential Revision: https://reviews.llvm.org/D127659
2022-01-11Revert D109159 : Revert "[amdgpu] Enable selection of `s_cselect_b64`."David Salinas
This reverts commit 640beb38e7710b939b3cfb3f4c54accc694b1d30. That commit caused performance degradtion in Quicksilver test QS:sGPU and a functional test failure in (rocPRIM rocprim.device_segmented_radix_sort). Reverting until we have a better solution to s_cselect_b64 codegen cleanup Change-Id: Ifc167b3c2dae7a65920676f22a97ba76485f3456 Reviewed By: kzhuravl Differential Revision: https://reviews.llvm.org/D116686 Change-Id: I1abf49b74a7e2ba0e0205f747a4154a468b9d7f2
2022-01-05Revert "Revert D109159 "[amdgpu] Enable selection of `s_cselect_b64`.""Nico Weber
This reverts commit 859ebca744e634dcc89a2294ffa41574f947bd62. The change contained many unrelated changes and e.g. restored unit test failes for the old lld port.
2022-01-05Revert D109159 "[amdgpu] Enable selection of `s_cselect_b64`."David Salinas
This reverts commit 640beb38e7710b939b3cfb3f4c54accc694b1d30. That commit caused performance degradtion in Quicksilver test QS:sGPU and a functional test failure in (rocPRIM rocprim.device_segmented_radix_sort). Reverting until we have a better solution to s_cselect_b64 codegen cleanup Change-Id: Ibf8e397df94001f248fba609f072088a46abae08 Reviewed By: kzhuravl Differential Revision: https://reviews.llvm.org/D115960 Change-Id: Id169459ce4dfffa857d5645a0af50b0063ce1105
2021-11-24[AMDGPU] Implement widening multiplies with v_mad_i64_i32/v_mad_u64_u32Jay Foad
Select SelectionDAG ops smul_lohi/umul_lohi to v_mad_i64_i32/v_mad_u64_u32 respectively, with an addend of 0. v_mul_lo, v_mul_hi and v_mad_i64/u64 are all quarter-rate instructions so it is better to use one instruction than two. Further improvements are possible to make better use of the addend operand, but this is already a strict improvement over what we have now. Differential Revision: https://reviews.llvm.org/D113986
2021-11-23[AMDGPU] Fix the name of a test caseJay Foad
2021-11-19[AMDGPU] Use new opcode for indexed vgpr readsJay Foad
Introduce V_MOV_B32_indirect_read for indexed vgpr reads (and rename the old V_MOV_B32_indirect to V_MOV_B32_indirect_write) so they can be unambiguously distinguished from regular V_MOV_B32_e32. Previously they were distinguished by looking for extra implicit operands but this is fragile because regular moves sometimes have extra implicit operands too: - either by accident, when instructions end up with duplicate implicit operands (see e.g. D100939) - or by design, when SIInstrInfo::copyPhysReg breaks a multi-dword copy into individual subreg mov instructions and adds implicit operands for the super-register. The effect of this is that SIInstrInfo::isFoldableCopy can be simplified and identifies more foldable copies. The test diffs show that more immediate 0 values have been folded as inline operands. SIInstrInfo::isReallyTriviallyReMaterializable could probably be simplified too but that is not part of this patch. Differential Revision: https://reviews.llvm.org/D114230
2021-09-14[AMDGPU] Switch PostRA sched to MachineSchedJoe Nash
Use GCNHazardRecognizer in postra sched. Updated tests for the new schedules. Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D109536 Change-Id: Ia86ba2ae168f12fb34b4d8efdab491f84d936cde
2021-09-07[amdgpu] Enable selection of `s_cselect_b64`.Michael Liao
Differential Revision: https://reviews.llvm.org/D109159
2021-08-25[AMDGPU] Divergence-driven compare operations instruction selectionalex-t
Description: This change enables the compare operations to be selected to SALU/VALU form dependent of the SDNode divergence flag. Reviewed By: rampitec Differential Revision: https://reviews.llvm.org/D106079
2021-04-26[AMDGPU] Experiments show that the GCNRegBankReassign pass significantly impactsBaptiste Saleil
the compilation time and there is no case for which we see any improvement in performance. This patch removes this pass and its associated test cases from the tree. Differential Revision: https://reviews.llvm.org/D101313 Change-Id: I0599169a7609c19a887f8d847a71e664030cc141
2021-04-01[AMDGPU][MC][GFX10][GFX90A] Corrected _e32/_e64 sufficesDmitry Preobrazhensky
Fixed bugs https://bugs.llvm.org//show_bug.cgi?id=49643, https://bugs.llvm.org//show_bug.cgi?id=49644, https://bugs.llvm.org//show_bug.cgi?id=49645. Differential Revision: https://reviews.llvm.org/D99413
2021-03-29[AMDGPU] Extend gfx10 test coverage. NFC.Petar Avramovic
Differential Revision: https://reviews.llvm.org/D99267
generated by cgit v1.2.3 (git 2.25.1) at 2026-10-22 13:46:57 +0000