summaryrefslogtreecommitdiff
path: root/llvm/lib/CodeGen/MachineInstr.cpp
AgeCommit message (Collapse)Author
2021-02-10[CSSPGO] Unblock optimizations with pseudo probe instrumentation.Hongtao Yu
The IR/MIR pseudo probe intrinsics don't get materialized into real machine instructions and therefore they don't incur runtime cost directly. However, they come with indirect cost by blocking certain optimizations. Some of the blocking are intentional (such as blocking code merge) for better counts quality while the others are accidental. This change unblocks perf-critical optimizations that do not affect counts quality. They include: 1. IR InstCombine, sinking load operation to shorten lifetimes. 2. MIR LiveRangeShrink, similar to #1 3. MIR TwoAddressInstructionPass, i.e, opeq transform 4. MIR function argument copy elision 5. IR stack protection. (though not perf-critical but nice to have). Reviewed By: wmi Differential Revision: https://reviews.llvm.org/D95982
2021-01-25[XRay] Support DW_TAG_call_site and delete unneeded ↵Fangrui Song
PATCHABLE_EVENT_CALL/PATCHABLE_TYPED_EVENT_CALL lowering
2021-01-06[NFC] Don't copy MachineFrameInfo on each invocation of HasAliasSanjoy Das
Also fix a typo in a comment. This fixes a compile time issue in XLA (https://www.tensorflow.org/xla). Differential Revision: https://reviews.llvm.org/D94182
2020-12-09Prevent FENTRY_CALL reorderingIlya Leoshkevich
FEntryInserter prepends FENTRY_CALL to the first basic block. In case there are other instructions, PostRA Machine Instruction Scheduler can move FENTRY_CALL call around. This actually occurs on SystemZ (see the testcase). This is bad for the following reasons: * FENTRY_CALL clobbers registers. * Linux Kernel depends on whatever FENTRY_CALL expands to to be the very first instruction in the function. Fix by adding isCall attribute to FENTRY_CALL, which prevents reordering by making it a scheduling boundary for PostRA Machine Instruction Scheduler. Reviewed By: niravd Differential Revision: https://reviews.llvm.org/D91218
2020-11-03[MachineInstr] Add support for instructions with multiple memory operands.Michael Liao
- Basically iterate each pair of memory operands from both instructions and return true if any of them may alias. - The exception are memory instructions without any memory operand. They may touch everything and could alias to any memory instruction. Differential Revision: https://reviews.llvm.org/D89447
2020-10-15[Statepoints] Unlimited tied operands.Denis Antrushin
Current limit on amount of tied operands (15) sometimes is too low for statepoint. We may get couple dozens of gc pointer operands on statepoint. Review D87154 changed format of statepoint to list every gc pointer only once, which makes it trivial to find tiedness relation between statepoint operands: defs are mapped 1-1 to gc pointer operands passed on registers. Reviewed By: skatkov Differential Revision: https://reviews.llvm.org/D87915
2020-10-07[MachineInstr] exclude call instruction in mayAliasChen Zheng
we now get noAlias result for a call instruction and other load/store/call instructions if we query mayAlias. This is not right as call instruction is not with mayloadorstore, but it may alter the memory. This patch fixes this wrong alias query. Differential Revision: https://reviews.llvm.org/D87490
2020-09-14[DebugInstrRef][1/9] Add fields for instr-ref variable locationsJeremy Morse
Add a DBG_INSTR_REF instruction and a "debug instruction number" field to MachineInstr. The two allow variable values to be specified by identifying where the value is computed, rather than the register it lies in, like so: %0 = fooinst, debug-instr-number 1 [...] DBG_INSTR_REF 1, 0 See the original RFC for motivation: http://lists.llvm.org/pipermail/llvm-dev/2020-February/139440.html This patch is NFCI; it only adds fields and other boiler plate. Differential Revision: https://reviews.llvm.org/D85741
2020-08-20Fix a couple of typos. NFCJon Roelofs
2020-07-17[ScheduleDAG] Move DBG_VALUEs after first term forward.Florian Hahn
MBBs are not allowed to have non-terminator instructions after the first terminator. Currently in some cases (see the modified test), EmitSchedule can add DBG_VALUEs after the last terminator, for example when referring a debug value that gets folded into a TCRETURN instruction on ARM. This patch updates EmitSchedule to move inserted DBG_VALUEs just before the first terminator. I am not sure if there are terminators produce values that can in turn be used by a DBG_VALUE. In that case, moving the DBG_VALUE might result in referencing an undefined register. But in any case, it seems like currently there is no way to insert a proper DBG_VALUEs for such registers anyways. Alternatively it might make sense to just remove those extra DBG_VALUES. I am not too familiar with the details of debug info in the backend and would appreciate any suggestions on how to address the issue in the best possible way. Reviewers: vsk, aprantl, jpaquette, efriedma, paquette Reviewed By: aprantl Differential Revision: https://reviews.llvm.org/D83561
2020-06-22[DebugInfo] Update MachineInstr to help support variadic DBG_VALUE instructionsstozer
Following on from this RFC[0] from a while back, this is the first patch towards implementing variadic debug values. This patch specifically adds a set of functions to MachineInstr for performing operations specific to debug values, and replacing uses of the more general functions where appropriate. The most prevalent of these is replacing getOperand(0) with getDebugOperand(0) for debug-value-specific code, as the operands corresponding to values will no longer be at index 0, but index 2 and upwards: getDebugOperand(x) == getOperand(x+2). Similar replacements have been added for the other operands, along with some helper functions to replace oft-repeated code and operate on a variable number of value operands. [0] http://lists.llvm.org/pipermail/llvm-dev/2020-February/139376.html<Paste> Differential Revision: https://reviews.llvm.org/D81852
2020-05-29Add NoMerge MIFlag to avoid MIR branch foldingZequan Wu
Let the codegen recognized the nomerge attribute and disable branch folding when the attribute is given Differential Revision: https://reviews.llvm.org/D79537
2020-05-22Revert "[CodeGen] Add support for multiple memory operands in ↵Jean-Michel Gorius
MachineInstr::mayAlias" This temporarily reverts commit 7019cea26dfef5882c96f278c32d0f9c49a5e516. It seems that, for some targets, there are instructions with a lot of memory operands (probably more than would be expected). This causes a lot of buildbots to timeout and notify failed builds. While investigations are ongoing to find out why this happens, revert the changes.
2020-05-21[CodeGen] Add support for multiple memory operands in MachineInstr::mayAliasJean-Michel Gorius
Summary: To support all targets, the mayAlias member function needs to support instructions with multiple operands. This revision also changes the order of the emitted instructions in some test cases. Reviewers: efriedma, hfinkel, craig.topper, dmgreen Reviewed By: efriedma Subscribers: MatzeB, dmgreen, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D80161
2020-04-24[llvm][CodeGen] Check for memory instructions when querying for alias statusJean-Michel Gorius
Summary: Add a check to make sure that MachineInstr::mayAlias returns prematurely if at least one of its instruction parameters does not access memory. This prevents calls to TargetInstrInfo::areMemAccessesTriviallyDisjoint with incompatible instructions. A side effect of this change is to render the mayAlias helper in the AArch64 load/store optimizer obsolete. We can now directly call the MachineInstr::mayAlias member function. Reviewers: hfinkel, t.p.northover, mcrosier, eli.friedman, efriedma Reviewed By: efriedma Subscribers: efriedma, kristof.beyls, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D78823
2020-04-16[MIR] Add comments to INLINEASM immediate flag MachineOperandsKonstantin Schwarz
Summary: The INLINEASM MIR instructions use immediate operands to encode the values of some operands. The MachineInstr pretty printer function already handles those operands and prints human readable annotations instead of the immediates. This patch adds similar annotations to the output of the MIRPrinter, however uses the new MIROperandComment feature. Reviewers: SjoerdMeijer, arsenm, efriedma Reviewed By: arsenm Subscribers: qcolombet, sdardis, jvesely, wdng, nhaehnle, hiraditya, jrtc27, atanasyan, kerbowa, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D78088
2020-02-29[MachineInst] Remove dead code. NFCI.Simon Pilgrim
The MachineFunction MF value is not used any more and is always null.
2020-02-29Make argument const to silence cppcheck warning. NFCI.Simon Pilgrim
2020-02-27[CallSiteInfo] Handle bundles when updating call site infoDjordje Todorovic
This will address the issue: P8198 and P8199 (from D73534). The methods was not handle bundles properly. Differential Revision: https://reviews.llvm.org/D74904
2020-02-25[MachineInstr] Add a dumpr methodQuentin Colombet
Add a dump method that recursively prints an instruction and all the instructions defining its operands and so on. This is helpful when looking at combiner issue. NFC Differential Revision: https://reviews.llvm.org/D75094
2020-02-07[MachineInstr] Add isCandidateForCallSiteEntry predicateVedant Kumar
Add the isCandidateForCallSiteEntry predicate to MachineInstr to determine whether a DWARF call site entry should be created for an instruction. For now, it's enough to have any call instruction that doesn't belong to a blacklisted set of opcodes. For these opcodes, a call site entry isn't meaningful. Differential Revision: https://reviews.llvm.org/D74159
2020-01-11moveOperands - assert Src/Dst MachineOperands are non-null.Simon Pilgrim
Fixes static-analyzer warnings.
2020-01-10[FPEnv] Invert sense of MIFlag::FPExcept flagUlrich Weigand
In D71841 we inverted the sense of the SDNode-level flag to ensure all nodes default to potentially raising FP exceptions unless otherwise specified -- i.e. if we forget to propagate the flag somewhere, the effect is now only lost performance, not incorrect code. However, the related flag at the MI level still defaults to nodes not raising FP exceptions unless otherwise specified. To be fully on the (conservatively) safe side, we should invert that flag as well. This patch does so by replacing MIFlag::FPExcept with MIFlag::NoFPExcept. (Note that this does also introduce an incompatible change in the MIR format.) Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D72466
2020-01-10[MIR] Fix cyclic dependency of MIR formatterPeng Guo
Summary: Move MIR formatter pointer from TargetMachine to TargetInstrInfo to avoid cyclic dependency between target & codegen. Reviewers: dsanders, bkramer, arsenm Subscribers: wdng, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D72485
2020-01-08Revert "Revert "[MIR] Target specific MIR formating and parsing""Daniel Sanders
There was an unguarded dereference of MF in a function that permitted nullptr. Fixed This reverts commit 71d64f72f934631aa2f12b9542c23f74f256f494.
2020-01-08Revert "[MIR] Target specific MIR formating and parsing"Nico Weber
This reverts commit 3ef05d85be8c3666ebfa3ad986eb334da5195a47. It broke check-llvm on many bots, see comments on D69836.
2020-01-08[MIR] Target specific MIR formating and parsingPeng Guo
Summary: Added MIRFormatter for target specific MIR formating and parsing with immediate and custom pseudo source values. Target machine can subclass MIRFormatter and implement custom logic for printing and parsing immediate and custom pseudo source values for better readability. * Target specific immediate mnemonic need to start with "." follows by identifier string. When MIR parser sees immediate it will call target specific parsing function. * Custom pseudo source value need to start with custom follows by double-quoted string. MIR parser will pass the quoted string to target specific PSV parsing function. * MIRFormatter have 2 helper functions to facilitate LLVM value printing and parsing for custom PSV if they refers LLVM values. Patch by Peng Guo Reviewers: dsanders, arsenm Reviewed By: dsanders Subscribers: wdng, jvesely, nhaehnle, hiraditya, jfb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69836
2020-01-08Revert "[MIR] Target specific MIR formating and parsing"Daniel Sanders
Forgot to credit Peng in the commit message. This reverts commit be841f89d0014b1e0246a4feae941b2f74abd908.
2020-01-08[MIR] Target specific MIR formating and parsingPeng Guo
Summary: Added MIRFormatter for target specific MIR formating and parsing with immediate and custom pseudo source values. Target machine can subclass MIRFormatter and implement custom logic for printing and parsing immediate and custom pseudo source values for better readability. * Target specific immediate mnemonic need to start with "." follows by identifier string. When MIR parser sees immediate it will call target specific parsing function. * Custom pseudo source value need to start with custom follows by double-quoted string. MIR parser will pass the quoted string to target specific PSV parsing function. * MIRFormatter have 2 helper functions to facilitate LLVM value printing and parsing for custom PSV if they refers LLVM values. Reviewers: dsanders, arsenm Reviewed By: dsanders Subscribers: wdng, jvesely, nhaehnle, hiraditya, jfb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69836
2019-11-15[CodeGen] Increase the size of a SmallVectorJay Foad
The SmallVector reserve() call in MachineInstrExpressionTrait::getHashValue accounted for over 3% of all calls to malloc() when I compiled a bunch of graphics shaders for the AMDGPU target. Its initial size was only enough for machine instructions with up to 7 operands, but for AMDGPU 8 and 10 operands are very common. Here's a histogram of number of operands for each call to getHashValue, gathered from the same collection of shaders: 1 13503 2 254273 3 135781 4 422508 5 614997 6 194953 7 287248 8 1517255 9 31218 10 1191269 11 70731 12 24 13 77 15 84 17 4692 27 16 33 705 49 6 Typical instructions with 8 and 10 operands are floating point arithmetic and multiply-accumulate instructions like: %83:vgpr_32 = V_MUL_F32_e64 0, killed %82:vgpr_32, 0, killed %81:vgpr_32, 0, 0, implicit $exec %330:vgpr_32 = V_MAC_F32_e64 0, killed %327:vgpr_32, 0, killed %329:sgpr_32, 0, %328:vgpr_32(tied-def 0), 0, 0, implicit $exec Differential Revision: https://reviews.llvm.org/D70301
2019-11-05[MIR] Add MIR parsing for heap alloc site instruction markersAmy Huang
Summary: This patch adds MIR parsing and printing for heap alloc markers, which were added in D69136. They are printed as an operand similar to pre-/post-instr symbols, with a heap-alloc-marker token and a metadata node. Reviewers: rnk Subscribers: hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69864
2019-10-29Fix unused variable warning. NFCI.Simon Pilgrim
2019-10-28Recommit "Add a heap alloc site marker field to the ExtraInfo in MachineInstrs"Amy Huang
Summary: Fixes some things from original commit at https://reviews.llvm.org/D69136. The main change is that the heap alloc marker is always stored as ExtraInfo in the machine instruction instead of in the PointerSumType because it cannot hold more than 4 pointer types. Add instruction marker to MachineInstr ExtraInfo. This does almost the same thing as Pre/PostInstrSymbols, except that it doesn't create a label until printing instructions. This allows for labels to be put around instructions that are deleted/duplicated somewhere. Use this marker to track heap alloc site call instructions. Reviewers: rnk Subscribers: MatzeB, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69536
2019-10-25Revert "Add an instruction marker field to the ExtraInfo in MachineInstrs."Amy Huang
Reverting commit b85b4e5a6f8579c137fecb59a4d75d7bfb111f79 due to some buildbot failures/ out of memory errors.
2019-10-25Add an instruction marker field to the ExtraInfo in MachineInstrs.Amy Huang
Summary: Add instruction marker to MachineInstr ExtraInfo. This does almost the same thing as Pre/PostInstrSymbols, except that it doesn't create a label until printing instructions. This allows for labels to be put around instructions that are deleted/duplicated somewhere. Also undo the workaround in r375137. Reviewers: rnk Subscribers: MatzeB, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D69136
2019-10-19Prune two MachineInstr.h includes, fix up depsReid Kleckner
MachineInstr.h included AliasAnalysis.h, which includes a world of IR constructs mostly unneeded in CodeGen. Prune it. Same for DebugInfoMetadata.h. Noticed with -ftime-trace. llvm-svn: 375311
2019-09-26Remove the AliasAnalysis argument in function areMemAccessesTriviallyDisjointChangpeng Fang
Reviewers: arsenm Differential Revision: https://reviews.llvm.org/D58360 llvm-svn: 373024
2019-09-02[DebugInfo] LiveDebugValues: correctly discriminate kinds of variable locationsJeremy Morse
The missing line added by this patch ensures that only spilt variable locations are candidates for being restored from the stack. Otherwise, register or constant-value information can be interpreted as a spill location, through a union. The added regression test replicates a scenario where this occurs: the stack load from [rsp] causes the register-location DBG_VALUE to be "restored" to rsi, when it should be left alone. See PR43058 for details. Un x-fail a test that was suffering from this from a previous patch. Differential Revision: https://reviews.llvm.org/D66895 llvm-svn: 370648
2019-08-15[llvm] Migrate llvm::make_unique to std::make_uniqueJonas Devlieghere
Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-06CodeGen: Migration to using RegisterMatt Arsenault
llvm-svn: 367974
2019-08-01Finish moving TargetRegisterInfo::isVirtualRegister() and friends to ↵Daniel Sanders
llvm::Register as started by r367614. NFC llvm-svn: 367633
2019-06-05Allow target to handle STRICT floating-point nodesUlrich Weigand
The ISD::STRICT_ nodes used to implement the constrained floating-point intrinsics are currently never passed to the target back-end, which makes it impossible to handle them correctly (e.g. mark instructions are depending on a floating-point status and control register, or mark instructions as possibly trapping). This patch allows the target to use setOperationAction to switch the action on ISD::STRICT_ nodes to Legal. If this is done, the SelectionDAG common code will stop converting the STRICT nodes to regular floating-point nodes, but instead pass the STRICT nodes to the target using normal SelectionDAG matching rules. To avoid having the back-end duplicate all the floating-point instruction patterns to handle both strict and non-strict variants, we make the MI codegen explicitly aware of the floating-point exceptions by introducing two new concepts: - A new MCID flag "mayRaiseFPException" that the target should set on any instruction that possibly can raise FP exception according to the architecture definition. - A new MI flag FPExcept that CodeGen/SelectionDAG will set on any MI instruction resulting from expansion of any constrained FP intrinsic. Any MI instruction that is *both* marked as mayRaiseFPException *and* FPExcept then needs to be considered as raising exceptions by MI-level codegen (e.g. scheduling). Setting those two new flags is straightforward. The mayRaiseFPException flag is simply set via TableGen by marking all relevant instruction patterns in the .td files. The FPExcept flag is set in SDNodeFlags when creating the STRICT_ nodes in the SelectionDAG, and gets inherited in the MachineSDNode nodes created from it during instruction selection. The flag is then transfered to an MIFlag when creating the MI from the MachineSDNode. This is handled just like fast-math flags like no-nans are handled today. This patch includes both common code changes required to implement the new features, and the SystemZ implementation. Reviewed By: andrew.w.kaylor Differential Revision: https://reviews.llvm.org/D55506 llvm-svn: 362663
2019-06-02[X86] Fix several places that weren't passing what they though they were to ↵Craig Topper
MachineInstr::print Over a year ago, MachineInstr gained a fourth boolean parameter that occurs before the TII pointer. When this happened, several places started accidentally passing TII into this boolean parameter instead of the TII parameter. llvm-svn: 362312
2019-05-20[DebugInfoMetadata] Refactor DIExpression::prepend constants (NFC)Petar Jovanovic
Refactor DIExpression::With* into a flag enum in order to be less error-prone to use (as discussed on D60866). Patch by Djordje Todorovic. Differential Revision: https://reviews.llvm.org/D61943 llvm-svn: 361137
2019-04-24Recommitting r358783 and r358786 "[MS] Emit S_HEAPALLOCSITE debug info" with ↵Amy Huang
fixes for buildbot error (undefined assembler label). Summary: This emits labels around heapallocsite calls and S_HEAPALLOCSITE debug info in codeview. Currently only changes FastISel, so emitting labels still needs to be implemented in SelectionDAG. Reviewers: rnk Subscribers: aprantl, hiraditya, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D61083 llvm-svn: 359149
2019-04-19[CodeGen] Add "const" to MachineInstr::mayAliasBjorn Pettersson
Summary: The basic idea here is to make it possible to use MachineInstr::mayAlias also when the MachineInstr is const (or the "Other" MachineInstr is const). The addition of const in MachineInstr::mayAlias then rippled down to the need for adding const in several other places, such as TargetTransformInfo::getMemOperandWithOffset. Reviewers: hfinkel Reviewed By: hfinkel Subscribers: hfinkel, MatzeB, arsenm, jvesely, nhaehnle, hiraditya, javed.absar, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60856 llvm-svn: 358744
2019-03-19Allow unordered loads to be considered invariant in CodeGenPhilip Reames
The actual code change is fairly straight forward, but exercising it isn't. First, it turned out we weren't adding the appropriate flags in SelectionDAG. Second, it turned out that we've got some optimization gaps, so obvious test cases don't work. My first attempt (in atomic-unordered.ll) points out a deficiency in our peephole-opt folding logic which I plan to fix separately. Instead, I'm exercising this through MachineLICM. Differential Revision: https://reviews.llvm.org/D59375 llvm-svn: 356494
2019-03-14Allow code motion (and thus folding) for atomic (but unordered) memory operandsPhilip Reames
Building on the work done in D57601, now that we can distinguish between atomic and volatile memory accesses, go ahead and allow code motion of unordered atomics. As seen in the diffs, this allows much better folding of memory operations into using instructions. (Mostly done by the PeepholeOpt pass.) Note: I have not reviewed all callers of hasOrderedMemoryRef since one of them - isSafeToMove - is very widely used. I'm relying on the documented semantics of each method to judge correctness. Differential Revision: https://reviews.llvm.org/D59345 llvm-svn: 356170
2019-02-20[NFC] add/modify wrapper function for findRegisterDefOperand().Chen Zheng
llvm-svn: 354438
2019-02-11Be conservative about unordered accesses for the momentPhilip Reames
Background: As described in https://reviews.llvm.org/D57601, I'm working towards separating volatile and atomic in the MMO uses for atomic instructions. In https://reviews.llvm.org/D57593, I fixed a bug where isUnordered was returning the wrong result, but didn't account for the fact I was getting slightly ahead of myself. While both uses of isUnordered are correct (as far as I can tell), we don't have tests to demonstrate this and being aggressive gets in the way of having the removal of volatile truly be non-functional. Once D57601 lands, I will return to these call sites, revert this patch, and add the appropriate tests to show the expected behaviour. Differential Revision: https://reviews.llvm.org/D57802 llvm-svn: 353766
generated by cgit v1.2.3 (git 2.25.1) at 2026-06-18 09:16:41 +0000