RPCS3/llvm-mirror
1
0
Fork 0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-10-21 12:02:58 +02:00
Code Issues Projects Releases Wiki Activity
8c57546852
llvm-mirror/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

3816 lines
148 KiB
C++
Raw Normal View History

Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
//===-- SelectionDAGISel.cpp - Implement the SelectionDAGISel class -------===//
Remove trailing whitespace llvm-svn: 21420
2005-04-22 00:36:52 +02:00
//
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
// The LLVM Compiler Infrastructure
//
Remove attribution from file headers, per discussion on llvmdev. llvm-svn: 45418
2007-12-29 21:36:04 +01:00
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
Remove trailing whitespace llvm-svn: 21420
2005-04-22 00:36:52 +02:00
//
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
//===----------------------------------------------------------------------===//
//
// This implements the SelectionDAGISel class.
//
//===----------------------------------------------------------------------===//
Move ScheduleDAGSDNodes.h to be a private header. Front-ends that previously included this header should include SchedulerRegistry.h instead. llvm-svn: 63937
2009-02-06 18:22:58 +01:00
#include "ScheduleDAGSDNodes.h"
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
#include "SelectionDAGBuilder.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/ADT/APInt.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/None.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-03 17:50:05 +01:00
#include "llvm/ADT/PostOrderIterator.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/SmallSet.h"
#include "llvm/ADT/SmallVector.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-03 17:50:05 +01:00
#include "llvm/ADT/Statistic.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/StringRef.h"
Move lib/Analysis/DebugInfo.cpp to lib/VMCore/DebugInfo.cpp and include/llvm/Analysis/DebugInfo.h to include/llvm/DebugInfo.h. The reasoning is because the DebugInfo module is simply an interface to the debug info MDNodes and has nothing to do with analysis. llvm-svn: 159312
2012-06-28 02:05:13 +02:00
#include "llvm/Analysis/AliasAnalysis.h"
#include "llvm/Analysis/BranchProbabilityInfo.h"
Reimplement isPotentiallyReachable to make nocapture deduction much stronger. Adds unit tests for it too. Split BasicBlockUtils into an analysis-half and a transforms-half, and put the analysis bits into a new Analysis/CFG.{h,cpp}. Promote isPotentiallyReachable into llvm::isPotentiallyReachable and move it into Analysis/CFG. llvm-svn: 187283
2013-07-27 03:24:00 +02:00
#include "llvm/Analysis/CFG.h"
[PM] Move TargetLibraryInfo into the Analysis library. While the term "Target" is in the name, it doesn't really have to do with the LLVM Target library -- this isn't an abstraction which LLVM targets generally need to implement or extend. It has much more to do with modeling the various runtime libraries on different OSes and with different runtime environments. The "target" in this sense is the more general sense of a target of cross compilation. This is in preparation for porting this analysis to the new pass manager. No functionality changed, and updates inbound for Clang and Polly. llvm-svn: 226078
2015-01-15 03:16:27 +01:00
#include "llvm/Analysis/TargetLibraryInfo.h"
Add code to call FastISel, and a command-line option to enable it. llvm-svn: 55015
2008-08-20 00:33:34 +02:00
#include "llvm/CodeGen/FastISel.h"
Move lib/Analysis/DebugInfo.cpp to lib/VMCore/DebugInfo.cpp and include/llvm/Analysis/DebugInfo.h to include/llvm/DebugInfo.h. The reasoning is because the DebugInfo module is simply an interface to the debug info MDNodes and has nothing to do with analysis. llvm-svn: 159312
2012-06-28 02:05:13 +02:00
#include "llvm/CodeGen/FunctionLoweringInfo.h"
Rename some GC classes so that their roll will hopefully be clearer. In particular, Collector was confusing to implementors. Several thought that this compile-time class was the place to implement their runtime GC heap. Of course, it doesn't even exist at runtime. Specifically, the renames are: Collector -> GCStrategy CollectorMetadata -> GCFunctionInfo CollectorModuleMetadata -> GCModuleInfo CollectorRegistry -> GCRegistry Function::getCollector -> getGC (setGC, hasGC, clearGC) Several accessors and nested types have also been renamed to be consistent. These changes should be obvious. llvm-svn: 54899
2008-08-17 20:44:35 +02:00
#include "llvm/CodeGen/GCMetadata.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/CodeGen/MachineBasicBlock.h"
Rename "HasCalls" in MachineFrameInfo to "AdjustsStack" to better describe what the variable actually tracks. N.B., several back-ends are using "HasCalls" as being synonymous for something that adjusts the stack. This isn't 100% correct and should be looked into. llvm-svn: 103802
2010-05-14 23:14:32 +02:00
#include "llvm/CodeGen/MachineFrameInfo.h"
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
#include "llvm/CodeGen/MachineFunction.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineInstr.h"
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
#include "llvm/CodeGen/MachineInstrBuilder.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/CodeGen/MachineMemOperand.h"
#include "llvm/CodeGen/MachineOperand.h"
#include "llvm/CodeGen/MachinePassRegistry.h"
Rename SSARegMap -> MachineRegisterInfo in keeping with the idea that "machine" classes are used to represent the current state of the code being compiled. Given this expanded name, we can start moving other stuff into it. For now, move the UsedPhysRegs and LiveIn/LoveOuts vectors from MachineFunction into it. Update all the clients to match. This also reduces some needless #includes, such as MachineModuleInfo from MachineFunction. llvm-svn: 45467
2007-12-31 05:13:23 +01:00
#include "llvm/CodeGen/MachineRegisterInfo.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/CodeGen/MachineValueType.h"
Final polish on machine pass registries. llvm-svn: 29471
2006-08-02 14:30:23 +02:00
#include "llvm/CodeGen/SchedulerRegistry.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/CodeGen/SelectionDAG.h"
Re-sort #include lines using my handy dandy ./utils/sort_includes.py script. This is in preparation for changes to lots of include lines. llvm-svn: 229088
2015-02-13 10:09:03 +01:00
#include "llvm/CodeGen/SelectionDAGISel.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/CodeGen/SelectionDAGNodes.h"
[SSP] Remove llvm.stackprotectorcheck. This is a cleanup patch for SSP support in LLVM. There is no functional change. llvm.stackprotectorcheck is not needed, because SelectionDAG isn't actually lowering it in SelectBasicBlock; rather, it adds check code in FinishBasicBlock, ignoring the position where the intrinsic is inserted (See FindSplitPointForStackProtector()). llvm-svn: 265851
2016-04-08 23:26:31 +02:00
#include "llvm/CodeGen/StackProtector.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/CodeGen/ValueTypes.h"
#include "llvm/IR/BasicBlock.h"
Move all of the header files which are involved in modelling the LLVM IR into their new header subdirectory: include/llvm/IR. This matches the directory structure of lib, and begins to correct a long standing point of file layout clutter in LLVM. There are still more header files to move here, but I wanted to handle them in separate commits to make tracking what files make sense at each layer easier. The only really questionable files here are the target intrinsic tablegen files. But that's a battle I'd rather not fight today. I've updated both CMake and Makefile build systems (I think, and my tests think, but I may have missed something). I've also re-sorted the includes throughout the project. I'll be committing updates to Clang, DragonEgg, and Polly momentarily. llvm-svn: 171366
2013-01-02 12:36:10 +01:00
#include "llvm/IR/Constants.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/IR/DebugInfoMetadata.h"
#include "llvm/IR/DebugLoc.h"
[FastISel] Add a diagnostic to warm on fallback. This is consistent with what we do for GlobalISel. That way, it is easy to see whether or not FastISel is able to fully select a function. At some point we may want to switch that to an optimization remark. llvm-svn: 294970
2017-02-13 18:38:59 +01:00
#include "llvm/IR/DiagnosticInfo.h"
Move all of the header files which are involved in modelling the LLVM IR into their new header subdirectory: include/llvm/IR. This matches the directory structure of lib, and begins to correct a long standing point of file layout clutter in LLVM. There are still more header files to move here, but I wanted to handle them in separate commits to make tracking what files make sense at each layer easier. The only really questionable files here are the target intrinsic tablegen files. But that's a battle I'd rather not fight today. I've updated both CMake and Makefile build systems (I think, and my tests think, but I may have missed something). I've also re-sorted the includes throughout the project. I'll be committing updates to Clang, DragonEgg, and Polly momentarily. llvm-svn: 171366
2013-01-02 12:36:10 +01:00
#include "llvm/IR/Function.h"
#include "llvm/IR/InlineAsm.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/IR/InstrTypes.h"
#include "llvm/IR/Instruction.h"
Move all of the header files which are involved in modelling the LLVM IR into their new header subdirectory: include/llvm/IR. This matches the directory structure of lib, and begins to correct a long standing point of file layout clutter in LLVM. There are still more header files to move here, but I wanted to handle them in separate commits to make tracking what files make sense at each layer easier. The only really questionable files here are the target intrinsic tablegen files. But that's a battle I'd rather not fight today. I've updated both CMake and Makefile build systems (I think, and my tests think, but I may have missed something). I've also re-sorted the includes throughout the project. I'll be committing updates to Clang, DragonEgg, and Polly momentarily. llvm-svn: 171366
2013-01-02 12:36:10 +01:00
#include "llvm/IR/Instructions.h"
#include "llvm/IR/IntrinsicInst.h"
#include "llvm/IR/Intrinsics.h"
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include "llvm/IR/Metadata.h"
#include "llvm/IR/Type.h"
#include "llvm/IR/User.h"
#include "llvm/MC/MCInstrDesc.h"
#include "llvm/MC/MCRegisterInfo.h"
#include "llvm/Pass.h"
#include "llvm/Support/BranchProbability.h"
#include "llvm/Support/Casting.h"
#include "llvm/Support/CodeGen.h"
#include "llvm/Support/CommandLine.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-03 17:50:05 +01:00
#include "llvm/Support/Compiler.h"
#include "llvm/Support/Debug.h"
#include "llvm/Support/ErrorHandling.h"
#include "llvm/Support/Timer.h"
#include "llvm/Support/raw_ostream.h"
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
#include "llvm/Target/TargetInstrInfo.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-03 17:50:05 +01:00
#include "llvm/Target/TargetIntrinsicInfo.h"
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
#include "llvm/Target/TargetLowering.h"
#include "llvm/Target/TargetMachine.h"
Fix missing include llvm-svn: 28435
2006-05-23 15:43:15 +02:00
#include "llvm/Target/TargetOptions.h"
Use the new script to sort the includes of every file under lib. Sooooo many of these had incorrect or strange main module includes. I have manually inspected all of these, and fixed the main module include to be the nearest plausible thing I could find. If you own or care about any of these source files, I encourage you to take some time and check that these edits were sensible. I can't have broken anything (I strictly added headers, and reordered them, never removed), but they may not be the headers you'd really like to identify as containing the API being implemented. Many forward declarations and missing includes were added to a header files to allow them to parse cleanly when included first. The main module rule does in fact have its merits. =] llvm-svn: 169131
2012-12-03 17:50:05 +01:00
#include "llvm/Target/TargetRegisterInfo.h"
misched: Allow subtargets to enable misched and dependent options. This allows me to begin enabling (or backing out) misched by default for one subtarget at a time. To run misched we typically want to: - Disable SelectionDAG scheduling (use the source order scheduler) - Enable more aggressive coalescing (until we decide to always run the coalescer this way) - Enable MachineScheduler pass itself. Disabling PostRA sched may follow for some subtargets. llvm-svn: 167826
2012-11-13 09:47:29 +01:00
#include "llvm/Target/TargetSubtargetInfo.h"
fix PR8642: if a critical edge has a PHI value that can trap, isel is *required* to split the edge. PHI values get evaluated on the edge, not in their predecessor block. llvm-svn: 122170
2010-12-19 05:58:57 +01:00
#include "llvm/Transforms/Utils/BasicBlockUtils.h"
Get VC++ building again. llvm-svn: 26351
2006-02-24 03:52:40 +01:00
#include <algorithm>
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
#include <cassert>
#include <cstdint>
#include <iterator>
#include <memory>
#include <string>
#include <utility>
#include <vector>
Fix Clang-tidy readability-redundant-control-flow warnings; other minor fixes. Differential revision: http://reviews.llvm.org/D16793 llvm-svn: 259539
2016-02-02 19:20:45 +01:00
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
using namespace llvm;
[Modules] Remove potential ODR violations by sinking the DEBUG_TYPE define below all header includes in the lib/CodeGen/... tree. While the current modules implementation doesn't check for this kind of ODR violation yet, it is likely to grow support for it in the future. It also removes one layer of macro pollution across all the included headers. Other sub-trees will follow. llvm-svn: 206837
2014-04-22 04:02:50 +02:00
#define DEBUG_TYPE "isel"
Disable statistics on Release builds and move tests that depend on -stats. Summary: Statistics are still available in Release+Asserts (any +Asserts builds), and stats can also be turned on with LLVM_ENABLE_STATS. Move some of the FastISel stats that were moved under DEBUG() back out of DEBUG(), since stats are disabled across the board now. Many tests depend on grepping "-stats" output. Move those into a orig_dir/Stats/. so that they can be marked as unsupported when building without statistics. Differential Revision: http://llvm-reviews.chandlerc.com/D486 llvm-svn: 176733
2013-03-08 23:56:31 +01:00
STATISTIC(NumFastIselFailures, "Number of instructions fast isel failed on");
STATISTIC(NumFastIselSuccess, "Number of instructions fast isel selected");
Add counters to count basic blocks and machine basic blocks with out of order line number info. Add counters to count how many basic blocks are entirely selected by fastisel. llvm-svn: 117310
2010-10-25 22:55:43 +02:00
STATISTIC(NumFastIselBlocks, "Number of blocks selected entirely by fast isel");
STATISTIC(NumDAGBlocks, "Number of blocks selected using DAG");
Silence the unused variable warning. llvm-svn: 176218
2013-02-27 23:52:54 +01:00
STATISTIC(NumDAGIselRetries,"Number of times dag isel has to try another path");
Add some more stats for fast isel vs. SelectionDAG, w.r.t lowering function arguments in entry BBs. llvm-svn: 179824
2013-04-19 03:04:40 +02:00
STATISTIC(NumEntryBlocks, "Number of entry blocks encountered");
STATISTIC(NumFastIselFailLowerArguments,
"Number of entry blocks where fast isel failed to lower arguments");
Silence the unused variable warning. llvm-svn: 176218
2013-02-27 23:52:54 +01:00
Disable statistics on Release builds and move tests that depend on -stats. Summary: Statistics are still available in Release+Asserts (any +Asserts builds), and stats can also be turned on with LLVM_ENABLE_STATS. Move some of the FastISel stats that were moved under DEBUG() back out of DEBUG(), since stats are disabled across the board now. Many tests depend on grepping "-stats" output. Move those into a orig_dir/Stats/. so that they can be marked as unsupported when building without statistics. Differential Revision: http://llvm-reviews.chandlerc.com/D486 llvm-svn: 176733
2013-03-08 23:56:31 +01:00
#ifndef NDEBUG
[fast-isel] Guard "exhastive" fast-isel output with -fast-isel-verbose2. llvm-svn: 146453
2011-12-13 01:05:11 +01:00
static cl::opt<bool>
EnableFastISelVerbose2("fast-isel-verbose2", cl::Hidden,
cl::desc("Enable extra verbose messages in the \"fast\" "
"instruction selector"));
Add some more stats for fast isel vs. SelectionDAG, w.r.t lowering function arguments in entry BBs. llvm-svn: 179824
2013-04-19 03:04:40 +02:00
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
// Terminators
STATISTIC(NumFastIselFailRet,"Fast isel fails on Ret");
STATISTIC(NumFastIselFailBr,"Fast isel fails on Br");
STATISTIC(NumFastIselFailSwitch,"Fast isel fails on Switch");
STATISTIC(NumFastIselFailIndirectBr,"Fast isel fails on IndirectBr");
STATISTIC(NumFastIselFailInvoke,"Fast isel fails on Invoke");
STATISTIC(NumFastIselFailResume,"Fast isel fails on Resume");
STATISTIC(NumFastIselFailUnreachable,"Fast isel fails on Unreachable");
// Standard binary operators...
STATISTIC(NumFastIselFailAdd,"Fast isel fails on Add");
STATISTIC(NumFastIselFailFAdd,"Fast isel fails on FAdd");
STATISTIC(NumFastIselFailSub,"Fast isel fails on Sub");
STATISTIC(NumFastIselFailFSub,"Fast isel fails on FSub");
STATISTIC(NumFastIselFailMul,"Fast isel fails on Mul");
STATISTIC(NumFastIselFailFMul,"Fast isel fails on FMul");
STATISTIC(NumFastIselFailUDiv,"Fast isel fails on UDiv");
STATISTIC(NumFastIselFailSDiv,"Fast isel fails on SDiv");
STATISTIC(NumFastIselFailFDiv,"Fast isel fails on FDiv");
STATISTIC(NumFastIselFailURem,"Fast isel fails on URem");
STATISTIC(NumFastIselFailSRem,"Fast isel fails on SRem");
STATISTIC(NumFastIselFailFRem,"Fast isel fails on FRem");
// Logical operators...
STATISTIC(NumFastIselFailAnd,"Fast isel fails on And");
STATISTIC(NumFastIselFailOr,"Fast isel fails on Or");
STATISTIC(NumFastIselFailXor,"Fast isel fails on Xor");
// Memory instructions...
STATISTIC(NumFastIselFailAlloca,"Fast isel fails on Alloca");
STATISTIC(NumFastIselFailLoad,"Fast isel fails on Load");
STATISTIC(NumFastIselFailStore,"Fast isel fails on Store");
STATISTIC(NumFastIselFailAtomicCmpXchg,"Fast isel fails on AtomicCmpXchg");
STATISTIC(NumFastIselFailAtomicRMW,"Fast isel fails on AtomicRWM");
STATISTIC(NumFastIselFailFence,"Fast isel fails on Frence");
STATISTIC(NumFastIselFailGetElementPtr,"Fast isel fails on GetElementPtr");
// Convert instructions...
STATISTIC(NumFastIselFailTrunc,"Fast isel fails on Trunc");
STATISTIC(NumFastIselFailZExt,"Fast isel fails on ZExt");
STATISTIC(NumFastIselFailSExt,"Fast isel fails on SExt");
STATISTIC(NumFastIselFailFPTrunc,"Fast isel fails on FPTrunc");
STATISTIC(NumFastIselFailFPExt,"Fast isel fails on FPExt");
STATISTIC(NumFastIselFailFPToUI,"Fast isel fails on FPToUI");
STATISTIC(NumFastIselFailFPToSI,"Fast isel fails on FPToSI");
STATISTIC(NumFastIselFailUIToFP,"Fast isel fails on UIToFP");
STATISTIC(NumFastIselFailSIToFP,"Fast isel fails on SIToFP");
STATISTIC(NumFastIselFailIntToPtr,"Fast isel fails on IntToPtr");
STATISTIC(NumFastIselFailPtrToInt,"Fast isel fails on PtrToInt");
STATISTIC(NumFastIselFailBitCast,"Fast isel fails on BitCast");
// Other instructions...
STATISTIC(NumFastIselFailICmp,"Fast isel fails on ICmp");
STATISTIC(NumFastIselFailFCmp,"Fast isel fails on FCmp");
STATISTIC(NumFastIselFailPHI,"Fast isel fails on PHI");
STATISTIC(NumFastIselFailSelect,"Fast isel fails on Select");
STATISTIC(NumFastIselFailCall,"Fast isel fails on Call");
STATISTIC(NumFastIselFailShl,"Fast isel fails on Shl");
STATISTIC(NumFastIselFailLShr,"Fast isel fails on LShr");
STATISTIC(NumFastIselFailAShr,"Fast isel fails on AShr");
STATISTIC(NumFastIselFailVAArg,"Fast isel fails on VAArg");
STATISTIC(NumFastIselFailExtractElement,"Fast isel fails on ExtractElement");
STATISTIC(NumFastIselFailInsertElement,"Fast isel fails on InsertElement");
STATISTIC(NumFastIselFailShuffleVector,"Fast isel fails on ShuffleVector");
STATISTIC(NumFastIselFailExtractValue,"Fast isel fails on ExtractValue");
STATISTIC(NumFastIselFailInsertValue,"Fast isel fails on InsertValue");
STATISTIC(NumFastIselFailLandingPad,"Fast isel fails on LandingPad");
[FastISel] Collect statistics about failing intrinsic calls. Add more instruction-specific statistics about failing intrinsic calls during FastISel. llvm-svn: 210556
2014-06-10 20:17:00 +02:00
// Intrinsic instructions...
STATISTIC(NumFastIselFailIntrinsicCall, "Fast isel fails on Intrinsic call");
STATISTIC(NumFastIselFailSAddWithOverflow,
"Fast isel fails on sadd.with.overflow");
STATISTIC(NumFastIselFailUAddWithOverflow,
"Fast isel fails on uadd.with.overflow");
STATISTIC(NumFastIselFailSSubWithOverflow,
"Fast isel fails on ssub.with.overflow");
STATISTIC(NumFastIselFailUSubWithOverflow,
"Fast isel fails on usub.with.overflow");
STATISTIC(NumFastIselFailSMulWithOverflow,
"Fast isel fails on smul.with.overflow");
STATISTIC(NumFastIselFailUMulWithOverflow,
"Fast isel fails on umul.with.overflow");
STATISTIC(NumFastIselFailFrameaddress, "Fast isel fails on Frameaddress");
STATISTIC(NumFastIselFailSqrt, "Fast isel fails on sqrt call");
STATISTIC(NumFastIselFailStackMap, "Fast isel fails on StackMap call");
STATISTIC(NumFastIselFailPatchPoint, "Fast isel fails on PatchPoint call");
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
#endif
Add code to call FastISel, and a command-line option to enable it. llvm-svn: 55015
2008-08-20 00:33:34 +02:00
static cl::opt<bool>
Add a new option, -fast-isel-verbose, that can be used with -fast-isel-no-abort to get a dump of all unhandled instructions, without an abort. llvm-svn: 56021
2008-09-10 00:06:46 +02:00
EnableFastISelVerbose("fast-isel-verbose", cl::Hidden,
Fast-isel no longer an experiment. llvm-svn: 57845
2008-10-20 23:30:12 +02:00
cl::desc("Enable verbose messages in the \"fast\" "
Add a new option, -fast-isel-verbose, that can be used with -fast-isel-no-abort to get a dump of all unhandled instructions, without an abort. llvm-svn: 56021
2008-09-10 00:06:46 +02:00
"instruction selector"));
Change the fast-isel-abort option from bool to int to enable "levels" Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
2015-02-27 19:32:11 +01:00
static cl::opt<int> EnableFastISelAbort(
"fast-isel-abort", cl::Hidden,
cl::desc("Enable abort calls when \"fast\" instruction selection "
"fails to lower an instruction: 0 disable the abort, 1 will "
"abort but for args, calls and terminators, 2 will also "
"abort for argument lowering, and 3 will never fallback "
"to SelectionDAG."));
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
[FastISel] Add a diagnostic to warm on fallback. This is consistent with what we do for GlobalISel. That way, it is easy to see whether or not FastISel is able to fully select a function. At some point we may want to switch that to an optimization remark. llvm-svn: 294970
2017-02-13 18:38:59 +01:00
static cl::opt<bool> EnableFastISelFallbackReport(
"fast-isel-report-on-fallback", cl::Hidden,
cl::desc("Emit a diagnostic when \"fast\" instruction selection "
"falls back to SelectionDAG."));
Introduce MachineBranchProbabilityInfo class, which has similar API to BranchProbabilityInfo (expect setEdgeWeight which is not available here). Branch Weights are kept in MachineBasicBlocks. To turn off this analysis set -use-mbpi=false. llvm-svn: 133184
2011-06-16 22:22:37 +02:00
static cl::opt<bool>
UseMBPI("use-mbpi",
cl::desc("use Machine Branch Probability Info"),
cl::init(true), cl::Hidden);
It is NDEBUG not _NDEBUG llvm-svn: 23186
2005-09-01 20:44:10 +02:00
#ifndef NDEBUG
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
static cl::opt<std::string>
FilterDAGBasicBlockName("filter-view-dags", cl::Hidden,
cl::desc("Only display the basic block whose name "
"matches this for all view-*-dags options"));
Add an option to view the selection dags as they are generated. llvm-svn: 19498
2005-01-12 04:41:21 +01:00
static cl::opt<bool>
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
ViewDAGCombine1("view-dag-combine1-dags", cl::Hidden,
cl::desc("Pop up a window to show dags before the first "
"dag combine pass"));
static cl::opt<bool>
ViewLegalizeTypesDAGs("view-legalize-types-dags", cl::Hidden,
cl::desc("Pop up a window to show dags before legalize types"));
static cl::opt<bool>
ViewLegalizeDAGs("view-legalize-dags", cl::Hidden,
cl::desc("Pop up a window to show dags before legalize"));
static cl::opt<bool>
ViewDAGCombine2("view-dag-combine2-dags", cl::Hidden,
cl::desc("Pop up a window to show dags before the second "
"dag combine pass"));
static cl::opt<bool>
If the type legalizer actually legalized anything (this doesn't happen that often, since most code does not use illegal types) then follow it by a DAG combiner run that is allowed to generate illegal operations but not illegal types. I didn't modify the target combiner code to distinguish like this between illegal operations and illegal types, so it will not produce illegal operations as well as not producing illegal types. llvm-svn: 59960
2008-11-24 15:53:14 +01:00
ViewDAGCombineLT("view-dag-combine-lt-dags", cl::Hidden,
cl::desc("Pop up a window to show dags before the post legalize types"
" dag combine pass"));
static cl::opt<bool>
Do some code refactoring on Jim's scheduler in preparation of the new list scheduler. llvm-svn: 25493
2006-01-21 03:32:06 +01:00
ViewISelDAGs("view-isel-dags", cl::Hidden,
cl::desc("Pop up a window to show isel dags as they are selected"));
static cl::opt<bool>
ViewSchedDAGs("view-sched-dags", cl::Hidden,
cl::desc("Pop up a window to show sched dags as they are processed"));
Add an option, -view-sunit-dags, for viewing the actual SUnit DAGs used by scheduling. llvm-svn: 41556
2007-08-28 22:32:58 +02:00
static cl::opt<bool>
ViewSUnitDAGs("view-sunit-dags", cl::Hidden,
fix long lines. llvm-svn: 46355
2008-01-25 18:24:52 +01:00
cl::desc("Pop up a window to show SUnit dags after they are processed"));
Add an option to view the selection dags as they are generated. llvm-svn: 19498
2005-01-12 04:41:21 +01:00
#else
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
static const bool ViewDAGCombine1 = false,
ViewLegalizeTypesDAGs = false, ViewLegalizeDAGs = false,
ViewDAGCombine2 = false,
If the type legalizer actually legalized anything (this doesn't happen that often, since most code does not use illegal types) then follow it by a DAG combiner run that is allowed to generate illegal operations but not illegal types. I didn't modify the target combiner code to distinguish like this between illegal operations and illegal types, so it will not produce illegal operations as well as not producing illegal types. llvm-svn: 59960
2008-11-24 15:53:14 +01:00
ViewDAGCombineLT = false,
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
ViewISelDAGs = false, ViewSchedDAGs = false,
ViewSUnitDAGs = false;
Add an option to view the selection dags as they are generated. llvm-svn: 19498
2005-01-12 04:41:21 +01:00
#endif
Final polish on machine pass registries. llvm-svn: 29471
2006-08-02 14:30:23 +02:00
//===---------------------------------------------------------------------===//
///
/// RegisterScheduler class - Track the registration of instruction schedulers.
///
//===---------------------------------------------------------------------===//
MachinePassRegistry RegisterScheduler::Registry;
//===---------------------------------------------------------------------===//
///
/// ISHeuristic command line option for instruction schedulers.
///
//===---------------------------------------------------------------------===//
Clean up the use of static and anonymous namespaces. This turned up several things that were neither in an anonymous namespace nor static but not intended to be global. llvm-svn: 51017
2008-05-13 02:00:25 +02:00
static cl::opt<RegisterScheduler::FunctionPassCtor, false,
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
RegisterPassParser<RegisterScheduler>>
Clean up the use of static and anonymous namespaces. This turned up several things that were neither in an anonymous namespace nor static but not intended to be global. llvm-svn: 51017
2008-05-13 02:00:25 +02:00
ISHeuristic("pre-RA-sched",
Hide the pre-RA-sched= option. This is a very confusing option for a feature that will go away. -enable-misched is exposed instead to help triage issues with the new scheduler. llvm-svn: 199133
2014-01-13 21:08:27 +01:00
cl::init(&createDefaultScheduler), cl::Hidden,
Clean up the use of static and anonymous namespaces. This turned up several things that were neither in an anonymous namespace nor static but not intended to be global. llvm-svn: 51017
2008-05-13 02:00:25 +02:00
cl::desc("Instruction schedulers available (before register"
" allocation):"));
Introducing plugable register allocators and instruction schedulers. llvm-svn: 29434
2006-08-01 16:21:23 +02:00
Clean up the use of static and anonymous namespaces. This turned up several things that were neither in an anonymous namespace nor static but not intended to be global. llvm-svn: 51017
2008-05-13 02:00:25 +02:00
static RegisterScheduler
Fix command-line option printing to print two spaces where needed, instead of requiring all "short description" strings to begin with two spaces. This makes these strings less mysterious, and it fixes some cases where short description strings mistakenly did not begin with two spaces. llvm-svn: 57521
2008-10-14 22:25:08 +02:00
defaultListDAGScheduler("default", "Best scheduler for the target",
Clean up the use of static and anonymous namespaces. This turned up several things that were neither in an anonymous namespace nor static but not intended to be global. llvm-svn: 51017
2008-05-13 02:00:25 +02:00
createDefaultScheduler);
Factor out more instruction scheduler code to the base class. llvm-svn: 25532
2006-01-23 08:01:07 +01:00
Split the SelectionDAG-building code, including the FunctionLoweringInfo and SelectionDAGLowering classes, out of SelectionDAGISel.cpp and put it in a separate file, SelectionDAGBuild.cpp. llvm-svn: 55701
2008-09-03 18:12:24 +02:00
namespace llvm {
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
Teach ISel not to optimize 'optnone' functions (revised). Improvements over r195317: - Set/restore EnableFastISel flag instead of just running FastISel within SelectAllBasicBlocks; the flag is checked in various places, and FastISel won't run properly if those places don't do the right thing. - Test looks for normal ISel versus FastISel behavior, and not something more subtle that doesn't work everywhere. Based on work by Andrea Di Biagio. llvm-svn: 195491
2013-11-22 20:11:24 +01:00
//===--------------------------------------------------------------------===//
/// \brief This class is used by SelectionDAGISel to temporarily override
/// the optimization level on a per-function basis.
class OptLevelChanger {
SelectionDAGISel &IS;
CodeGenOpt::Level SavedOptLevel;
bool SavedFastISel;
public:
OptLevelChanger(SelectionDAGISel &ISel,
CodeGenOpt::Level NewOptLevel) : IS(ISel) {
SavedOptLevel = IS.OptLevel;
if (NewOptLevel == SavedOptLevel)
return;
IS.OptLevel = NewOptLevel;
IS.TM.setOptLevel(NewOptLevel);
DEBUG(dbgs() << "\nChanging optimization level for Function "
<< IS.MF->getFunction()->getName() << "\n");
DEBUG(dbgs() << "\tBefore: -O" << SavedOptLevel
<< " ; After: -O" << NewOptLevel << "\n");
Have 'optnone' respect the -fast-isel=false option. This is primarily useful for debugging optnone v. ISel issues. Differential Revision: http://reviews.llvm.org/D14792 llvm-svn: 254335
2015-11-30 22:56:16 +01:00
SavedFastISel = IS.TM.Options.EnableFastISel;
if (NewOptLevel == CodeGenOpt::None) {
IS.TM.setFastISel(IS.TM.getO0WantsFastISel());
DEBUG(dbgs() << "\tFastISel is "
<< (IS.TM.Options.EnableFastISel ? "enabled" : "disabled")
<< "\n");
}
Teach ISel not to optimize 'optnone' functions (revised). Improvements over r195317: - Set/restore EnableFastISel flag instead of just running FastISel within SelectAllBasicBlocks; the flag is checked in various places, and FastISel won't run properly if those places don't do the right thing. - Test looks for normal ISel versus FastISel behavior, and not something more subtle that doesn't work everywhere. Based on work by Andrea Di Biagio. llvm-svn: 195491
2013-11-22 20:11:24 +01:00
}
~OptLevelChanger() {
if (IS.OptLevel == SavedOptLevel)
return;
DEBUG(dbgs() << "\nRestoring optimization level for Function "
<< IS.MF->getFunction()->getName() << "\n");
DEBUG(dbgs() << "\tBefore: -O" << IS.OptLevel
<< " ; After: -O" << SavedOptLevel << "\n");
IS.OptLevel = SavedOptLevel;
IS.TM.setOptLevel(SavedOptLevel);
IS.TM.setFastISel(SavedFastISel);
}
};
Split the SelectionDAG-building code, including the FunctionLoweringInfo and SelectionDAGLowering classes, out of SelectionDAGISel.cpp and put it in a separate file, SelectionDAGBuild.cpp. llvm-svn: 55701
2008-09-03 18:12:24 +02:00
//===--------------------------------------------------------------------===//
/// createDefaultScheduler - This creates an instruction scheduler appropriate
/// for the target.
When scheduling a block in parts, keep track of the overall instruction index across each part. Instruction indices are used to make live range queries, and live ranges can extend beyond scheduling region boundaries. Refactor the ScheduleDAGSDNodes class some more so that it doesn't have to worry about this additional information. llvm-svn: 64288
2009-02-11 05:27:20 +01:00
ScheduleDAGSDNodes* createDefaultScheduler(SelectionDAGISel *IS,
Instead of passing in an unsigned value for the optimization level, use an enum, which better identifies what the optimization is doing. And is more flexible for future uses. llvm-svn: 70440
2009-04-30 01:29:43 +02:00
CodeGenOpt::Level OptLevel) {
Cache TargetLowering on SelectionDAGISel and update previous calls to getTargetLowering() with the cached variable. llvm-svn: 219284
2014-10-08 09:32:17 +02:00
const TargetLowering *TLI = IS->TLI;
Cache SelectionDAGISel TargetInstrInfo lookups on the class and propagate. Also use the TargetSubtargetInfo and the MachineFunction and move TargetRegisterInfo query closer to uses. llvm-svn: 219273
2014-10-08 03:58:03 +02:00
const TargetSubtargetInfo &ST = IS->MF->getSubtarget();
More consts on TargetLowering references. llvm-svn: 62262
2009-01-15 17:58:17 +01:00
Move the Target way of overriding DAG Scheduler to a target hook Summary: The previous way of overriding it was relying on calling "setDefault" on the global registry, which implies global mutable state. Reviewers: echristo, atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11538 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 243388
2015-07-28 08:18:04 +02:00
// Try first to see if the Target has its own way of selecting a scheduler
if (auto *SchedulerCtor = ST.getDAGScheduler(OptLevel)) {
return SchedulerCtor(IS, OptLevel);
}
Remove useMachineScheduler and replace it with subtarget options that control, individually, all of the disparate things it was controlling. At the same time move a FIXME in the Hexagon port to a new subtarget function that will enable a user of the machine scheduler to avoid using the source scheduler for pre-RA-scheduling. The FIXME would have this removed, but involves either testcase changes or adding -pre-RA-sched=source to a few testcases. llvm-svn: 231980
2015-03-11 23:56:10 +01:00
if (OptLevel == CodeGenOpt::None ||
(ST.enableMachineScheduler() && ST.enableMachineSchedDefaultSched()) ||
Cache the TargetLowering info object as a pointer. Caching it as a pointer allows us to reset it if the TargetMachine object changes. llvm-svn: 183361
2013-06-06 02:43:09 +02:00
TLI->getSchedulingPreference() == Sched::Source)
Use the source-order scheduler instead of the "fast" scheduler at -O0, because it's more likely to keep debug line information in its original order. llvm-svn: 108496
2010-07-16 04:01:19 +02:00
return createSourceListDAGScheduler(IS, OptLevel);
Cache the TargetLowering info object as a pointer. Caching it as a pointer allows us to reset it if the TargetMachine object changes. llvm-svn: 183361
2013-06-06 02:43:09 +02:00
if (TLI->getSchedulingPreference() == Sched::RegPressure)
Add a hybrid bottom up scheduler that reduce register usage while avoiding pipeline stall. It's useful for targets like ARM cortex-a8. NEON has a lot of long latency instructions so a strict register pressure reduction scheduler does not work well. Early experiments show this speeds up some NEON loops by over 30%. llvm-svn: 104216
2010-05-20 08:13:19 +02:00
return createBURRListDAGScheduler(IS, OptLevel);
Cache the TargetLowering info object as a pointer. Caching it as a pointer allows us to reset it if the TargetMachine object changes. llvm-svn: 183361
2013-06-06 02:43:09 +02:00
if (TLI->getSchedulingPreference() == Sched::Hybrid)
Add an ILP scheduler. This is a register pressure aware scheduler that's appropriate for targets without detailed instruction iterineries. The scheduler schedules for increased instruction level parallelism in low register pressure situation; it schedules to reduce register pressure when the register pressure becomes high. On x86_64, this is a win for all tests in CFP2000. It also sped up 256.bzip2 by 16%. llvm-svn: 109300
2010-07-24 02:39:05 +02:00
return createHybridListDAGScheduler(IS, OptLevel);
Cache the TargetLowering info object as a pointer. Caching it as a pointer allows us to reset it if the TargetMachine object changes. llvm-svn: 183361
2013-06-06 02:43:09 +02:00
if (TLI->getSchedulingPreference() == Sched::VLIW)
VLIW specific scheduler framework that utilizes deterministic finite automaton (DFA). This new scheduler plugs into the existing selection DAG scheduling framework. It is a top-down critical path scheduler that tracks register pressure and uses a DFA for pipeline modeling. Patch by Sergei Larin! llvm-svn: 149547
2012-02-01 23:13:57 +01:00
return createVLIWDAGScheduler(IS, OptLevel);
Cache the TargetLowering info object as a pointer. Caching it as a pointer allows us to reset it if the TargetMachine object changes. llvm-svn: 183361
2013-06-06 02:43:09 +02:00
assert(TLI->getSchedulingPreference() == Sched::ILP &&
Code refactoring: pull SchedPreference enum from TargetLowering.h to TargetMachine.h and put it in its own namespace. llvm-svn: 104147
2010-05-19 22:19:50 +02:00
"Unknown sched type!");
Add an ILP scheduler. This is a register pressure aware scheduler that's appropriate for targets without detailed instruction iterineries. The scheduler schedules for increased instruction level parallelism in low register pressure situation; it schedules to reduce register pressure when the register pressure becomes high. On x86_64, this is a win for all tests in CFP2000. It also sped up 256.bzip2 by 16%. llvm-svn: 109300
2010-07-24 02:39:05 +02:00
return createILPListDAGScheduler(IS, OptLevel);
the result of an inline asm copy can be an arbitrary VT that the register class supports. In the case of vectors, this means we often get the wrong type (e.g. we get v4f32 instead of v8i16). Make sure to convert the vector result to the right type. This fixes CodeGen/X86/2007-04-11-InlineAsmVectorResult.ll llvm-svn: 35944
2007-04-12 08:00:20 +02:00
}
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
Fix Clang-tidy readability-redundant-control-flow warnings; other minor fixes. Differential revision: http://reviews.llvm.org/D16793 llvm-svn: 259539
2016-02-02 19:20:45 +01:00
} // end namespace llvm
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
Even though InsertAtEndOfBasicBlock is an ugly hack it still deserves a proper name. Rename it to EmitInstrWithCustomInserter since it does not necessarily insert instruction at the end. llvm-svn: 46562
2008-01-30 19:18:23 +01:00
// EmitInstrWithCustomInserter - This method should be implemented by targets
Rename usesCustomDAGSchedInserter to usesCustomInserter, and update a bunch of associated comments, because it doesn't have anything to do with DAGs or scheduling. This is another step in decoupling MachineInstr emitting from scheduling. llvm-svn: 85517
2009-10-29 19:10:34 +01:00
// that mark instructions with the 'usesCustomInserter' flag. These
Call the InsertAtEndOfBasicBlock hook if the usesCustomDAGSchedInserter flag is set on an instruction. llvm-svn: 23098
2005-08-26 22:54:47 +02:00
// instructions are special in various ways, which require special support to
// insert. The specified MachineInstr is created but not inserted into any
Rename usesCustomDAGSchedInserter to usesCustomInserter, and update a bunch of associated comments, because it doesn't have anything to do with DAGs or scheduling. This is another step in decoupling MachineInstr emitting from scheduling. llvm-svn: 85517
2009-10-29 19:10:34 +01:00
// basic blocks, and this method is called to expand it into a sequence of
// instructions, potentially also creating new basic blocks and control flow.
// When new basic blocks are inserted and the edges from MBB to its successors
// are modified, the method should insert pairs of <OldSucc, NewSucc> into the
// DenseMap.
Get rid of the EdgeMapping map. Instead, just check for BasicBlock changes before doing phi lowering for switches. llvm-svn: 102809
2010-05-01 02:01:06 +02:00
MachineBasicBlock *
CodeGen: Use MachineInstr& in TargetLowering, NFC This is a mechanical change to make TargetLowering API take MachineInstr& (instead of MachineInstr*), since the argument is expected to be a valid MachineInstr. In one case, changed a parameter from MachineInstr* to MachineBasicBlock::iterator, since it was used as an insertion point. As a side effect, this removes a bunch of MachineInstr* to MachineBasicBlock::iterator implicit conversions, a necessary step toward fixing PR26753. llvm-svn: 274287
2016-07-01 00:52:52 +02:00
TargetLowering::EmitInstrWithCustomInserter(MachineInstr &MI,
Get rid of the EdgeMapping map. Instead, just check for BasicBlock changes before doing phi lowering for switches. llvm-svn: 102809
2010-05-01 02:01:06 +02:00
MachineBasicBlock *MBB) const {
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-12 22:07:01 +02:00
#ifndef NDEBUG
Change errs() to dbgs(). llvm-svn: 92597
2010-01-05 02:26:11 +01:00
dbgs() << "If a target marks an instruction with "
Rename usesCustomDAGSchedInserter to usesCustomInserter, and update a bunch of associated comments, because it doesn't have anything to do with DAGs or scheduling. This is another step in decoupling MachineInstr emitting from scheduling. llvm-svn: 85517
2009-10-29 19:10:34 +01:00
"'usesCustomInserter', it must implement "
Fix assert(0) conversion, as suggested by Chris. llvm-svn: 75423
2009-07-12 22:07:01 +02:00
"TargetLowering::EmitInstrWithCustomInserter!";
#endif
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
llvm_unreachable(nullptr);
Call the InsertAtEndOfBasicBlock hook if the usesCustomDAGSchedInserter flag is set on an instruction. llvm-svn: 23098
2005-08-26 22:54:47 +02:00
}
CodeGen: Use MachineInstr& in TargetLowering, NFC This is a mechanical change to make TargetLowering API take MachineInstr& (instead of MachineInstr*), since the argument is expected to be a valid MachineInstr. In one case, changed a parameter from MachineInstr* to MachineBasicBlock::iterator, since it was used as an insertion point. As a side effect, this removes a bunch of MachineInstr* to MachineBasicBlock::iterator implicit conversions, a necessary step toward fixing PR26753. llvm-svn: 274287
2016-07-01 00:52:52 +02:00
void TargetLowering::AdjustInstrPostInstrSelection(MachineInstr &MI,
Follow up to r138791. Add a instruction flag: hasPostISelHook which tells the pre-RA scheduler to call a target hook to adjust the instruction. For ARM, this is used to adjust instructions which may be setting the 's' flag. ADC, SBC, RSB, and RSC instructions have implicit def of CPSR (required since it now uses CPSR physical register dependency rather than "glue"). If the carry flag is used, then the target hook will *fill in* the optional operand with CPSR. Otherwise, the hook will remove the CPSR implicit def from the MachineInstr. llvm-svn: 138810
2011-08-30 21:09:48 +02:00
SDNode *Node) const {
CodeGen: Use MachineInstr& in TargetLowering, NFC This is a mechanical change to make TargetLowering API take MachineInstr& (instead of MachineInstr*), since the argument is expected to be a valid MachineInstr. In one case, changed a parameter from MachineInstr* to MachineBasicBlock::iterator, since it was used as an insertion point. As a side effect, this removes a bunch of MachineInstr* to MachineBasicBlock::iterator implicit conversions, a necessary step toward fixing PR26753. llvm-svn: 274287
2016-07-01 00:52:52 +02:00
assert(!MI.hasPostISelHook() &&
Lower ARM adds/subs to add/sub after adding optional CPSR operand. This is still a hack until we can teach tblgen to generate the optional CPSR operand rather than an implicit CPSR def. But the strangeness is now limited to the selection DAG. ADD/SUB MI's no longer have implicit CPSR defs, nor do we allow flag setting variants of these opcodes in machine code. There are several corner cases to consider, and getting one wrong would previously lead to nasty miscompilation. It's not the first time I've debugged one, so this time I added enough verification to ensure it won't happen again. llvm-svn: 140228
2011-09-21 04:20:46 +02:00
"If a target marks an instruction with 'hasPostISelHook', "
"it must implement TargetLowering::AdjustInstrPostInstrSelection!");
Follow up to r138791. Add a instruction flag: hasPostISelHook which tells the pre-RA scheduler to call a target hook to adjust the instruction. For ARM, this is used to adjust instructions which may be setting the 's' flag. ADC, SBC, RSB, and RSC instructions have implicit def of CPSR (required since it now uses CPSR physical register dependency rather than "glue"). If the carry flag is used, then the target hook will *fill in* the optional operand with CPSR. Otherwise, the hook will remove the CPSR implicit def from the MachineInstr. llvm-svn: 138810
2011-08-30 21:09:48 +02:00
}
Turn memset/memcpy/memmove into the corresponding operations. llvm-svn: 19463
2005-01-11 06:56:49 +01:00
//===----------------------------------------------------------------------===//
// SelectionDAGISel code
//===----------------------------------------------------------------------===//
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
Access the TargetLoweringInfo from the TargetMachine object instead of caching it. The TLI may change between functions. No functionality change. llvm-svn: 184360
2013-06-19 23:36:55 +02:00
SelectionDAGISel::SelectionDAGISel(TargetMachine &tm,
80-cols. llvm-svn: 122909
2011-01-05 22:45:56 +01:00
CodeGenOpt::Level OL) :
Access the TargetLoweringInfo from the TargetMachine object instead of caching it. The TLI may change between functions. No functionality change. llvm-svn: 184360
2013-06-19 23:36:55 +02:00
MachineFunctionPass(ID), TM(tm),
Remove uses of the TargetMachine from FunctionLoweringInfo via caching TargetLowering and using the MachineFunction. llvm-svn: 219375
2014-10-09 02:57:31 +02:00
FuncInfo(new FunctionLoweringInfo()),
Update DebugLoc while merging nodes at -O0. Patch by Kyriakos Georgiou! llvm-svn: 146670
2011-12-15 19:21:18 +01:00
CurDAG(new SelectionDAG(tm, OL)),
Code that needs a TargetMachine should have access to one directly, rather than just getting one through a TargetLowering. llvm-svn: 101802
2010-04-19 21:05:59 +02:00
SDB(new SelectionDAGBuilder(*CurDAG, *FuncInfo, OL)),
Reorganize the lifetimes of the major objects SelectionDAGISel works with. SelectionDAG, FunctionLoweringInfo, and SelectionDAGLowering objects now get created once per SelectionDAGISel instance, and can be reused across blocks and across functions. Previously, they were created and destroyed each time they were needed. This reorganization simplifies the handling of PHI nodes, and also SwitchCases, JumpTables, and BitTestBlocks. This simplification has the side effect of fixing a bug in FastISel where successor PHI nodes weren't being updated correctly. This is also a step towards making the transition from FastISel into and out of SelectionDAG faster, and also making plain SelectionDAG faster on code with lots of little blocks. llvm-svn: 55450
2008-08-28 01:52:12 +02:00
GFI(),
Second attempt: Massive check in. This changes the "-fast" flag to "-O#" in llc. If you want to use the old behavior, the flag is -O0. This change allows for finer-grained control over which optimizations are run at different -O levels. Most of this work was pretty mechanical. The majority of the fixes came from verifying that a "fast" variable wasn't used anymore. The JIT still uses a "Fast" flag. I'll change the JIT with a follow-up patch. llvm-svn: 70343
2009-04-29 02:15:41 +02:00
OptLevel(OL),
Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which must be called in the pass's constructor. This function uses static dependency declarations to recursively initialize the pass's dependencies. Clients that only create passes through the createFooPass() APIs will require no changes. Clients that want to use the CommandLine options for passes will need to manually call the appropriate initialization functions in PassInitialization.h before parsing commandline arguments. I have tested this with all standard configurations of clang and llvm-gcc on Darwin. It is possible that there are problems with the static dependencies that will only be visible with non-standard options. If you encounter any crash in pass registration/creation, please send the testcase to me directly. llvm-svn: 116820
2010-10-19 19:21:58 +02:00
DAGSize(0) {
initializeGCModuleInfoPass(*PassRegistry::getPassRegistry());
Create a wrapper pass for BranchProbabilityInfo. This new wrapper pass is useful when we want to do branch probability analysis conditionally (e.g. only in PGO mode) but don't want to add one more pass dependence. http://reviews.llvm.org/D11241 llvm-svn: 242349
2015-07-16 00:48:29 +02:00
initializeBranchProbabilityInfoWrapperPassPass(
*PassRegistry::getPassRegistry());
[PM/AA] Rebuild LLVM's alias analysis infrastructure in a way compatible with the new pass manager, and no longer relying on analysis groups. This builds essentially a ground-up new AA infrastructure stack for LLVM. The core ideas are the same that are used throughout the new pass manager: type erased polymorphism and direct composition. The design is as follows: - FunctionAAResults is a type-erasing alias analysis results aggregation interface to walk a single query across a range of results from different alias analyses. Currently this is function-specific as we always assume that aliasing queries are *within* a function. - AAResultBase is a CRTP utility providing stub implementations of various parts of the alias analysis result concept, notably in several cases in terms of other more general parts of the interface. This can be used to implement only a narrow part of the interface rather than the entire interface. This isn't really ideal, this logic should be hoisted into FunctionAAResults as currently it will cause a significant amount of redundant work, but it faithfully models the behavior of the prior infrastructure. - All the alias analysis passes are ported to be wrapper passes for the legacy PM and new-style analysis passes for the new PM with a shared result object. In some cases (most notably CFL), this is an extremely naive approach that we should revisit when we can specialize for the new pass manager. - BasicAA has been restructured to reflect that it is much more fundamentally a function analysis because it uses dominator trees and loop info that need to be constructed for each function. All of the references to getting alias analysis results have been updated to use the new aggregation interface. All the preservation and other pass management code has been updated accordingly. The way the FunctionAAResultsWrapperPass works is to detect the available alias analyses when run, and add them to the results object. This means that we should be able to continue to respect when various passes are added to the pipeline, for example adding CFL or adding TBAA passes should just cause their results to be available and to get folded into this. The exception to this rule is BasicAA which really needs to be a function pass due to using dominator trees and loop info. As a consequence, the FunctionAAResultsWrapperPass directly depends on BasicAA and always includes it in the aggregation. This has significant implications for preserving analyses. Generally, most passes shouldn't bother preserving FunctionAAResultsWrapperPass because rebuilding the results just updates the set of known AA passes. The exception to this rule are LoopPass instances which need to preserve all the function analyses that the loop pass manager will end up needing. This means preserving both BasicAAWrapperPass and the aggregating FunctionAAResultsWrapperPass. Now, when preserving an alias analysis, you do so by directly preserving that analysis. This is only necessary for non-immutable-pass-provided alias analyses though, and there are only three of interest: BasicAA, GlobalsAA (formerly GlobalsModRef), and SCEVAA. Usually BasicAA is preserved when needed because it (like DominatorTree and LoopInfo) is marked as a CFG-only pass. I've expanded GlobalsAA into the preserved set everywhere we previously were preserving all of AliasAnalysis, and I've added SCEVAA in the intersection of that with where we preserve SCEV itself. One significant challenge to all of this is that the CGSCC passes were actually using the alias analysis implementations by taking advantage of a pretty amazing set of loop holes in the old pass manager's analysis management code which allowed analysis groups to slide through in many cases. Moving away from analysis groups makes this problem much more obvious. To fix it, I've leveraged the flexibility the design of the new PM components provides to just directly construct the relevant alias analyses for the relevant functions in the IPO passes that need them. This is a bit hacky, but should go away with the new pass manager, and is already in many ways cleaner than the prior state. Another significant challenge is that various facilities of the old alias analysis infrastructure just don't fit any more. The most significant of these is the alias analysis 'counter' pass. That pass relied on the ability to snoop on AA queries at different points in the analysis group chain. Instead, I'm planning to build printing functionality directly into the aggregation layer. I've not included that in this patch merely to keep it smaller. Note that all of this needs a nearly complete rewrite of the AA documentation. I'm planning to do that, but I'd like to make sure the new design settles, and to flesh out a bit more of what it looks like in the new pass manager first. Differential Revision: http://reviews.llvm.org/D12080 llvm-svn: 247167
2015-09-09 19:55:00 +02:00
initializeAAResultsWrapperPassPass(*PassRegistry::getPassRegistry());
[PM] Separate the TargetLibraryInfo object from the immutable pass. The pass is really just a means of accessing a cached instance of the TargetLibraryInfo object, and this way we can re-use that object for the new pass manager as its result. Lots of delta, but nothing interesting happening here. This is the common pattern that is developing to allow analyses to live in both the old and new pass manager -- a wrapper pass in the old pass manager emulates the separation intrinsic to the new pass manager between the result and pass for analyses. llvm-svn: 226157
2015-01-15 11:41:28 +01:00
initializeTargetLibraryInfoWrapperPassPass(
*PassRegistry::getPassRegistry());
Get rid of static constructors for pass registration. Instead, every pass exposes an initializeMyPassFunction(), which must be called in the pass's constructor. This function uses static dependency declarations to recursively initialize the pass's dependencies. Clients that only create passes through the createFooPass() APIs will require no changes. Clients that want to use the CommandLine options for passes will need to manually call the appropriate initialization functions in PassInitialization.h before parsing commandline arguments. I have tested this with all standard configurations of clang and llvm-gcc on Darwin. It is possible that there are problems with the static dependencies that will only be visible with non-standard options. If you encounter any crash in pass registration/creation, please send the testcase to me directly. llvm-svn: 116820
2010-10-19 19:21:58 +02:00
}
Reorganize the lifetimes of the major objects SelectionDAGISel works with. SelectionDAG, FunctionLoweringInfo, and SelectionDAGLowering objects now get created once per SelectionDAGISel instance, and can be reused across blocks and across functions. Previously, they were created and destroyed each time they were needed. This reorganization simplifies the handling of PHI nodes, and also SwitchCases, JumpTables, and BitTestBlocks. This simplification has the side effect of fixing a bug in FastISel where successor PHI nodes weren't being updated correctly. This is also a step towards making the transition from FastISel into and out of SelectionDAG faster, and also making plain SelectionDAG faster on code with lots of little blocks. llvm-svn: 55450
2008-08-28 01:52:12 +02:00
SelectionDAGISel::~SelectionDAGISel() {
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
delete SDB;
Reorganize the lifetimes of the major objects SelectionDAGISel works with. SelectionDAG, FunctionLoweringInfo, and SelectionDAGLowering objects now get created once per SelectionDAGISel instance, and can be reused across blocks and across functions. Previously, they were created and destroyed each time they were needed. This reorganization simplifies the handling of PHI nodes, and also SwitchCases, JumpTables, and BitTestBlocks. This simplification has the side effect of fixing a bug in FastISel where successor PHI nodes weren't being updated correctly. This is also a step towards making the transition from FastISel into and out of SelectionDAG faster, and also making plain SelectionDAG faster on code with lots of little blocks. llvm-svn: 55450
2008-08-28 01:52:12 +02:00
delete CurDAG;
delete FuncInfo;
}
Add a new beta option for critical edge splitting, to avoid a problem that Nate noticed in yacr2 (and I know occurs in other places as well). This is still rough, as the critical edge blocks are not intelligently placed but is added to get some idea to see if this improves performance. llvm-svn: 22825
2005-08-17 08:37:43 +02:00
void SelectionDAGISel::getAnalysisUsage(AnalysisUsage &AU) const {
[PM/AA] Rebuild LLVM's alias analysis infrastructure in a way compatible with the new pass manager, and no longer relying on analysis groups. This builds essentially a ground-up new AA infrastructure stack for LLVM. The core ideas are the same that are used throughout the new pass manager: type erased polymorphism and direct composition. The design is as follows: - FunctionAAResults is a type-erasing alias analysis results aggregation interface to walk a single query across a range of results from different alias analyses. Currently this is function-specific as we always assume that aliasing queries are *within* a function. - AAResultBase is a CRTP utility providing stub implementations of various parts of the alias analysis result concept, notably in several cases in terms of other more general parts of the interface. This can be used to implement only a narrow part of the interface rather than the entire interface. This isn't really ideal, this logic should be hoisted into FunctionAAResults as currently it will cause a significant amount of redundant work, but it faithfully models the behavior of the prior infrastructure. - All the alias analysis passes are ported to be wrapper passes for the legacy PM and new-style analysis passes for the new PM with a shared result object. In some cases (most notably CFL), this is an extremely naive approach that we should revisit when we can specialize for the new pass manager. - BasicAA has been restructured to reflect that it is much more fundamentally a function analysis because it uses dominator trees and loop info that need to be constructed for each function. All of the references to getting alias analysis results have been updated to use the new aggregation interface. All the preservation and other pass management code has been updated accordingly. The way the FunctionAAResultsWrapperPass works is to detect the available alias analyses when run, and add them to the results object. This means that we should be able to continue to respect when various passes are added to the pipeline, for example adding CFL or adding TBAA passes should just cause their results to be available and to get folded into this. The exception to this rule is BasicAA which really needs to be a function pass due to using dominator trees and loop info. As a consequence, the FunctionAAResultsWrapperPass directly depends on BasicAA and always includes it in the aggregation. This has significant implications for preserving analyses. Generally, most passes shouldn't bother preserving FunctionAAResultsWrapperPass because rebuilding the results just updates the set of known AA passes. The exception to this rule are LoopPass instances which need to preserve all the function analyses that the loop pass manager will end up needing. This means preserving both BasicAAWrapperPass and the aggregating FunctionAAResultsWrapperPass. Now, when preserving an alias analysis, you do so by directly preserving that analysis. This is only necessary for non-immutable-pass-provided alias analyses though, and there are only three of interest: BasicAA, GlobalsAA (formerly GlobalsModRef), and SCEVAA. Usually BasicAA is preserved when needed because it (like DominatorTree and LoopInfo) is marked as a CFG-only pass. I've expanded GlobalsAA into the preserved set everywhere we previously were preserving all of AliasAnalysis, and I've added SCEVAA in the intersection of that with where we preserve SCEV itself. One significant challenge to all of this is that the CGSCC passes were actually using the alias analysis implementations by taking advantage of a pretty amazing set of loop holes in the old pass manager's analysis management code which allowed analysis groups to slide through in many cases. Moving away from analysis groups makes this problem much more obvious. To fix it, I've leveraged the flexibility the design of the new PM components provides to just directly construct the relevant alias analyses for the relevant functions in the IPO passes that need them. This is a bit hacky, but should go away with the new pass manager, and is already in many ways cleaner than the prior state. Another significant challenge is that various facilities of the old alias analysis infrastructure just don't fit any more. The most significant of these is the alias analysis 'counter' pass. That pass relied on the ability to snoop on AA queries at different points in the analysis group chain. Instead, I'm planning to build printing functionality directly into the aggregation layer. I've not included that in this patch merely to keep it smaller. Note that all of this needs a nearly complete rewrite of the AA documentation. I'm planning to do that, but I'd like to make sure the new design settles, and to flesh out a bit more of what it looks like in the new pass manager first. Differential Revision: http://reviews.llvm.org/D12080 llvm-svn: 247167
2015-09-09 19:55:00 +02:00
AU.addRequired<AAResultsWrapperPass>();
Rename some GC classes so that their roll will hopefully be clearer. In particular, Collector was confusing to implementors. Several thought that this compile-time class was the place to implement their runtime GC heap. Of course, it doesn't even exist at runtime. Specifically, the renames are: Collector -> GCStrategy CollectorMetadata -> GCFunctionInfo CollectorModuleMetadata -> GCModuleInfo CollectorRegistry -> GCRegistry Function::getCollector -> getGC (setGC, hasGC, clearGC) Several accessors and nested types have also been renamed to be consistent. These changes should be obvious. llvm-svn: 54899
2008-08-17 20:44:35 +02:00
AU.addRequired<GCModuleInfo>();
[SSP] Remove llvm.stackprotectorcheck. This is a cleanup patch for SSP support in LLVM. There is no functional change. llvm.stackprotectorcheck is not needed, because SelectionDAG isn't actually lowering it in SelectBasicBlock; rather, it adds check code in FinishBasicBlock, ignoring the position where the intrinsic is inserted (See FindSplitPointForStackProtector()). llvm-svn: 265851
2016-04-08 23:26:31 +02:00
AU.addRequired<StackProtector>();
AU.addPreserved<StackProtector>();
SelectionDAGISel does not "preserve all", since it makes lots of changes to the MachineFunction. llvm-svn: 77753
2009-08-01 01:36:22 +02:00
AU.addPreserved<GCModuleInfo>();
[PM] Separate the TargetLibraryInfo object from the immutable pass. The pass is really just a means of accessing a cached instance of the TargetLibraryInfo object, and this way we can re-use that object for the new pass manager as its result. Lots of delta, but nothing interesting happening here. This is the common pattern that is developing to allow analyses to live in both the old and new pass manager -- a wrapper pass in the old pass manager emulates the separation intrinsic to the new pass manager between the result and pass for analyses. llvm-svn: 226157
2015-01-15 11:41:28 +01:00
AU.addRequired<TargetLibraryInfoWrapperPass>();
Introduce MachineBranchProbabilityInfo class, which has similar API to BranchProbabilityInfo (expect setEdgeWeight which is not available here). Branch Weights are kept in MachineBasicBlocks. To turn off this analysis set -use-mbpi=false. llvm-svn: 133184
2011-06-16 22:22:37 +02:00
if (UseMBPI && OptLevel != CodeGenOpt::None)
Create a wrapper pass for BranchProbabilityInfo. This new wrapper pass is useful when we want to do branch probability analysis conditionally (e.g. only in PGO mode) but don't want to add one more pass dependence. http://reviews.llvm.org/D11241 llvm-svn: 242349
2015-07-16 00:48:29 +02:00
AU.addRequired<BranchProbabilityInfoWrapperPass>();
Reapply r77654 with a fix: MachineFunctionPass's getAnalysisUsage shouldn't do AU.setPreservesCFG(), because even though CodeGen passes don't modify the LLVM IR CFG, they may modify the MachineFunction CFG, and passes like MachineLoop are registered with isCFGOnly set to true. llvm-svn: 77691
2009-07-31 20:16:33 +02:00
MachineFunctionPass::getAnalysisUsage(AU);
Add a new beta option for critical edge splitting, to avoid a problem that Nate noticed in yacr2 (and I know occurs in other places as well). This is still rough, as the critical edge blocks are not intelligently placed but is added to get some idea to see if this improves performance. llvm-svn: 22825
2005-08-17 08:37:43 +02:00
}
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
fix PR8642: if a critical edge has a PHI value that can trap, isel is *required* to split the edge. PHI values get evaluated on the edge, not in their predecessor block. llvm-svn: 122170
2010-12-19 05:58:57 +01:00
/// SplitCriticalSideEffectEdges - Look for critical edges with a PHI value that
/// may trap on it. In this case we have to split the edge so that the path
/// through the predecessor block that doesn't go to the phi block doesn't
/// execute the possibly trapping instruction.
///
/// This is required for correctness, so it must be done at -O0.
///
[PM/AA] Remove all of the dead AliasAnalysis pointers being threaded through APIs that are no longer necessary now that the update API has been removed. This will make changes to the AA interfaces significantly less disruptive (I hope). Either way, it seems like a really nice cleanup. llvm-svn: 242882
2015-07-22 11:52:54 +02:00
static void SplitCriticalSideEffectEdges(Function &Fn) {
fix PR8642: if a critical edge has a PHI value that can trap, isel is *required* to split the edge. PHI values get evaluated on the edge, not in their predecessor block. llvm-svn: 122170
2010-12-19 05:58:57 +01:00
// Loop for blocks with phi nodes.
SelectionDAG: Remove implicit ilist iterator conversions, NFC llvm-svn: 250214
2015-10-13 21:47:46 +02:00
for (BasicBlock &BB : Fn) {
PHINode *PN = dyn_cast<PHINode>(BB.begin());
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
if (!PN) continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
fix PR8642: if a critical edge has a PHI value that can trap, isel is *required* to split the edge. PHI values get evaluated on the edge, not in their predecessor block. llvm-svn: 122170
2010-12-19 05:58:57 +01:00
ReprocessBlock:
// For each block with a PHI node, check to see if any of the input values
// are potentially trapping constant expressions. Constant expressions are
// the only potentially trapping value that can occur as the argument to a
// PHI.
SelectionDAG: Remove implicit ilist iterator conversions, NFC llvm-svn: 250214
2015-10-13 21:47:46 +02:00
for (BasicBlock::iterator I = BB.begin(); (PN = dyn_cast<PHINode>(I)); ++I)
fix PR8642: if a critical edge has a PHI value that can trap, isel is *required* to split the edge. PHI values get evaluated on the edge, not in their predecessor block. llvm-svn: 122170
2010-12-19 05:58:57 +01:00
for (unsigned i = 0, e = PN->getNumIncomingValues(); i != e; ++i) {
ConstantExpr *CE = dyn_cast<ConstantExpr>(PN->getIncomingValue(i));
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
if (!CE || !CE->canTrap()) continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
fix PR8642: if a critical edge has a PHI value that can trap, isel is *required* to split the edge. PHI values get evaluated on the edge, not in their predecessor block. llvm-svn: 122170
2010-12-19 05:58:57 +01:00
// The only case we have to worry about is when the edge is critical.
// Since this block has a PHI Node, we assume it has multiple input
// edges: check to see if the pred has multiple successors.
BasicBlock *Pred = PN->getIncomingBlock(i);
if (Pred->getTerminator()->getNumSuccessors() == 1)
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
fix PR8642: if a critical edge has a PHI value that can trap, isel is *required* to split the edge. PHI values get evaluated on the edge, not in their predecessor block. llvm-svn: 122170
2010-12-19 05:58:57 +01:00
// Okay, we have to split this edge.
[PM] Remove the Pass argument from all of the critical edge splitting APIs and replace it and numerous booleans with an option struct. The critical edge splitting API has a really large surface of flags and so it seems worth burning a small option struct / builder. This struct can be constructed with the various preserved analyses and then flags can be flipped in a builder style. The various users are now responsible for directly passing along their analysis information. This should be enough for the critical edge splitting to work cleanly with the new pass manager as well. This API is still pretty crufty and could be cleaned up a lot, but I've focused on this change just threading an option struct rather than a pass through the API. llvm-svn: 226456
2015-01-19 13:09:11 +01:00
SplitCriticalEdge(
SelectionDAG: Remove implicit ilist iterator conversions, NFC llvm-svn: 250214
2015-10-13 21:47:46 +02:00
Pred->getTerminator(), GetSuccessorNumber(Pred, &BB),
[PM/AA] Remove all of the dead AliasAnalysis pointers being threaded through APIs that are no longer necessary now that the update API has been removed. This will make changes to the AA interfaces significantly less disruptive (I hope). Either way, it seems like a really nice cleanup. llvm-svn: 242882
2015-07-22 11:52:54 +02:00
CriticalEdgeSplittingOptions().setMergeIdenticalEdges());
fix PR8642: if a critical edge has a PHI value that can trap, isel is *required* to split the edge. PHI values get evaluated on the edge, not in their predecessor block. llvm-svn: 122170
2010-12-19 05:58:57 +01:00
goto ReprocessBlock;
}
}
}
Reapply r77654 with a fix: MachineFunctionPass's getAnalysisUsage shouldn't do AU.setPreservesCFG(), because even though CodeGen passes don't modify the LLVM IR CFG, they may modify the MachineFunction CFG, and passes like MachineLoop are registered with isCFGOnly set to true. llvm-svn: 77691
2009-07-31 20:16:33 +02:00
bool SelectionDAGISel::runOnMachineFunction(MachineFunction &mf) {
[SelectionDAG] Do not run the ISel process on already selected code. Right now, this cannot happen, but with the fall back path of GlobalISel it will show up eventually. llvm-svn: 279877
2016-08-27 00:32:55 +02:00
// If we already selected that function, we do not need to run SDISel.
if (mf.getProperties().hasProperty(
MachineFunctionProperties::Property::Selected))
return false;
Change -fast-isel-no-abort to -fast-isel-abort, which now defaults to being off by default. Also, add assertion checks to check that the various fast-isel-related command-line options are only used when -fast-isel itself is enabled. llvm-svn: 56029
2008-09-10 01:05:00 +02:00
// Do some sanity-checking on the command-line options.
Move global variables in TargetMachine into new TargetOptions class. As an API change, now you need a TargetOptions object to create a TargetMachine. Clang patch to follow. One small functionality change in PTX. PTX had commented out the machine verifier parts in their copy of printAndVerify. That now calls the version in LLVMTargetMachine. Users of PTX who need verification disabled should rely on not passing the command-line flag to enable it. llvm-svn: 145714
2011-12-02 23:16:29 +01:00
assert((!EnableFastISelVerbose || TM.Options.EnableFastISel) &&
Change -fast-isel-no-abort to -fast-isel-abort, which now defaults to being off by default. Also, add assertion checks to check that the various fast-isel-related command-line options are only used when -fast-isel itself is enabled. llvm-svn: 56029
2008-09-10 01:05:00 +02:00
"-fast-isel-verbose requires -fast-isel");
Move global variables in TargetMachine into new TargetOptions class. As an API change, now you need a TargetOptions object to create a TargetMachine. Clang patch to follow. One small functionality change in PTX. PTX had commented out the machine verifier parts in their copy of printAndVerify. That now calls the version in LLVMTargetMachine. Users of PTX who need verification disabled should rely on not passing the command-line flag to enable it. llvm-svn: 145714
2011-12-02 23:16:29 +01:00
assert((!EnableFastISelAbort || TM.Options.EnableFastISel) &&
Change the fast-isel-abort option from bool to int to enable "levels" Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
2015-02-27 19:32:11 +01:00
"-fast-isel-abort > 0 requires -fast-isel");
Change -fast-isel-no-abort to -fast-isel-abort, which now defaults to being off by default. Also, add assertion checks to check that the various fast-isel-related command-line options are only used when -fast-isel itself is enabled. llvm-svn: 56029
2008-09-10 01:05:00 +02:00
Add more const qualifiers for LLVM IR pointers in CodeGen. llvm-svn: 101342
2010-04-15 06:33:49 +02:00
const Function &Fn = *mf.getFunction();
Reorgnaize this code to be more tidy and readable. llvm-svn: 101256
2010-04-14 19:02:07 +02:00
MF = &mf;
Reset the target options and optimization level as the first thing we do inside selection dag. This code needs to be migrated to queries on the function rather than global data, but this organizes things before we start grabbing the subtarget. llvm-svn: 219271
2014-10-08 03:58:01 +02:00
// Reset the target options before resetting the optimization
// level below.
// FIXME: This is a horrible hack and should be processed via
// codegen looking at the optimization level explicitly when
// it wants to look at it.
Move resetTargetOptions from taking a MachineFunction to a Function since we are accessing the TargetMachine that we're a member function of. llvm-svn: 218489
2014-09-26 03:28:10 +02:00
TM.resetTargetOptions(Fn);
Teach ISel not to optimize 'optnone' functions (revised). Improvements over r195317: - Set/restore EnableFastISel flag instead of just running FastISel within SelectAllBasicBlocks; the flag is checked in various places, and FastISel won't run properly if those places don't do the right thing. - Test looks for normal ISel versus FastISel behavior, and not something more subtle that doesn't work everywhere. Based on work by Andrea Di Biagio. llvm-svn: 195491
2013-11-22 20:11:24 +01:00
// Reset OptLevel to None for optnone functions.
CodeGenOpt::Level NewOptLevel = OptLevel;
Include SelectionDAGISel in the opt-bisect process Differential Revision: http://reviews.llvm.org/D21143 llvm-svn: 274786
2016-07-07 20:55:02 +02:00
if (OptLevel != CodeGenOpt::None && skipFunction(Fn))
Teach ISel not to optimize 'optnone' functions (revised). Improvements over r195317: - Set/restore EnableFastISel flag instead of just running FastISel within SelectAllBasicBlocks; the flag is checked in various places, and FastISel won't run properly if those places don't do the right thing. - Test looks for normal ISel versus FastISel behavior, and not something more subtle that doesn't work everywhere. Based on work by Andrea Di Biagio. llvm-svn: 195491
2013-11-22 20:11:24 +01:00
NewOptLevel = CodeGenOpt::None;
OptLevelChanger OLC(*this, NewOptLevel);
Cache TargetLowering on SelectionDAGISel and update previous calls to getTargetLowering() with the cached variable. llvm-svn: 219284
2014-10-08 09:32:17 +02:00
TII = MF->getSubtarget().getInstrInfo();
TLI = MF->getSubtarget().getTargetLowering();
Reset the target options and optimization level as the first thing we do inside selection dag. This code needs to be migrated to queries on the function rather than global data, but this organizes things before we start grabbing the subtarget. llvm-svn: 219271
2014-10-08 03:58:01 +02:00
RegInfo = &MF->getRegInfo();
[PM/AA] Rebuild LLVM's alias analysis infrastructure in a way compatible with the new pass manager, and no longer relying on analysis groups. This builds essentially a ground-up new AA infrastructure stack for LLVM. The core ideas are the same that are used throughout the new pass manager: type erased polymorphism and direct composition. The design is as follows: - FunctionAAResults is a type-erasing alias analysis results aggregation interface to walk a single query across a range of results from different alias analyses. Currently this is function-specific as we always assume that aliasing queries are *within* a function. - AAResultBase is a CRTP utility providing stub implementations of various parts of the alias analysis result concept, notably in several cases in terms of other more general parts of the interface. This can be used to implement only a narrow part of the interface rather than the entire interface. This isn't really ideal, this logic should be hoisted into FunctionAAResults as currently it will cause a significant amount of redundant work, but it faithfully models the behavior of the prior infrastructure. - All the alias analysis passes are ported to be wrapper passes for the legacy PM and new-style analysis passes for the new PM with a shared result object. In some cases (most notably CFL), this is an extremely naive approach that we should revisit when we can specialize for the new pass manager. - BasicAA has been restructured to reflect that it is much more fundamentally a function analysis because it uses dominator trees and loop info that need to be constructed for each function. All of the references to getting alias analysis results have been updated to use the new aggregation interface. All the preservation and other pass management code has been updated accordingly. The way the FunctionAAResultsWrapperPass works is to detect the available alias analyses when run, and add them to the results object. This means that we should be able to continue to respect when various passes are added to the pipeline, for example adding CFL or adding TBAA passes should just cause their results to be available and to get folded into this. The exception to this rule is BasicAA which really needs to be a function pass due to using dominator trees and loop info. As a consequence, the FunctionAAResultsWrapperPass directly depends on BasicAA and always includes it in the aggregation. This has significant implications for preserving analyses. Generally, most passes shouldn't bother preserving FunctionAAResultsWrapperPass because rebuilding the results just updates the set of known AA passes. The exception to this rule are LoopPass instances which need to preserve all the function analyses that the loop pass manager will end up needing. This means preserving both BasicAAWrapperPass and the aggregating FunctionAAResultsWrapperPass. Now, when preserving an alias analysis, you do so by directly preserving that analysis. This is only necessary for non-immutable-pass-provided alias analyses though, and there are only three of interest: BasicAA, GlobalsAA (formerly GlobalsModRef), and SCEVAA. Usually BasicAA is preserved when needed because it (like DominatorTree and LoopInfo) is marked as a CFG-only pass. I've expanded GlobalsAA into the preserved set everywhere we previously were preserving all of AliasAnalysis, and I've added SCEVAA in the intersection of that with where we preserve SCEV itself. One significant challenge to all of this is that the CGSCC passes were actually using the alias analysis implementations by taking advantage of a pretty amazing set of loop holes in the old pass manager's analysis management code which allowed analysis groups to slide through in many cases. Moving away from analysis groups makes this problem much more obvious. To fix it, I've leveraged the flexibility the design of the new PM components provides to just directly construct the relevant alias analyses for the relevant functions in the IPO passes that need them. This is a bit hacky, but should go away with the new pass manager, and is already in many ways cleaner than the prior state. Another significant challenge is that various facilities of the old alias analysis infrastructure just don't fit any more. The most significant of these is the alias analysis 'counter' pass. That pass relied on the ability to snoop on AA queries at different points in the analysis group chain. Instead, I'm planning to build printing functionality directly into the aggregation layer. I've not included that in this patch merely to keep it smaller. Note that all of this needs a nearly complete rewrite of the AA documentation. I'm planning to do that, but I'd like to make sure the new design settles, and to flesh out a bit more of what it looks like in the new pass manager first. Differential Revision: http://reviews.llvm.org/D12080 llvm-svn: 247167
2015-09-09 19:55:00 +02:00
AA = &getAnalysis<AAResultsWrapperPass>().getAAResults();
[PM] Separate the TargetLibraryInfo object from the immutable pass. The pass is really just a means of accessing a cached instance of the TargetLibraryInfo object, and this way we can re-use that object for the new pass manager as its result. Lots of delta, but nothing interesting happening here. This is the common pattern that is developing to allow analyses to live in both the old and new pass manager -- a wrapper pass in the old pass manager emulates the separation intrinsic to the new pass manager between the result and pass for analyses. llvm-svn: 226157
2015-01-15 11:41:28 +01:00
LibInfo = &getAnalysis<TargetLibraryInfoWrapperPass>().getTLI();
Reset the target options and optimization level as the first thing we do inside selection dag. This code needs to be migrated to queries on the function rather than global data, but this organizes things before we start grabbing the subtarget. llvm-svn: 219271
2014-10-08 03:58:01 +02:00
GFI = Fn.hasGC() ? &getAnalysis<GCModuleInfo>().getFunctionInfo(Fn) : nullptr;
Change errs() to dbgs(). llvm-svn: 92597
2010-01-05 02:26:11 +01:00
DEBUG(dbgs() << "\n\n\n=== " << Fn.getName() << "\n");
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
[PM/AA] Remove all of the dead AliasAnalysis pointers being threaded through APIs that are no longer necessary now that the update API has been removed. This will make changes to the AA interfaces significantly less disruptive (I hope). Either way, it seems like a really nice cleanup. llvm-svn: 242882
2015-07-22 11:52:54 +02:00
SplitCriticalSideEffectEdges(const_cast<Function &>(Fn));
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Have the selection dag grab TargetLowering off of the subtarget inside init rather than have it passed in as an argument. llvm-svn: 219270
2014-10-08 03:57:58 +02:00
CurDAG->init(*MF);
Check for dynamic allocas and inline asm that clobbers sp before building selection dag (PR19012) In X86SelectionDagInfo::EmitTargetCodeForMemcpy we check with MachineFrameInfo to make sure that ESI isn't used as a base pointer register before we choose to emit rep movs (which clobbers esi). The problem is that MachineFrameInfo wouldn't know about dynamic allocas or inline asm that clobbers the stack pointer until SelectionDAGBuilder has encountered them. This patch fixes the problem by checking for such things when building the FunctionLoweringInfo. Differential Revision: http://llvm-reviews.chandlerc.com/D2954 llvm-svn: 202930
2014-03-05 03:43:26 +01:00
FuncInfo->set(Fn, *MF, CurDAG);
Introduce MachineBranchProbabilityInfo class, which has similar API to BranchProbabilityInfo (expect setEdgeWeight which is not available here). Branch Weights are kept in MachineBasicBlocks. To turn off this analysis set -use-mbpi=false. llvm-svn: 133184
2011-06-16 22:22:37 +02:00
if (UseMBPI && OptLevel != CodeGenOpt::None)
Create a wrapper pass for BranchProbabilityInfo. This new wrapper pass is useful when we want to do branch probability analysis conditionally (e.g. only in PGO mode) but don't want to add one more pass dependence. http://reviews.llvm.org/D11241 llvm-svn: 242349
2015-07-16 00:48:29 +02:00
FuncInfo->BPI = &getAnalysis<BranchProbabilityInfoWrapperPass>().getBPI();
Introduce MachineBranchProbabilityInfo class, which has similar API to BranchProbabilityInfo (expect setEdgeWeight which is not available here). Branch Weights are kept in MachineBasicBlocks. To turn off this analysis set -use-mbpi=false. llvm-svn: 133184
2011-06-16 22:22:37 +02:00
else
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
FuncInfo->BPI = nullptr;
Introduce MachineBranchProbabilityInfo class, which has similar API to BranchProbabilityInfo (expect setEdgeWeight which is not available here). Branch Weights are kept in MachineBasicBlocks. To turn off this analysis set -use-mbpi=false. llvm-svn: 133184
2011-06-16 22:22:37 +02:00
Enhance both TargetLibraryInfo and SelectionDAGBuilder so that the latter can use the former to prevent the formation of libm SDNode's when -fno-builtin is passed. llvm-svn: 146193
2011-12-08 23:15:21 +01:00
SDB->init(GFI, *AA, LibInfo);
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
Reland "Fix miscompile of MS inline assembly with stack realignment" This re-lands commit r196876, which was reverted in r196879. The tests have been fixed to pass on platforms with a stack alignment larger than 4. Update to clang side tests will land shortly. llvm-svn: 196939
2013-12-10 19:27:32 +01:00
MF->setHasInlineAsm(false);
CXX_FAST_TLS calling convention: target independent portion. The access function has a short entry and a short exit, the initialization block is only run the first time. To improve the performance, we want to have a short frame at the entry and exit. We explicitly handle most of the CSRs via copies. Only the CSRs that are not handled via copies will be in CSR_SaveList. Frame lowering and prologue/epilogue insertion will generate a short frame in the entry and exit according to CSR_SaveList. The majority of the CSRs will be handled by register allcoator. Register allocator will try to spill and reload them in the initialization block. We add CSRsViaCopy, it will be explicitly handled during lowering. 1> we first set FunctionLoweringInfo->SplitCSR if conditions are met (the target supports it for the given calling convention and the function has only return exits). We also call TLI->initializeSplitCSR to perform initialization. 2> we call TLI->insertCopiesSplitCSR to insert copies from CSRsViaCopy to virtual registers at beginning of the entry block and copies from virtual registers to CSRsViaCopy at beginning of the exit blocks. 3> we also need to make sure the explicit copies will not be eliminated. rdar://problem/23557469 Differential Revision: http://reviews.llvm.org/D15340 llvm-svn: 255353
2015-12-11 19:24:30 +01:00
FuncInfo->SplitCSR = false;
CXX_FAST_TLS calling convention: target independent portion. Update supportSplitCSR's interface to take machine function instead of the calling convention. Review comments for http://reviews.llvm.org/D15341 llvm-svn: 255818
2015-12-16 21:45:48 +01:00
// We split CSR if the target supports it for the given function
CXX_FAST_TLS calling convention: target independent portion. The access function has a short entry and a short exit, the initialization block is only run the first time. To improve the performance, we want to have a short frame at the entry and exit. We explicitly handle most of the CSRs via copies. Only the CSRs that are not handled via copies will be in CSR_SaveList. Frame lowering and prologue/epilogue insertion will generate a short frame in the entry and exit according to CSR_SaveList. The majority of the CSRs will be handled by register allcoator. Register allocator will try to spill and reload them in the initialization block. We add CSRsViaCopy, it will be explicitly handled during lowering. 1> we first set FunctionLoweringInfo->SplitCSR if conditions are met (the target supports it for the given calling convention and the function has only return exits). We also call TLI->initializeSplitCSR to perform initialization. 2> we call TLI->insertCopiesSplitCSR to insert copies from CSRsViaCopy to virtual registers at beginning of the entry block and copies from virtual registers to CSRsViaCopy at beginning of the exit blocks. 3> we also need to make sure the explicit copies will not be eliminated. rdar://problem/23557469 Differential Revision: http://reviews.llvm.org/D15340 llvm-svn: 255353
2015-12-11 19:24:30 +01:00
// and the function has only return exits.
[CXX_FAST_TLS] fix issues with O0 on ARM, AArch64 and X86. Since at O0, explicit copies via SplitCSR may not be removed even if they are unnecessary, we choose not to use SplitCSR at O0. llvm-svn: 263855
2016-03-19 00:38:49 +01:00
if (OptLevel != CodeGenOpt::None && TLI->supportSplitCSR(MF)) {
CXX_FAST_TLS calling convention: target independent portion. The access function has a short entry and a short exit, the initialization block is only run the first time. To improve the performance, we want to have a short frame at the entry and exit. We explicitly handle most of the CSRs via copies. Only the CSRs that are not handled via copies will be in CSR_SaveList. Frame lowering and prologue/epilogue insertion will generate a short frame in the entry and exit according to CSR_SaveList. The majority of the CSRs will be handled by register allcoator. Register allocator will try to spill and reload them in the initialization block. We add CSRsViaCopy, it will be explicitly handled during lowering. 1> we first set FunctionLoweringInfo->SplitCSR if conditions are met (the target supports it for the given calling convention and the function has only return exits). We also call TLI->initializeSplitCSR to perform initialization. 2> we call TLI->insertCopiesSplitCSR to insert copies from CSRsViaCopy to virtual registers at beginning of the entry block and copies from virtual registers to CSRsViaCopy at beginning of the exit blocks. 3> we also need to make sure the explicit copies will not be eliminated. rdar://problem/23557469 Differential Revision: http://reviews.llvm.org/D15340 llvm-svn: 255353
2015-12-11 19:24:30 +01:00
FuncInfo->SplitCSR = true;
// Collect all the return blocks.
for (const BasicBlock &BB : Fn) {
if (!succ_empty(&BB))
continue;
const TerminatorInst *Term = BB.getTerminator();
CXX TLS: collect return blocks after SelectAllBasicBlocks. It is incorrect to get the corresponding MBB for a ReturnInst before SelectAllBasicBlocks since SelectAllBasicBlocks can change the correspondence between a ReturnInst and the MBB it is in. PR27062 llvm-svn: 264358
2016-03-25 00:21:29 +01:00
if (isa<UnreachableInst>(Term) || isa<ReturnInst>(Term))
CXX_FAST_TLS calling convention: target independent portion. The access function has a short entry and a short exit, the initialization block is only run the first time. To improve the performance, we want to have a short frame at the entry and exit. We explicitly handle most of the CSRs via copies. Only the CSRs that are not handled via copies will be in CSR_SaveList. Frame lowering and prologue/epilogue insertion will generate a short frame in the entry and exit according to CSR_SaveList. The majority of the CSRs will be handled by register allcoator. Register allocator will try to spill and reload them in the initialization block. We add CSRsViaCopy, it will be explicitly handled during lowering. 1> we first set FunctionLoweringInfo->SplitCSR if conditions are met (the target supports it for the given calling convention and the function has only return exits). We also call TLI->initializeSplitCSR to perform initialization. 2> we call TLI->insertCopiesSplitCSR to insert copies from CSRsViaCopy to virtual registers at beginning of the entry block and copies from virtual registers to CSRsViaCopy at beginning of the exit blocks. 3> we also need to make sure the explicit copies will not be eliminated. rdar://problem/23557469 Differential Revision: http://reviews.llvm.org/D15340 llvm-svn: 255353
2015-12-11 19:24:30 +01:00
continue;
// Bail out if the exit block is not Return nor Unreachable.
FuncInfo->SplitCSR = false;
break;
}
}
MachineBasicBlock *EntryMBB = &MF->front();
if (FuncInfo->SplitCSR)
// This performs initialization so lowering for SplitCSR will be correct.
TLI->initializeSplitCSR(EntryMBB);
Delete unused arguments. llvm-svn: 101275
2010-04-14 22:05:00 +02:00
SelectAllBasicBlocks(Fn);
[FastISel] Add a diagnostic to warm on fallback. This is consistent with what we do for GlobalISel. That way, it is easy to see whether or not FastISel is able to fully select a function. At some point we may want to switch that to an optimization remark. llvm-svn: 294970
2017-02-13 18:38:59 +01:00
if (FastISelFailed && EnableFastISelFallbackReport) {
DiagnosticInfoISelFallback DiagFallback(Fn);
Fn.getContext().diagnose(DiagFallback);
}
Remove trailing whitespace llvm-svn: 21420
2005-04-22 00:36:52 +02:00
Fix whitespace. llvm-svn: 102817
2010-05-01 02:33:28 +02:00
// If the first basic block in the function has live ins that need to be
Move the code that inserts copies for function livein registers out of ScheduleDAGEmit.cpp and into SelectionDAGISel.cpp. This allows it to be run exactly once per function, even if multiple SelectionDAG iterations happen in the entry block, as may happen with FastISel. llvm-svn: 55863
2008-09-06 00:59:21 +02:00
// copied into vregs, emit the copies into the top of the block before
// emitting the code for the block.
Grab the TargetRegisterInfo off of the subtarget from the MachineFunction rather than a lookup on the TargetMachine to avoid unnecessary lookups. llvm-svn: 219291
2014-10-08 11:50:52 +02:00
const TargetRegisterInfo &TRI = *MF->getSubtarget().getRegisterInfo();
Cache SelectionDAGISel TargetInstrInfo lookups on the class and propagate. Also use the TargetSubtargetInfo and the MachineFunction and move TargetRegisterInfo query closer to uses. llvm-svn: 219273
2014-10-08 03:58:03 +02:00
RegInfo->EmitLiveInCopies(EntryMBB, TRI, *TII);
Replace r102368 with code that's less fragile. This creates DBG_VALUE instructions for function arguments early and insert them after instruction selection is done. llvm-svn: 102554
2010-04-29 01:08:54 +02:00
CXX_FAST_TLS calling convention: target independent portion. The access function has a short entry and a short exit, the initialization block is only run the first time. To improve the performance, we want to have a short frame at the entry and exit. We explicitly handle most of the CSRs via copies. Only the CSRs that are not handled via copies will be in CSR_SaveList. Frame lowering and prologue/epilogue insertion will generate a short frame in the entry and exit according to CSR_SaveList. The majority of the CSRs will be handled by register allcoator. Register allocator will try to spill and reload them in the initialization block. We add CSRsViaCopy, it will be explicitly handled during lowering. 1> we first set FunctionLoweringInfo->SplitCSR if conditions are met (the target supports it for the given calling convention and the function has only return exits). We also call TLI->initializeSplitCSR to perform initialization. 2> we call TLI->insertCopiesSplitCSR to insert copies from CSRsViaCopy to virtual registers at beginning of the entry block and copies from virtual registers to CSRsViaCopy at beginning of the exit blocks. 3> we also need to make sure the explicit copies will not be eliminated. rdar://problem/23557469 Differential Revision: http://reviews.llvm.org/D15340 llvm-svn: 255353
2015-12-11 19:24:30 +01:00
// Insert copies in the entry block and the return blocks.
CXX TLS: collect return blocks after SelectAllBasicBlocks. It is incorrect to get the corresponding MBB for a ReturnInst before SelectAllBasicBlocks since SelectAllBasicBlocks can change the correspondence between a ReturnInst and the MBB it is in. PR27062 llvm-svn: 264358
2016-03-25 00:21:29 +01:00
if (FuncInfo->SplitCSR) {
SmallVector<MachineBasicBlock*, 4> Returns;
// Collect all the return blocks.
for (MachineBasicBlock &MBB : mf) {
if (!MBB.succ_empty())
continue;
MachineBasicBlock::iterator Term = MBB.getFirstTerminator();
if (Term != MBB.end() && Term->isReturn()) {
Returns.push_back(&MBB);
continue;
}
}
CXX_FAST_TLS calling convention: target independent portion. The access function has a short entry and a short exit, the initialization block is only run the first time. To improve the performance, we want to have a short frame at the entry and exit. We explicitly handle most of the CSRs via copies. Only the CSRs that are not handled via copies will be in CSR_SaveList. Frame lowering and prologue/epilogue insertion will generate a short frame in the entry and exit according to CSR_SaveList. The majority of the CSRs will be handled by register allcoator. Register allocator will try to spill and reload them in the initialization block. We add CSRsViaCopy, it will be explicitly handled during lowering. 1> we first set FunctionLoweringInfo->SplitCSR if conditions are met (the target supports it for the given calling convention and the function has only return exits). We also call TLI->initializeSplitCSR to perform initialization. 2> we call TLI->insertCopiesSplitCSR to insert copies from CSRsViaCopy to virtual registers at beginning of the entry block and copies from virtual registers to CSRsViaCopy at beginning of the exit blocks. 3> we also need to make sure the explicit copies will not be eliminated. rdar://problem/23557469 Differential Revision: http://reviews.llvm.org/D15340 llvm-svn: 255353
2015-12-11 19:24:30 +01:00
TLI->insertCopiesSplitCSR(EntryMBB, Returns);
CXX TLS: collect return blocks after SelectAllBasicBlocks. It is incorrect to get the corresponding MBB for a ReturnInst before SelectAllBasicBlocks since SelectAllBasicBlocks can change the correspondence between a ReturnInst and the MBB it is in. PR27062 llvm-svn: 264358
2016-03-25 00:21:29 +01:00
}
CXX_FAST_TLS calling convention: target independent portion. The access function has a short entry and a short exit, the initialization block is only run the first time. To improve the performance, we want to have a short frame at the entry and exit. We explicitly handle most of the CSRs via copies. Only the CSRs that are not handled via copies will be in CSR_SaveList. Frame lowering and prologue/epilogue insertion will generate a short frame in the entry and exit according to CSR_SaveList. The majority of the CSRs will be handled by register allcoator. Register allocator will try to spill and reload them in the initialization block. We add CSRsViaCopy, it will be explicitly handled during lowering. 1> we first set FunctionLoweringInfo->SplitCSR if conditions are met (the target supports it for the given calling convention and the function has only return exits). We also call TLI->initializeSplitCSR to perform initialization. 2> we call TLI->insertCopiesSplitCSR to insert copies from CSRsViaCopy to virtual registers at beginning of the entry block and copies from virtual registers to CSRsViaCopy at beginning of the exit blocks. 3> we also need to make sure the explicit copies will not be eliminated. rdar://problem/23557469 Differential Revision: http://reviews.llvm.org/D15340 llvm-svn: 255353
2015-12-11 19:24:30 +01:00
Update debug info when live-in reg is copied into a vreg. llvm-svn: 104732
2010-05-26 22:18:50 +02:00
DenseMap<unsigned, unsigned> LiveInMap;
if (!FuncInfo->ArgDbgValues.empty())
for (MachineRegisterInfo::livein_iterator LI = RegInfo->livein_begin(),
E = RegInfo->livein_end(); LI != E; ++LI)
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
if (LI->second)
Update debug info when live-in reg is copied into a vreg. llvm-svn: 104732
2010-05-26 22:18:50 +02:00
LiveInMap.insert(std::make_pair(LI->first, LI->second));
Replace r102368 with code that's less fragile. This creates DBG_VALUE instructions for function arguments early and insert them after instruction selection is done. llvm-svn: 102554
2010-04-29 01:08:54 +02:00
// Insert DBG_VALUE instructions for function arguments to the entry block.
for (unsigned i = 0, e = FuncInfo->ArgDbgValues.size(); i != e; ++i) {
MachineInstr *MI = FuncInfo->ArgDbgValues[e-i-1];
Debug Info: Simplify Frame Index handling in DBG_VALUE Machine Instructions Rather than using the full power of target-specific addressing modes in DBG_VALUEs with Frame Indicies, simply use Frame Index + Offset. This reduces the complexity of debug info handling down to two representations of values (reg+offset and frame index+offset) rather than three or four. Ideally we could ensure that frame indicies had been eliminated by the time we reached an assembly or dwarf generation, but I haven't spent the time to figure out where the FIs are leaking through into that & whether there's a good place to convert them. Some FI+offset=>reg+offset conversion is done (see PrologEpilogInserter, for example) which is necessary for some SelectionDAG assumptions about registers, I believe, but it might be possible to make this a more thorough conversion & ensure there are no remaining FIs no matter how instruction selection is performed. llvm-svn: 184066
2013-06-16 22:34:15 +02:00
bool hasFI = MI->getOperand(0).isFI();
long line correction llvm-svn: 195179
2013-11-20 01:32:32 +01:00
unsigned Reg =
hasFI ? TRI.getFrameRegister(*MF) : MI->getOperand(0).getReg();
Replace r102368 with code that's less fragile. This creates DBG_VALUE instructions for function arguments early and insert them after instruction selection is done. llvm-svn: 102554
2010-04-29 01:08:54 +02:00
if (TargetRegisterInfo::isPhysicalRegister(Reg))
EntryMBB->insert(EntryMBB->begin(), MI);
else {
MachineInstr *Def = RegInfo->getVRegDef(Reg);
Debug info: Don't crash in SelectionDAGISel when a vreg that is being pointed to by a dbg_value belonging to a function argument is eliminated during instruction selection. rdar://problem/15094721. llvm-svn: 192011
2013-10-05 02:08:27 +02:00
if (Def) {
MachineBasicBlock::iterator InsertPos = Def;
// FIXME: VR def may not be in entry block.
[C++11] Replace llvm::next and llvm::prior with std::next and std::prev. Remove the old functions. llvm-svn: 202636
2014-03-02 13:27:27 +01:00
Def->getParent()->insert(std::next(InsertPos), MI);
Debug info: Don't crash in SelectionDAGISel when a vreg that is being pointed to by a dbg_value belonging to a function argument is eliminated during instruction selection. rdar://problem/15094721. llvm-svn: 192011
2013-10-05 02:08:27 +02:00
} else
DEBUG(dbgs() << "Dropping debug info for dead vreg"
<< TargetRegisterInfo::virtReg2Index(Reg) << "\n");
Replace r102368 with code that's less fragile. This creates DBG_VALUE instructions for function arguments early and insert them after instruction selection is done. llvm-svn: 102554
2010-04-29 01:08:54 +02:00
}
Update debug info when live-in reg is copied into a vreg. llvm-svn: 104732
2010-05-26 22:18:50 +02:00
// If Reg is live-in then update debug info to track its copy in a vreg.
DenseMap<unsigned, unsigned>::iterator LDI = LiveInMap.find(Reg);
if (LDI != LiveInMap.end()) {
Debug Info: Simplify Frame Index handling in DBG_VALUE Machine Instructions Rather than using the full power of target-specific addressing modes in DBG_VALUEs with Frame Indicies, simply use Frame Index + Offset. This reduces the complexity of debug info handling down to two representations of values (reg+offset and frame index+offset) rather than three or four. Ideally we could ensure that frame indicies had been eliminated by the time we reached an assembly or dwarf generation, but I haven't spent the time to figure out where the FIs are leaking through into that & whether there's a good place to convert them. Some FI+offset=>reg+offset conversion is done (see PrologEpilogInserter, for example) which is necessary for some SelectionDAG assumptions about registers, I believe, but it might be possible to make this a more thorough conversion & ensure there are no remaining FIs no matter how instruction selection is performed. llvm-svn: 184066
2013-06-16 22:34:15 +02:00
assert(!hasFI && "There's no handling of frame pointer updating here yet "
"- add if needed");
Update debug info when live-in reg is copied into a vreg. llvm-svn: 104732
2010-05-26 22:18:50 +02:00
MachineInstr *Def = RegInfo->getVRegDef(LDI->second);
MachineBasicBlock::iterator InsertPos = Def;
Move the complex address expression out of DIVariable and into an extra argument of the llvm.dbg.declare/llvm.dbg.value intrinsics. Previously, DIVariable was a variable-length field that has an optional reference to a Metadata array consisting of a variable number of complex address expressions. In the case of OpPiece expressions this is wasting a lot of storage in IR, because when an aggregate type is, e.g., SROA'd into all of its n individual members, the IR will contain n copies of the DIVariable, all alike, only differing in the complex address reference at the end. By making the complex address into an extra argument of the dbg.value/dbg.declare intrinsics, all of the pieces can reference the same variable and the complex address expressions can be uniqued across the CU, too. Down the road, this will allow us to move other flags, such as "indirection" out of the DIVariable, too. The new intrinsics look like this: declare void @llvm.dbg.declare(metadata %storage, metadata %var, metadata %expr) declare void @llvm.dbg.value(metadata %storage, i64 %offset, metadata %var, metadata %expr) This patch adds a new LLVM-local tag to DIExpressions, so we can detect and pretty-print DIExpression metadata nodes. What this patch doesn't do: This patch does not touch the "Indirect" field in DIVariable; but moving that into the expression would be a natural next step. http://reviews.llvm.org/D4919 rdar://problem/17994491 Thanks to dblaikie and dexonsmith for reviewing this patch! Note: I accidentally committed a bogus older version of this patch previously. llvm-svn: 218787
2014-10-01 20:55:02 +02:00
const MDNode *Variable = MI->getDebugVariable();
const MDNode *Expr = MI->getDebugExpression();
CodeGen: Assert that inlined-at locations agree As a follow-up to r234021, assert that a debug info intrinsic variable's `MDLocalVariable::getInlinedAt()` always matches the `MDLocation::getInlinedAt()` of its `!dbg` attachment. The goal here is to get rid of `MDLocalVariable::getInlinedAt()` entirely (PR22778), but I'll let these assertions bake for a while first. If you have an out-of-tree backend that just broke, you're probably attaching the wrong `DebugLoc` to a `DBG_VALUE` instruction. The one you want is the location that was attached to the corresponding `@llvm.dbg.declare` or `@llvm.dbg.value` call that you started with. llvm-svn: 234038
2015-04-03 21:20:26 +02:00
DebugLoc DL = MI->getDebugLoc();
Debug info: Fix PR16736 and rdar://problem/14990587. A DBG_VALUE is register-indirect iff the first operand is a register _and_ the second operand is an immediate. llvm-svn: 190821
2013-09-17 01:29:03 +02:00
bool IsIndirect = MI->isIndirectDebugValue();
Reapply an improved version of r180816/180817. Change the informal convention of DBG_VALUE machine instructions so that we can express a register-indirect address with an offset of 0. The old convention was that a DBG_VALUE is a register-indirect value if the offset (operand 1) is nonzero. The new convention is that a DBG_VALUE is register-indirect if the first operand is a register and the second operand is an immediate. For plain register values the combination reg, reg is used. MachineInstrBuilder::BuildMI knows how to build the new DBG_VALUES. rdar://problem/13658587 llvm-svn: 185966
2013-07-09 22:28:37 +02:00
unsigned Offset = IsIndirect ? MI->getOperand(1).getImm() : 0;
IR: Give 'DI' prefix to debug info metadata Finish off PR23080 by renaming the debug info IR constructs from `MD*` to `DI*`. The last of the `DIDescriptor` classes were deleted in r235356, and the last of the related typedefs removed in r235413, so this has all baked for about a week. Note: If you have out-of-tree code (like a frontend), I recommend that you get everything compiling and tests passing with the *previous* commit before updating to this one. It'll be easier to keep track of what code is using the `DIDescriptor` hierarchy and what you've already updated, and I think you're extremely unlikely to insert bugs. YMMV of course. Back to *this* commit: I did this using the rename-md-di-nodes.sh upgrade script I've attached to PR23080 (both code and testcases) and filtered through clang-format-diff.py. I edited the tests for test/Assembler/invalid-generic-debug-node-*.ll by hand since the columns were off-by-three. It should work on your out-of-tree testcases (and code, if you've followed the advice in the previous paragraph). Some of the tests are in badly named files now (e.g., test/Assembler/invalid-mdcompositetype-missing-tag.ll should be 'dicompositetype'); I'll come back and move the files in a follow-up commit. llvm-svn: 236120
2015-04-29 18:38:44 +02:00
assert(cast<DILocalVariable>(Variable)->isValidLocationForIntrinsic(DL) &&
CodeGen: Assert that inlined-at locations agree As a follow-up to r234021, assert that a debug info intrinsic variable's `MDLocalVariable::getInlinedAt()` always matches the `MDLocation::getInlinedAt()` of its `!dbg` attachment. The goal here is to get rid of `MDLocalVariable::getInlinedAt()` entirely (PR22778), but I'll let these assertions bake for a while first. If you have an out-of-tree backend that just broke, you're probably attaching the wrong `DebugLoc` to a `DBG_VALUE` instruction. The one you want is the location that was attached to the corresponding `@llvm.dbg.declare` or `@llvm.dbg.value` call that you started with. llvm-svn: 234038
2015-04-03 21:20:26 +02:00
"Expected inlined-at fields to agree");
Update debug info when live-in reg is copied into a vreg. llvm-svn: 104732
2010-05-26 22:18:50 +02:00
// Def is never a terminator here, so it is ok to increment InsertPos.
CodeGen: Assert that inlined-at locations agree As a follow-up to r234021, assert that a debug info intrinsic variable's `MDLocalVariable::getInlinedAt()` always matches the `MDLocation::getInlinedAt()` of its `!dbg` attachment. The goal here is to get rid of `MDLocalVariable::getInlinedAt()` entirely (PR22778), but I'll let these assertions bake for a while first. If you have an out-of-tree backend that just broke, you're probably attaching the wrong `DebugLoc` to a `DBG_VALUE` instruction. The one you want is the location that was attached to the corresponding `@llvm.dbg.declare` or `@llvm.dbg.value` call that you started with. llvm-svn: 234038
2015-04-03 21:20:26 +02:00
BuildMI(*EntryMBB, ++InsertPos, DL, TII->get(TargetOpcode::DBG_VALUE),
IsIndirect, LDI->second, Offset, Variable, Expr);
If only user of a vreg is an copy instruction to export copy of vreg out of current basic block then insert DBG_VALUE so that debug value of the variable is also transfered to new vreg. Testcase is in r114476. This fixes radar 8412415. llvm-svn: 114478
2010-09-21 22:56:33 +02:00
// If this vreg is directly copied into an exported register then
// that COPY instructions also need DBG_VALUE, if it is the only
// user of LDI->second.
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
MachineInstr *CopyUseMI = nullptr;
Phase 1 of refactoring the MachineRegisterInfo iterators to make them suitable for use with C++11 range-based for-loops. The gist of phase 1 is to remove the skipInstruction() and skipBundle() methods from these iterators, instead splitting each iterator into a version that walks operands, a version that walks instructions, and a version that walks bundles. This has the result of making some "clever" loops in lib/CodeGen more verbose, but also makes their iterator invalidation characteristics much more obvious to the casual reader. (Making them concise again in the future is a good motivating case for a pre-incrementing range adapter!) Phase 2 of this undertaking with consist of removing the getOperand() method, and changing operator*() of the operand-walker to return a MachineOperand&. At that point, it should be possible to add range views for them that work as one might expect. llvm-svn: 203757
2014-03-13 07:02:25 +01:00
for (MachineRegisterInfo::use_instr_iterator
UI = RegInfo->use_instr_begin(LDI->second),
E = RegInfo->use_instr_end(); UI != E; ) {
MachineInstr *UseMI = &*(UI++);
If only user of a vreg is an copy instruction to export copy of vreg out of current basic block then insert DBG_VALUE so that debug value of the variable is also transfered to new vreg. Testcase is in r114476. This fixes radar 8412415. llvm-svn: 114478
2010-09-21 22:56:33 +02:00
if (UseMI->isDebugValue()) continue;
if (UseMI->isCopy() && !CopyUseMI && UseMI->getParent() == EntryMBB) {
CopyUseMI = UseMI; continue;
}
// Otherwise this is another use or second copy use.
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
CopyUseMI = nullptr; break;
If only user of a vreg is an copy instruction to export copy of vreg out of current basic block then insert DBG_VALUE so that debug value of the variable is also transfered to new vreg. Testcase is in r114476. This fixes radar 8412415. llvm-svn: 114478
2010-09-21 22:56:33 +02:00
}
if (CopyUseMI) {
CodeGen: Assert that inlined-at locations agree As a follow-up to r234021, assert that a debug info intrinsic variable's `MDLocalVariable::getInlinedAt()` always matches the `MDLocation::getInlinedAt()` of its `!dbg` attachment. The goal here is to get rid of `MDLocalVariable::getInlinedAt()` entirely (PR22778), but I'll let these assertions bake for a while first. If you have an out-of-tree backend that just broke, you're probably attaching the wrong `DebugLoc` to a `DBG_VALUE` instruction. The one you want is the location that was attached to the corresponding `@llvm.dbg.declare` or `@llvm.dbg.value` call that you started with. llvm-svn: 234038
2015-04-03 21:20:26 +02:00
// Use MI's debug location, which describes where Variable was
// declared, rather than whatever is attached to CopyUseMI.
If only user of a vreg is an copy instruction to export copy of vreg out of current basic block then insert DBG_VALUE so that debug value of the variable is also transfered to new vreg. Testcase is in r114476. This fixes radar 8412415. llvm-svn: 114478
2010-09-21 22:56:33 +02:00
MachineInstr *NewMI =
CodeGen: Assert that inlined-at locations agree As a follow-up to r234021, assert that a debug info intrinsic variable's `MDLocalVariable::getInlinedAt()` always matches the `MDLocation::getInlinedAt()` of its `!dbg` attachment. The goal here is to get rid of `MDLocalVariable::getInlinedAt()` entirely (PR22778), but I'll let these assertions bake for a while first. If you have an out-of-tree backend that just broke, you're probably attaching the wrong `DebugLoc` to a `DBG_VALUE` instruction. The one you want is the location that was attached to the corresponding `@llvm.dbg.declare` or `@llvm.dbg.value` call that you started with. llvm-svn: 234038
2015-04-03 21:20:26 +02:00
BuildMI(*MF, DL, TII->get(TargetOpcode::DBG_VALUE), IsIndirect,
Move the complex address expression out of DIVariable and into an extra argument of the llvm.dbg.declare/llvm.dbg.value intrinsics. Previously, DIVariable was a variable-length field that has an optional reference to a Metadata array consisting of a variable number of complex address expressions. In the case of OpPiece expressions this is wasting a lot of storage in IR, because when an aggregate type is, e.g., SROA'd into all of its n individual members, the IR will contain n copies of the DIVariable, all alike, only differing in the complex address reference at the end. By making the complex address into an extra argument of the dbg.value/dbg.declare intrinsics, all of the pieces can reference the same variable and the complex address expressions can be uniqued across the CU, too. Down the road, this will allow us to move other flags, such as "indirection" out of the DIVariable, too. The new intrinsics look like this: declare void @llvm.dbg.declare(metadata %storage, metadata %var, metadata %expr) declare void @llvm.dbg.value(metadata %storage, i64 %offset, metadata %var, metadata %expr) This patch adds a new LLVM-local tag to DIExpressions, so we can detect and pretty-print DIExpression metadata nodes. What this patch doesn't do: This patch does not touch the "Indirect" field in DIVariable; but moving that into the expression would be a natural next step. http://reviews.llvm.org/D4919 rdar://problem/17994491 Thanks to dblaikie and dexonsmith for reviewing this patch! Note: I accidentally committed a bogus older version of this patch previously. llvm-svn: 218787
2014-10-01 20:55:02 +02:00
CopyUseMI->getOperand(0).getReg(), Offset, Variable, Expr);
First chunk of MachineInstr bundle support. 1. Added opcode BUNDLE 2. Taught MachineInstr class to deal with bundled MIs 3. Changed MachineBasicBlock iterator to skip over bundled MIs; added an iterator to walk all the MIs 4. Taught MachineBasicBlock methods about bundled MIs llvm-svn: 145975
2011-12-06 23:12:01 +01:00
MachineBasicBlock::iterator Pos = CopyUseMI;
EntryMBB->insertAfter(Pos, NewMI);
If only user of a vreg is an copy instruction to export copy of vreg out of current basic block then insert DBG_VALUE so that debug value of the variable is also transfered to new vreg. Testcase is in r114476. This fixes radar 8412415. llvm-svn: 114478
2010-09-21 22:56:33 +02:00
}
Update debug info when live-in reg is copied into a vreg. llvm-svn: 104732
2010-05-26 22:18:50 +02:00
}
Replace r102368 with code that's less fragile. This creates DBG_VALUE instructions for function arguments early and insert them after instruction selection is done. llvm-svn: 102554
2010-04-29 01:08:54 +02:00
}
- Set the "HasCalls" flag after instruction selection is finished. - Change the logic DisableFramePointerElim() to check for the -disable-non-leaf-fp-elim before -disable-fp-elim. llvm-svn: 103990
2010-05-18 01:09:50 +02:00
// Determine if there are any calls in this machine function.
MachineFunction: Return reference for getFrameInfo(); NFC getFrameInfo() never returns nullptr so we should use a reference instead of a pointer. llvm-svn: 277017
2016-07-28 20:40:00 +02:00
MachineFrameInfo &MFI = MF->getFrameInfo();
Convert more loops to range-based equivalents llvm-svn: 207714
2014-05-01 00:17:38 +02:00
for (const auto &MBB : *MF) {
MachineFunction: Return reference for getFrameInfo(); NFC getFrameInfo() never returns nullptr so we should use a reference instead of a pointer. llvm-svn: 277017
2016-07-28 20:40:00 +02:00
if (MFI.hasCalls() && MF->hasInlineAsm())
[ms-inline asm] Do not omit the frame pointer if we have ms-inline assembly. If the frame pointer is omitted, and any stack changes occur in the inline assembly, e.g.: "pusha", then any C local variable or C argument references will be incorrect. I pass no judgement on anyone who would do such a thing. ;) rdar://13218191 llvm-svn: 175334
2013-02-16 02:25:28 +01:00
break;
Convert more loops to range-based equivalents llvm-svn: 207714
2014-05-01 00:17:38 +02:00
for (const auto &MI : MBB) {
Cache SelectionDAGISel TargetInstrInfo lookups on the class and propagate. Also use the TargetSubtargetInfo and the MachineFunction and move TargetRegisterInfo query closer to uses. llvm-svn: 219273
2014-10-08 03:58:03 +02:00
const MCInstrDesc &MCID = TII->get(MI.getOpcode());
[ms-inline asm] Do not omit the frame pointer if we have ms-inline assembly. If the frame pointer is omitted, and any stack changes occur in the inline assembly, e.g.: "pusha", then any C local variable or C argument references will be incorrect. I pass no judgement on anyone who would do such a thing. ;) rdar://13218191 llvm-svn: 175334
2013-02-16 02:25:28 +01:00
if ((MCID.isCall() && !MCID.isReturn()) ||
Convert more loops to range-based equivalents llvm-svn: 207714
2014-05-01 00:17:38 +02:00
MI.isStackAligningInlineAsm()) {
MachineFunction: Return reference for getFrameInfo(); NFC getFrameInfo() never returns nullptr so we should use a reference instead of a pointer. llvm-svn: 277017
2016-07-28 20:40:00 +02:00
MFI.setHasCalls(true);
[ms-inline asm] Do not omit the frame pointer if we have ms-inline assembly. If the frame pointer is omitted, and any stack changes occur in the inline assembly, e.g.: "pusha", then any C local variable or C argument references will be incorrect. I pass no judgement on anyone who would do such a thing. ;) rdar://13218191 llvm-svn: 175334
2013-02-16 02:25:28 +01:00
}
Convert more loops to range-based equivalents llvm-svn: 207714
2014-05-01 00:17:38 +02:00
if (MI.isInlineAsm()) {
Reland "Fix miscompile of MS inline assembly with stack realignment" This re-lands commit r196876, which was reverted in r196879. The tests have been fixed to pass on platforms with a stack alignment larger than 4. Update to clang side tests will land shortly. llvm-svn: 196939
2013-12-10 19:27:32 +01:00
MF->setHasInlineAsm(true);
- Set the "HasCalls" flag after instruction selection is finished. - Change the logic DisableFramePointerElim() to check for the -disable-non-leaf-fp-elim before -disable-fp-elim. llvm-svn: 103990
2010-05-18 01:09:50 +02:00
}
}
}
Move the check for "calls setjmp" to SelectionDAGISel so that it can be used by more than just the stack slot coloring algorithm. llvm-svn: 104722
2010-05-26 21:46:12 +02:00
// Determine if there is a call to setjmp in the machine function.
Allow inlining of functions with returns_twice calls, if they have the attribute themselve. llvm-svn: 146851
2011-12-18 21:35:43 +01:00
MF->setExposesReturnsTwice(Fn.callsFunctionThatReturnsTwice());
Move the check for "calls setjmp" to SelectionDAGISel so that it can be used by more than just the stack slot coloring algorithm. llvm-svn: 104722
2010-05-26 21:46:12 +02:00
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
// Replace forward-declared registers with the registers containing
// the desired value.
MachineRegisterInfo &MRI = MF->getRegInfo();
for (DenseMap<unsigned, unsigned>::iterator
I = FuncInfo->RegFixups.begin(), E = FuncInfo->RegFixups.end();
I != E; ++I) {
unsigned From = I->first;
unsigned To = I->second;
// If To is also scheduled to be replaced, find what its ultimate
// replacement is.
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
while (true) {
Add some constantness. No functionality change. llvm-svn: 155859
2012-05-01 01:41:30 +02:00
DenseMap<unsigned, unsigned>::iterator J = FuncInfo->RegFixups.find(To);
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
if (J == E) break;
To = J->second;
}
ARM: Fix more fast-isel verifier failures. Teach the generic instruction selection helper functions to constrain the register classes of their input operands. For non-physical register references, the generic code needs to be careful not to mess that up when replacing references to result registers. As the comment indicates for MachineRegisterInfo::replaceRegWith(), it's important to call constrainRegClass() first. rdar://12594152 llvm-svn: 188593
2013-08-17 01:37:31 +02:00
// Make sure the new register has a sufficiently constrained register class.
if (TargetRegisterInfo::isVirtualRegister(From) &&
TargetRegisterInfo::isVirtualRegister(To))
MRI.constrainRegClass(To, MRI.getRegClass(From));
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
// Replace it.
[Fast-ISel] Clear kill flags on registers replaced by updateValueMap. When selecting an extract instruction, we don't actually generate code but instead work out which register we are reading, and rewrite uses of the extract def to the source register. This is done via updateValueMap,. However, its possible that the source register we are rewriting *to* to also have uses. If those uses are after a kill of the value we are rewriting *from* then we have uses after a kill and the verifier fails. This code checks for the case where the to register is also used, and if so it clears all kill on the from register. This is conservative, but better that always clearing kills on the from register. llvm-svn: 236897
2015-05-08 22:46:54 +02:00
// Replacing one register with another won't touch the kill flags.
// We need to conservatively clear the kill flags as a kill on the old
// register might dominate existing uses of the new register.
if (!MRI.use_empty(To))
MRI.clearKillFlags(From);
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
MRI.replaceRegWith(From, To);
}
[X86] Make hasFP constant time We need a frame pointer if there is a push/pop sequence after the prologue in order to unwind the stack. Scanning the instructions to figure out if this happened made hasFP not constant-time which is a violation of expectations. Let's compute this up-front and reuse that computation when we need it. llvm-svn: 256730
2016-01-04 05:49:41 +01:00
if (TLI->hasCopyImplyingStackAdjustment(MF))
MachineFunction: Return reference for getFrameInfo(); NFC getFrameInfo() never returns nullptr so we should use a reference instead of a pointer. llvm-svn: 277017
2016-07-28 20:40:00 +02:00
MFI.setHasCopyImplyingStackAdjustment(true);
[X86] Make hasFP constant time We need a frame pointer if there is a push/pop sequence after the prologue in order to unwind the stack. Scanning the instructions to figure out if this happened made hasFP not constant-time which is a violation of expectations. Let's compute this up-front and reuse that computation when we need it. llvm-svn: 256730
2016-01-04 05:49:41 +01:00
Freeze the reserved registers as soon as isel is complete. Also provide an MRI::getReservedRegs() function to access the frozen register set, and isReserved() and isAllocatable() methods to test individual registers. The various implementations of TRI::getReservedRegs() are quite complicated, and many passes need to look at the reserved register set. This patch makes it possible for these passes to use the cached copy in MRI, avoiding a lot of malloc traffic and repeated calculations. llvm-svn: 165982
2012-10-15 23:33:06 +02:00
// Freeze the set of reserved registers now that MachineFrameInfo has been
// set up. All the information required by getReservedRegs() should be
// available now.
MRI.freezeReservedRegs(*MF);
Replace r102368 with code that's less fragile. This creates DBG_VALUE instructions for function arguments early and insert them after instruction selection is done. llvm-svn: 102554
2010-04-29 01:08:54 +02:00
// Release function-specific state. SDB and CurDAG are already cleared
// at this point.
FuncInfo->clear();
Move the code that inserts copies for function livein registers out of ScheduleDAGEmit.cpp and into SelectionDAGISel.cpp. This allows it to be run exactly once per function, even if multiple SelectionDAG iterations happen in the entry block, as may happen with FastISel. llvm-svn: 55863
2008-09-06 00:59:21 +02:00
Add a machine code print in DEBUG() following instruction selection. Make debugging ISel a bit easier by printing out a dump of the generated code at the end. llvm-svn: 197456
2013-12-17 03:01:10 +01:00
DEBUG(dbgs() << "*** MachineFunction at end of ISel ***\n");
DEBUG(MF->print(dbgs()));
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
return true;
}
fix an oversight which caused us to compile the testcase (and other less trivial things) into a dummy lea. Before we generated: _test: ## @test movq _G@GOTPCREL(%rip), %rax leaq (%rax), %rax ret now we produce: _test: ## @test movq _G@GOTPCREL(%rip), %rax ret This is part of rdar://9289558 llvm-svn: 129662
2011-04-17 19:12:08 +02:00
void SelectionDAGISel::SelectBasicBlock(BasicBlock::const_iterator Begin,
BasicBlock::const_iterator End,
bool &HadTailCall) {
SelectionDAGIsel: Fix comment about terminators being "handled below". That changed in r102128. llvm-svn: 232692
2015-03-19 01:02:22 +01:00
// Lower the instructions. If a call is emitted as a tail call, cease emitting
// nodes for this block.
Sink SelectionDAGBuilder's HandlePHINodesInSuccessorBlocks down into SelectionDAGBuilder itself. llvm-svn: 102128
2010-04-22 22:55:53 +02:00
for (BasicBlock::const_iterator I = Begin; I != End && !SDB->HasTailCall; ++I)
Refine further the scope where the global DebugLoc value is active. llvm-svn: 101443
2010-04-16 07:06:56 +02:00
SDB->visit(*I);
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
Don't call SelectionDAG.getRoot() directly, go through a forwarding method. llvm-svn: 19642
2005-01-17 20:43:36 +01:00
// Make sure the root of the DAG is up-to-date.
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
CurDAG->setRoot(SDB->getControlRoot());
HadTailCall = SDB->HasTailCall;
SDB->clear();
Reorgnaize more switch code lowering to clean up some tricky code, and to eliminate the need for the SelectionDAGBuilder state to be live during CodeGenAndEmitDAG calls. Call SDB->clear() before CodeGenAndEmitDAG calls instead of before it, and move the CurDAG->clear() out of SelectionDAGBuilder, which doesn't own the DAG, and into CodeGenAndEmitDAG. llvm-svn: 102814
2010-05-01 02:25:44 +02:00
// Final step, emit the lowered DAG as machine code.
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
CodeGenAndEmitDAG();
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
}
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
void SelectionDAGISel::ComputeLiveOutVRegInfo() {
Avoid overly large SmallPtrSet/SmallSet These sets perform linear searching in small mode so it is never a good idea to use SmallSize/N bigger than 32. llvm-svn: 259283
2016-01-30 02:24:31 +01:00
SmallPtrSet<SDNode*, 16> VisitedNodes;
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
SmallVector<SDNode*, 128> Worklist;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
erect abstraction boundaries for accessing SDValue members, rename Val -> Node to reflect semantics llvm-svn: 55504
2008-08-28 23:40:38 +02:00
Worklist.push_back(CurDAG->getRoot().getNode());
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
APInt KnownZero;
APInt KnownOne;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Use pop_back_val instead of back()+pop_back. llvm-svn: 92918
2010-01-07 18:27:56 +01:00
do {
SDNode *N = Worklist.pop_back_val();
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
// If we've already seen this node, ignore it.
Update SetVector to rely on the underlying set's insert to return a pair<iterator, bool> This is to be consistent with StringSet and ultimately with the standard library's associative container insert function. This lead to updating SmallSet::insert to return pair<iterator, bool>, and then to update SmallPtrSet::insert to return pair<iterator, bool>, and then to update all the existing users of those functions... llvm-svn: 222334
2014-11-19 08:49:26 +01:00
if (!VisitedNodes.insert(N).second)
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
continue;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
// Otherwise, add all chain operands to the worklist.
Convert a bunch of loops to foreach. NFC. This uses the new SDNode::op_values() iterator range committed in r240805. llvm-svn: 240822
2015-06-26 21:37:02 +02:00
for (const SDValue &Op : N->op_values())
if (Op.getValueType() == MVT::Other)
Worklist.push_back(Op.getNode());
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
// If this is a CopyToReg with a vreg dest, process it.
if (N->getOpcode() != ISD::CopyToReg)
continue;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
unsigned DestReg = cast<RegisterSDNode>(N->getOperand(1))->getReg();
if (!TargetRegisterInfo::isVirtualRegister(DestReg))
continue;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
// Ignore non-scalar or non-integer values.
Rename SDOperand to SDValue. llvm-svn: 54128
2008-07-27 23:46:04 +02:00
SDValue Src = N->getOperand(2);
Rename MVT to EVT, in preparation for splitting SimpleValueType out into its own struct type. llvm-svn: 78610
2009-08-11 00:56:29 +02:00
EVT SrcVT = Src.getValueType();
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
if (!SrcVT.isInteger() || SrcVT.isVector())
continue;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
unsigned NumSignBits = CurDAG->ComputeNumSignBits(Src);
Rename ComputeMaskedBits to computeKnownBits. "Masked" has been inappropriate since it lost its Mask parameter in r154011. llvm-svn: 208811
2014-05-14 23:14:37 +02:00
CurDAG->computeKnownBits(Src, KnownZero, KnownOne);
Refactor the LiveOutInfo interface into a few methods on FunctionLoweringInfo and make the actual map private. llvm-svn: 126376
2011-02-24 11:00:08 +01:00
FuncInfo->AddLiveOutRegInfo(DestReg, NumSignBits, KnownZero, KnownOne);
Use pop_back_val instead of back()+pop_back. llvm-svn: 92918
2010-01-07 18:27:56 +01:00
} while (!Worklist.empty());
add a new -enable-value-prop flag for llcbeta, that enables propagation of value info (sign/zero ext info) from one MBB to another. This doesn't handle much right now because of two limitations: 1) only handles zext/sext, not random bit propagation (no assert exists for this) 2) doesn't handle phis. llvm-svn: 52383
2008-06-17 08:09:18 +02:00
}
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
void SelectionDAGISel::CodeGenAndEmitDAG() {
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
StringRef GroupName = "sdag";
StringRef GroupDescription = "Instruction Selection and Scheduling";
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
std::string BlockName;
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
int BlockNumber = -1;
Silence a bunch (but not all) "variable written but not read" warnings when building with assertions disabled. llvm-svn: 137460
2011-08-12 16:54:45 +02:00
(void)BlockNumber;
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
bool MatchFilterBB = false; (void)MatchFilterBB;
[DAGCombine] Allow vector constant folding of any value type before type legalization The patch comes in 2 parts: 1 - it makes use of the SelectionDAG::NewNodesMustHaveLegalTypes flag to tell when it can safely constant fold illegal types. 2 - it correctly resets SelectionDAG::NewNodesMustHaveLegalTypes at the start of each call to SelectionDAGISel::CodeGenAndEmitDAG so all the pre-legalization stages can make use of it - not just the first basic block that gets handled. Fix for PR30760 Differential Revision: https://reviews.llvm.org/D29568 llvm-svn: 294749
2017-02-10 15:37:25 +01:00
// Pre-type legalization allow creation of any node types.
CurDAG->NewNodesMustHaveLegalTypes = false;
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
#ifndef NDEBUG
Fix SelectionDAG -view-*-dags filtering llvm-svn: 226163
2015-01-15 13:03:32 +01:00
MatchFilterBB = (FilterDAGBasicBlockName.empty() ||
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
FilterDAGBasicBlockName ==
FuncInfo->MBB->getBasicBlock()->getName().str());
#endif
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
#ifdef NDEBUG
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
if (ViewDAGCombine1 || ViewLegalizeTypesDAGs || ViewLegalizeDAGs ||
If the type legalizer actually legalized anything (this doesn't happen that often, since most code does not use illegal types) then follow it by a DAG combiner run that is allowed to generate illegal operations but not illegal types. I didn't modify the target combiner code to distinguish like this between illegal operations and illegal types, so it will not produce illegal operations as well as not producing illegal types. llvm-svn: 59960
2008-11-24 15:53:14 +01:00
ViewDAGCombine2 || ViewDAGCombineLT || ViewISelDAGs || ViewSchedDAGs ||
ViewSUnitDAGs)
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
#endif
{
BlockNumber = FuncInfo->MBB->getNumber();
Remove superfluous .str() and replace std::string concatenation with Twine. llvm-svn: 233392
2015-03-27 18:51:30 +01:00
BlockName =
(MF->getName() + ":" + FuncInfo->MBB->getBasicBlock()->getName()).str();
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
}
DEBUG(dbgs() << "Initial selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
if (ViewDAGCombine1 && MatchFilterBB)
CurDAG->viewGraph("dag-combine1 input for " + BlockName);
In -debug mode, dump SelectionDAGs both before and after the optimization passes. llvm-svn: 42749
2007-10-08 17:12:17 +02:00
Enable Nate's excellent DAG combiner work by default. This allows the removal of a bunch of ad-hoc and crufty code from SelectionDAG.cpp. llvm-svn: 23682
2005-10-10 18:47:10 +02:00
// Run the DAG combiner in pre-legalize mode.
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("combine1", "DAG Combining 1", GroupName,
GroupDescription, TimePassesIsEnabled);
Don't try to form pre/post-indexed loads/stores until after LegalizeDAG runs. Fixes PR11029. llvm-svn: 144438
2011-11-12 01:35:34 +01:00
CurDAG->Combine(BeforeLegalizeTypes, *AA, OptLevel);
Apply Chris' suggestion. llvm-svn: 52970
2008-07-01 19:59:20 +02:00
}
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
DEBUG(dbgs() << "Optimized lowered selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
// Second step, hack on the DAG until it only uses operations and types that
// the target supports.
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
if (ViewLegalizeTypesDAGs && MatchFilterBB)
CurDAG->viewGraph("legalize-types input for " + BlockName);
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
bool Changed;
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("legalize_types", "Type Legalization", GroupName,
GroupDescription, TimePassesIsEnabled);
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
Changed = CurDAG->LegalizeTypes();
}
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
DEBUG(dbgs() << "Type-legalized selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
[DAGCombine] Allow vector constant folding of any value type before type legalization The patch comes in 2 parts: 1 - it makes use of the SelectionDAG::NewNodesMustHaveLegalTypes flag to tell when it can safely constant fold illegal types. 2 - it correctly resets SelectionDAG::NewNodesMustHaveLegalTypes at the start of each call to SelectionDAGISel::CodeGenAndEmitDAG so all the pre-legalization stages can make use of it - not just the first basic block that gets handled. Fix for PR30760 Differential Revision: https://reviews.llvm.org/D29568 llvm-svn: 294749
2017-02-10 15:37:25 +01:00
// Only allow creation of legal node types.
Fixed tryFoldToZero() for vector types that need expansion. Summary: Moved the requirement for SelectionDAG::getConstant() to return legally typed nodes slightly earlier. There were two optional DAGCombine passes that were missed out and were required to produce type-legal DAGs. Simplified a code-path in tryFoldToZero() to use SelectionDAG::getConstant(). This provides support for both promoted and expanded vector types whereas the previous code only supported promoted vector types. Fixes a "Type for zero vector elements is not legal" assertion detected by an llvm-stress generated test. Reviewers: resistor CC: llvm-commits Differential Revision: http://llvm-reviews.chandlerc.com/D2251 llvm-svn: 195635
2013-11-25 12:14:43 +01:00
CurDAG->NewNodesMustHaveLegalTypes = true;
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
if (Changed) {
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
if (ViewDAGCombineLT && MatchFilterBB)
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
CurDAG->viewGraph("dag-combine-lt input for " + BlockName);
// Run the DAG combiner in post-type-legalize mode.
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("combine_lt", "DAG Combining after legalize types",
GroupName, GroupDescription, TimePassesIsEnabled);
Don't try to form pre/post-indexed loads/stores until after LegalizeDAG runs. Fixes PR11029. llvm-svn: 144438
2011-11-12 01:35:34 +01:00
CurDAG->Combine(AfterLegalizeTypes, *AA, OptLevel);
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
}
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
DEBUG(dbgs() << "Optimized type-legalized selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
Track IR ordering of SelectionDAG nodes 3/4. Remove the old IR ordering mechanism and switch to new one. Fix unit test failures. llvm-svn: 182704
2013-05-25 05:08:10 +02:00
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
}
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("legalize_vec", "Vector Legalization", GroupName,
GroupDescription, TimePassesIsEnabled);
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
Changed = CurDAG->LegalizeVectors();
}
Add a new step to legalization to legalize vector math operations. This will allow simplifying LegalizeDAG to eliminate type legalization. (I have a patch to do that, but it's not quite finished; I'll commit it once it's finished and I've fixed any review comments for this patch.) See the comment at the beginning of lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp for more details on the motivation for this patch. llvm-svn: 72325
2009-05-23 14:35:30 +02:00
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
if (Changed) {
[SelectionDAG] Dump the DAG after legalizing vector ops and after the second type legalization Summary: With -debug, we aren't dumping the DAG after legalizing vector ops. In particular, on X86 with AVX1 only, we don't dump the DAG after we split 256-bit integer ops into pairs of 128-bit ADDs since this occurs during vector legalization. I'm only dumping if the legalize vector ops changes something since we don't print anything during legalize vector ops. So this dump shows up right after the first type-legalization dump happens. So if nothing changed this second dump is unnecessary. Having said that though, I think we should probably fix legalize vector ops to log what its doing. Reviewers: RKSimon, eli.friedman, spatel, arsenm, chandlerc Reviewed By: RKSimon Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D29554 llvm-svn: 294711
2017-02-10 06:05:57 +01:00
DEBUG(dbgs() << "Vector-legalized selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("legalize_types2", "Type Legalization 2", GroupName,
GroupDescription, TimePassesIsEnabled);
Remove dead store. llvm-svn: 92190
2009-12-28 02:51:30 +01:00
CurDAG->LegalizeTypes();
Add a new step to legalization to legalize vector math operations. This will allow simplifying LegalizeDAG to eliminate type legalization. (I have a patch to do that, but it's not quite finished; I'll commit it once it's finished and I've fixed any review comments for this patch.) See the comment at the beginning of lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp for more details on the motivation for this patch. llvm-svn: 72325
2009-05-23 14:35:30 +02:00
}
[SelectionDAG] Dump the DAG after legalizing vector ops and after the second type legalization Summary: With -debug, we aren't dumping the DAG after legalizing vector ops. In particular, on X86 with AVX1 only, we don't dump the DAG after we split 256-bit integer ops into pairs of 128-bit ADDs since this occurs during vector legalization. I'm only dumping if the legalize vector ops changes something since we don't print anything during legalize vector ops. So this dump shows up right after the first type-legalization dump happens. So if nothing changed this second dump is unnecessary. Having said that though, I think we should probably fix legalize vector ops to log what its doing. Reviewers: RKSimon, eli.friedman, spatel, arsenm, chandlerc Reviewed By: RKSimon Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D29554 llvm-svn: 294711
2017-02-10 06:05:57 +01:00
DEBUG(dbgs() << "Vector/type-legalized selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
if (ViewDAGCombineLT && MatchFilterBB)
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
CurDAG->viewGraph("dag-combine-lv input for " + BlockName);
Add a new step to legalization to legalize vector math operations. This will allow simplifying LegalizeDAG to eliminate type legalization. (I have a patch to do that, but it's not quite finished; I'll commit it once it's finished and I've fixed any review comments for this patch.) See the comment at the beginning of lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp for more details on the motivation for this patch. llvm-svn: 72325
2009-05-23 14:35:30 +02:00
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
// Run the DAG combiner in post-type-legalize mode.
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("combine_lv", "DAG Combining after legalize vectors",
GroupName, GroupDescription, TimePassesIsEnabled);
Don't try to form pre/post-indexed loads/stores until after LegalizeDAG runs. Fixes PR11029. llvm-svn: 144438
2011-11-12 01:35:34 +01:00
CurDAG->Combine(AfterLegalizeVectorOps, *AA, OptLevel);
Add a new step to legalization to legalize vector math operations. This will allow simplifying LegalizeDAG to eliminate type legalization. (I have a patch to do that, but it's not quite finished; I'll commit it once it's finished and I've fixed any review comments for this patch.) See the comment at the beginning of lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp for more details on the motivation for this patch. llvm-svn: 72325
2009-05-23 14:35:30 +02:00
}
Remove the unused DisableLegalizeTypes option and related code. llvm-svn: 90668
2009-12-05 18:51:33 +01:00
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
DEBUG(dbgs() << "Optimized vector-legalized selection DAG: BB#"
<< BlockNumber << " '" << BlockName << "'\n"; CurDAG->dump());
make legalize types be a command line option: -enable-legalize-types. llvm-svn: 53434
2008-07-11 01:37:50 +02:00
}
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
if (ViewLegalizeDAGs && MatchFilterBB)
CurDAG->viewGraph("legalize input for " + BlockName);
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("legalize", "DAG Legalization", GroupName,
GroupDescription, TimePassesIsEnabled);
Delete unused variables. llvm-svn: 131430
2011-05-17 00:19:54 +02:00
CurDAG->Legalize();
Apply Chris' suggestion. llvm-svn: 52970
2008-07-01 19:59:20 +02:00
}
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
DEBUG(dbgs() << "Legalized selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
if (ViewDAGCombine2 && MatchFilterBB)
CurDAG->viewGraph("dag-combine2 input for " + BlockName);
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
Enable Nate's excellent DAG combiner work by default. This allows the removal of a bunch of ad-hoc and crufty code from SelectionDAG.cpp. llvm-svn: 23682
2005-10-10 18:47:10 +02:00
// Run the DAG combiner in post-legalize mode.
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("combine2", "DAG Combining 2", GroupName,
GroupDescription, TimePassesIsEnabled);
Don't try to form pre/post-indexed loads/stores until after LegalizeDAG runs. Fixes PR11029. llvm-svn: 144438
2011-11-12 01:35:34 +01:00
CurDAG->Combine(AfterLegalizeDAG, *AA, OptLevel);
Apply Chris' suggestion. llvm-svn: 52970
2008-07-01 19:59:20 +02:00
}
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
DEBUG(dbgs() << "Optimized legalized selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
In -debug mode, dump SelectionDAGs both before and after the optimization passes. llvm-svn: 42749
2007-10-08 17:12:17 +02:00
Reapply r106634, now that the bug it exposed is fixed. llvm-svn: 106746
2010-06-24 16:30:44 +02:00
if (OptLevel != CodeGenOpt::None)
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
ComputeLiveOutVRegInfo();
Added a temporary option -no-isel-fold-inflight to control whether a "inflight" node can be folded. llvm-svn: 28003
2006-04-28 04:09:19 +02:00
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
if (ViewISelDAGs && MatchFilterBB)
CurDAG->viewGraph("isel input for " + BlockName);
make -view-isel-dags print after the 'ShrinkDemandedOps' pass. llvm-svn: 98509
2010-03-14 20:27:55 +01:00
Instead of setting up the CFG edges at selectiondag construction time, set them up after the code has been emitted. This allows targets to select one mbb as multiple mbb's as needed. llvm-svn: 20937
2005-03-30 03:10:47 +02:00
// Third, instruction select all of the operations to machine code, adding the
// code to the MachineBasicBlock.
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("isel", "Instruction Selection", GroupName,
GroupDescription, TimePassesIsEnabled);
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
DoInstructionSelection();
Apply Chris' suggestion. llvm-svn: 52970
2008-07-01 19:59:20 +02:00
}
Split scheduling from instruction selection. llvm-svn: 52923
2008-06-30 22:45:06 +02:00
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
DEBUG(dbgs() << "Selected selection DAG: BB#" << BlockNumber
<< " '" << BlockName << "'\n"; CurDAG->dump());
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
SelectionDAG: add a -filter-view-dags option to llc This option takes the name of the basic block you want to visualize with -view-*-dags Differential Revision: http://reviews.llvm.org/D6948 llvm-svn: 225953
2015-01-14 07:03:18 +01:00
if (ViewSchedDAGs && MatchFilterBB)
CurDAG->viewGraph("scheduler input for " + BlockName);
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
Reapply 53476 and 53480, with a fix so that it properly updates the BB member to the current basic block after emitting instructions. llvm-svn: 53567
2008-07-14 20:19:29 +02:00
// Schedule machine code.
When scheduling a block in parts, keep track of the overall instruction index across each part. Instruction indices are used to make live range queries, and live ranges can extend beyond scheduling region boundaries. Refactor the ScheduleDAGSDNodes class some more so that it doesn't have to worry about this additional information. llvm-svn: 64288
2009-02-11 05:27:20 +01:00
ScheduleDAGSDNodes *Scheduler = CreateScheduler();
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("sched", "Instruction Scheduling", GroupName,
GroupDescription, TimePassesIsEnabled);
misched preparation: clarify ScheduleDAG and ScheduleDAGInstrs roles. ScheduleDAG is responsible for the DAG: SUnits and SDeps. It provides target hooks for latency computation. ScheduleDAGInstrs extends ScheduleDAG and defines the current scheduling region in terms of MachineInstr iterators. It has access to the target's scheduling itinerary data. ScheduleDAGInstrs provides the logic for building the ScheduleDAG for the sequence of MachineInstrs in the current region. Target's can implement highly custom schedulers by extending this class. ScheduleDAGPostRATDList provides the driver and diagnostics for current postRA scheduling. It maintains a current Sequence of scheduled machine instructions and logic for splicing them into the block. During scheduling, it uses the ScheduleHazardRecognizer provided by the target. Specific changes: - Removed driver code from ScheduleDAG. clearDAG is the only interface needed. - Added enterRegion/exitRegion hooks to ScheduleDAGInstrs to delimit the scope of each scheduling region and associated DAG. They should be used to setup and cleanup any region-specific state in addition to the DAG itself. This is necessary because we reuse the same ScheduleDAG object for the entire function. The target may extend these hooks to do things at regions boundaries, like bundle terminators. The hooks are called even if we decide not to schedule the region. So all instructions in a block are "covered" by these calls. - Added ScheduleDAGInstrs::begin()/end() public API. - Moved Sequence into the driver layer, which is specific to the scheduling algorithm. llvm-svn: 152208
2012-03-07 06:21:52 +01:00
Scheduler->Run(CurDAG, FuncInfo->MBB);
Reapply 53476 and 53480, with a fix so that it properly updates the BB member to the current basic block after emitting instructions. llvm-svn: 53567
2008-07-14 20:19:29 +02:00
}
Fix Clang-tidy readability-redundant-control-flow warnings; other minor fixes. Differential revision: http://reviews.llvm.org/D16793 llvm-svn: 259539
2016-02-02 19:20:45 +01:00
if (ViewSUnitDAGs && MatchFilterBB)
Scheduler->viewGraph();
Add titles to the various SelectionDAG viewGraph calls that include useful information like the name of the block being viewed and the current phase of compilation. llvm-svn: 53872
2008-07-21 22:00:07 +02:00
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
// Emit machine code to BB. This can change 'BB' to the last block being
Split scheduling from instruction selection. llvm-svn: 52923
2008-06-30 22:45:06 +02:00
// inserted into.
When isel is emitting instructions for an x86 target without CMOV, the CFG is edited during emission. If the basic block ends in a switch that gets lowered to a jump table, any phis at the default edge were getting updated wrong. The jump table data structure keeps a pointer to the header blocks that wasn't getting updated after the MBB is split. This bug was exposed on 32-bit Linux when disabling critical edge splitting in codegen prepare. The fix is to uipdate stale MBB pointers whenever a block is split during emission. llvm-svn: 115191
2010-09-30 21:44:31 +02:00
MachineBasicBlock *FirstMBB = FuncInfo->MBB, *LastMBB;
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("emit", "Instruction Creation", GroupName,
GroupDescription, TimePassesIsEnabled);
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
misched preparation: clarify ScheduleDAG and ScheduleDAGInstrs roles. ScheduleDAG is responsible for the DAG: SUnits and SDeps. It provides target hooks for latency computation. ScheduleDAGInstrs extends ScheduleDAG and defines the current scheduling region in terms of MachineInstr iterators. It has access to the target's scheduling itinerary data. ScheduleDAGInstrs provides the logic for building the ScheduleDAG for the sequence of MachineInstrs in the current region. Target's can implement highly custom schedulers by extending this class. ScheduleDAGPostRATDList provides the driver and diagnostics for current postRA scheduling. It maintains a current Sequence of scheduled machine instructions and logic for splicing them into the block. During scheduling, it uses the ScheduleHazardRecognizer provided by the target. Specific changes: - Removed driver code from ScheduleDAG. clearDAG is the only interface needed. - Added enterRegion/exitRegion hooks to ScheduleDAGInstrs to delimit the scope of each scheduling region and associated DAG. They should be used to setup and cleanup any region-specific state in addition to the DAG itself. This is necessary because we reuse the same ScheduleDAG object for the entire function. The target may extend these hooks to do things at regions boundaries, like bundle terminators. The hooks are called even if we decide not to schedule the region. So all instructions in a block are "covered" by these calls. - Added ScheduleDAGInstrs::begin()/end() public API. - Moved Sequence into the driver layer, which is specific to the scheduling algorithm. llvm-svn: 152208
2012-03-07 06:21:52 +01:00
// FuncInfo->InsertPt is passed by reference and set to the end of the
// scheduled instructions.
LastMBB = FuncInfo->MBB = Scheduler->EmitSchedule(FuncInfo->InsertPt);
Reapply 53476 and 53480, with a fix so that it properly updates the BB member to the current basic block after emitting instructions. llvm-svn: 53567
2008-07-14 20:19:29 +02:00
}
When isel is emitting instructions for an x86 target without CMOV, the CFG is edited during emission. If the basic block ends in a switch that gets lowered to a jump table, any phis at the default edge were getting updated wrong. The jump table data structure keeps a pointer to the header blocks that wasn't getting updated after the MBB is split. This bug was exposed on 32-bit Linux when disabling critical edge splitting in codegen prepare. The fix is to uipdate stale MBB pointers whenever a block is split during emission. llvm-svn: 115191
2010-09-30 21:44:31 +02:00
// If the block was split, make sure we update any references that are used to
// update PHI nodes later on.
if (FirstMBB != LastMBB)
SDB->UpdateSplitBlock(FirstMBB, LastMBB);
Reapply 53476 and 53480, with a fix so that it properly updates the BB member to the current basic block after emitting instructions. llvm-svn: 53567
2008-07-14 20:19:29 +02:00
// Free the scheduler state.
Give NamedRegionTimer an Enabled flag, allowing all its clients to switch from this: if (TimePassesIsEnabled) { NamedRegionTimer T(Name, GroupName); do_something(); } else { do_something(); // duplicate the code, this time without a timer! } to this: { NamedRegionTimer T(Name, GroupName, TimePassesIsEnabled); do_something(); } llvm-svn: 106285
2010-06-18 17:56:31 +02:00
{
Timer: Track name and description. The previously used "names" are rather descriptions (they use multiple words and contain spaces), use short programming language identifier like strings for the "names" which should be used when exporting to machine parseable formats. Also removed a unused TimerGroup from Hexxagon. Differential Revision: https://reviews.llvm.org/D25583 llvm-svn: 287369
2016-11-18 20:43:18 +01:00
NamedRegionTimer T("cleanup", "Instruction Scheduling Cleanup", GroupName,
GroupDescription, TimePassesIsEnabled);
Reapply 53476 and 53480, with a fix so that it properly updates the BB member to the current basic block after emitting instructions. llvm-svn: 53567
2008-07-14 20:19:29 +02:00
delete Scheduler;
Apply Chris' suggestion. llvm-svn: 52970
2008-07-01 19:59:20 +02:00
}
Split scheduling from instruction selection. llvm-svn: 52923
2008-06-30 22:45:06 +02:00
Reorgnaize more switch code lowering to clean up some tricky code, and to eliminate the need for the SelectionDAGBuilder state to be live during CodeGenAndEmitDAG calls. Call SDB->clear() before CodeGenAndEmitDAG calls instead of before it, and move the CurDAG->clear() out of SelectionDAGBuilder, which doesn't own the DAG, and into CodeGenAndEmitDAG. llvm-svn: 102814
2010-05-01 02:25:44 +02:00
// Free the SelectionDAG state, now that we're finished with it.
CurDAG->clear();
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
}
SelectionDAGISel can now natively handle Switch instructions, in the same manner that the LowerSwitch LLVM to LLVM pass does: emitting a binary search tree of basic blocks. The new approach has several advantages: it is faster, it generates significantly smaller code in many cases, and it paves the way for implementing dense switch tables as a jump table by handling switches directly in the instruction selector. This functionality is currently only enabled on x86, but should be safe for every target. In anticipation of making it the default, the cfg is now properly updated in the x86, ppc, and sparc select lowering code. llvm-svn: 27156
2006-03-27 03:32:24 +02:00
Make ISelPosition a local variable. Now that multiple DAGUpdateListeners can be active at the same time, ISelPosition can become a local variable in DoInstructionSelection. We simply register an ISelUpdater with CurDAG while ISelPosition exists. llvm-svn: 155249
2012-04-21 00:08:50 +02:00
namespace {
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
Make ISelPosition a local variable. Now that multiple DAGUpdateListeners can be active at the same time, ISelPosition can become a local variable in DoInstructionSelection. We simply register an ISelUpdater with CurDAG while ISelPosition exists. llvm-svn: 155249
2012-04-21 00:08:50 +02:00
/// ISelUpdater - helper class to handle updates of the instruction selection
/// graph.
class ISelUpdater : public SelectionDAG::DAGUpdateListener {
SelectionDAG::allnodes_iterator &ISelPosition;
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
Make ISelPosition a local variable. Now that multiple DAGUpdateListeners can be active at the same time, ISelPosition can become a local variable in DoInstructionSelection. We simply register an ISelUpdater with CurDAG while ISelPosition exists. llvm-svn: 155249
2012-04-21 00:08:50 +02:00
public:
ISelUpdater(SelectionDAG &DAG, SelectionDAG::allnodes_iterator &isp)
: SelectionDAG::DAGUpdateListener(DAG), ISelPosition(isp) {}
/// NodeDeleted - Handle nodes deleted from the graph. If the node being
/// deleted is the current ISelPosition node, update ISelPosition.
///
[C++11] Add 'override' keyword to virtual methods that override their base class. llvm-svn: 203339
2014-03-08 07:31:39 +01:00
void NodeDeleted(SDNode *N, SDNode *E) override {
Make ISelPosition a local variable. Now that multiple DAGUpdateListeners can be active at the same time, ISelPosition can become a local variable in DoInstructionSelection. We simply register an ISelUpdater with CurDAG while ISelPosition exists. llvm-svn: 155249
2012-04-21 00:08:50 +02:00
if (ISelPosition == SelectionDAG::allnodes_iterator(N))
++ISelPosition;
}
};
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
Make ISelPosition a local variable. Now that multiple DAGUpdateListeners can be active at the same time, ISelPosition can become a local variable in DoInstructionSelection. We simply register an ISelUpdater with CurDAG while ISelPosition exists. llvm-svn: 155249
2012-04-21 00:08:50 +02:00
} // end anonymous namespace
Add intrinsics for constrained floating point operations This commit introduces a set of experimental intrinsics intended to prevent optimizations that make assumptions about the rounding mode and floating point exception behavior. These intrinsics will later be extended to specify flush-to-zero behavior. More work is also required to model instruction dependencies in machine code and to generate these instructions from clang (when required by pragmas and/or command line options that are not currently supported). Differential Revision: https://reviews.llvm.org/D27028 llvm-svn: 293226
2017-01-27 00:27:59 +01:00
static bool isStrictFPOp(SDNode *Node, unsigned &NewOpc) {
unsigned OrigOpc = Node->getOpcode();
switch (OrigOpc) {
case ISD::STRICT_FADD: NewOpc = ISD::FADD; return true;
case ISD::STRICT_FSUB: NewOpc = ISD::FSUB; return true;
case ISD::STRICT_FMUL: NewOpc = ISD::FMUL; return true;
case ISD::STRICT_FDIV: NewOpc = ISD::FDIV; return true;
case ISD::STRICT_FREM: NewOpc = ISD::FREM; return true;
default: return false;
}
}
SDNode* SelectionDAGISel::MutateStrictFPToFP(SDNode *Node, unsigned NewOpc) {
assert(((Node->getOpcode() == ISD::STRICT_FADD && NewOpc == ISD::FADD) ||
(Node->getOpcode() == ISD::STRICT_FSUB && NewOpc == ISD::FSUB) ||
(Node->getOpcode() == ISD::STRICT_FMUL && NewOpc == ISD::FMUL) ||
(Node->getOpcode() == ISD::STRICT_FDIV && NewOpc == ISD::FDIV) ||
(Node->getOpcode() == ISD::STRICT_FREM && NewOpc == ISD::FREM)) &&
"Unexpected StrictFP opcode!");
// We're taking this node out of the chain, so we need to re-link things.
SDValue InputChain = Node->getOperand(0);
SDValue OutputChain = SDValue(Node, 1);
CurDAG->ReplaceAllUsesOfValueWith(OutputChain, InputChain);
SDVTList VTs = CurDAG->getVTList(Node->getOperand(1).getValueType());
SDValue Ops[2] = { Node->getOperand(1), Node->getOperand(2) };
SDNode *Res = CurDAG->MorphNodeTo(Node, NewOpc, VTs, Ops);
// MorphNodeTo can operate in two ways: if an existing node with the
// specified operands exists, it can just return it. Otherwise, it
// updates the node in place to have the requested operands.
if (Res == Node) {
// If we updated the node in place, reset the node ID. To the isel,
// this should be just like a newly allocated machine node.
Res->setNodeId(-1);
} else {
CurDAG->ReplaceAllUsesWith(Node, Res);
CurDAG->RemoveDeadNode(Node);
}
return Res;
}
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
void SelectionDAGISel::DoInstructionSelection() {
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << "===== Instruction selection begins: BB#"
Added block number and name to isel debug output. I'm tired of doing this manually for each checkout. If anyone knows a better way debug isel for non-trivial tests feel free to revert and let me know how to do it. llvm-svn: 128132
2011-03-23 02:38:28 +01:00
<< FuncInfo->MBB->getNumber()
<< " '" << FuncInfo->MBB->getName() << "'\n");
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
PreprocessISelDAG();
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
// Select target instructions for the DAG.
{
// Number all nodes with a topological order and set DAGSize.
DAGSize = CurDAG->AssignTopologicalOrder();
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
// Create a dummy node (which is not added to allnodes), that adds
// a reference to the root node, preventing it from being deleted,
// and tracking any changes of the root.
HandleSDNode Dummy(CurDAG->getRoot());
Make ISelPosition a local variable. Now that multiple DAGUpdateListeners can be active at the same time, ISelPosition can become a local variable in DoInstructionSelection. We simply register an ISelUpdater with CurDAG while ISelPosition exists. llvm-svn: 155249
2012-04-21 00:08:50 +02:00
SelectionDAG::allnodes_iterator ISelPosition (CurDAG->getRoot().getNode());
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
++ISelPosition;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Make ISelPosition a local variable. Now that multiple DAGUpdateListeners can be active at the same time, ISelPosition can become a local variable in DoInstructionSelection. We simply register an ISelUpdater with CurDAG while ISelPosition exists. llvm-svn: 155249
2012-04-21 00:08:50 +02:00
// Make sure that ISelPosition gets properly updated when nodes are deleted
// in calls made from this function.
ISelUpdater ISU(*CurDAG, ISelPosition);
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
// The AllNodes list is now topological-sorted. Visit the
// nodes by starting at the end of the list (the root of the
// graph) and preceding back toward the beginning (the entry
// node).
while (ISelPosition != CurDAG->allnodes_begin()) {
SelectionDAG: Remove implicit ilist iterator conversions, NFC llvm-svn: 250214
2015-10-13 21:47:46 +02:00
SDNode *Node = &*--ISelPosition;
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
// Skip dead nodes. DAGCombiner is expected to eliminate all dead nodes,
// but there are currently some corner cases that it misses. Also, this
// makes it theoretically possible to disable the DAGCombiner.
if (Node->use_empty())
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Add intrinsics for constrained floating point operations This commit introduces a set of experimental intrinsics intended to prevent optimizations that make assumptions about the rounding mode and floating point exception behavior. These intrinsics will later be extended to specify flush-to-zero behavior. More work is also required to model instruction dependencies in machine code and to generate these instructions from clang (when required by pragmas and/or command line options that are not currently supported). Differential Revision: https://reviews.llvm.org/D27028 llvm-svn: 293226
2017-01-27 00:27:59 +01:00
// When we are using non-default rounding modes or FP exception behavior
// FP operations are represented by StrictFP pseudo-operations. They
// need to be simplified here so that the target-specific instruction
// selectors know how to handle them.
//
// If the current node is a strict FP pseudo-op, the isStrictFPOp()
// function will provide the corresponding normal FP opcode to which the
// node should be mutated.
unsigned NormalFPOpc = ISD::UNDEF;
bool IsStrictFPOp = isStrictFPOp(Node, NormalFPOpc);
if (IsStrictFPOp)
Node = MutateStrictFPToFP(Node, NormalFPOpc);
SDAG: Rename Select->SelectImpl and repurpose Select as returning void This is a step towards removing the rampant undefined behaviour in SelectionDAG, which is a part of llvm.org/PR26808. We rename SelectionDAGISel::Select to SelectImpl and update targets to match, and then change Select to return void and consolidate the sketchy behaviour we're trying to get away from there. Next, we'll update backends to implement `void Select(...)` instead of SelectImpl and eventually drop the base Select implementation. llvm-svn: 268693
2016-05-06 01:19:08 +02:00
Select(Node);
Add intrinsics for constrained floating point operations This commit introduces a set of experimental intrinsics intended to prevent optimizations that make assumptions about the rounding mode and floating point exception behavior. These intrinsics will later be extended to specify flush-to-zero behavior. More work is also required to model instruction dependencies in machine code and to generate these instructions from clang (when required by pragmas and/or command line options that are not currently supported). Differential Revision: https://reviews.llvm.org/D27028 llvm-svn: 293226
2017-01-27 00:27:59 +01:00
// FIXME: Add code here to attach an implicit def and use of
// target-specific FP environment registers.
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
CurDAG->setRoot(Dummy.getValue());
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
}
- Set the "HasCalls" flag after instruction selection is finished. - Change the logic DisableFramePointerElim() to check for the -disable-non-leaf-fp-elim before -disable-fp-elim. llvm-svn: 103990
2010-05-18 01:09:50 +02:00
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << "===== Instruction selection ends:\n");
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
PostprocessISelDAG();
}
[SEH] Add llvm.eh.exceptioncode intrinsic This will support the Clang __exception_code intrinsic. llvm-svn: 249492
2015-10-07 02:27:33 +02:00
static bool hasExceptionPointerOrCodeUser(const CatchPadInst *CPI) {
for (const User *U : CPI->users()) {
if (const IntrinsicInst *EHPtrCall = dyn_cast<IntrinsicInst>(U)) {
Intrinsic::ID IID = EHPtrCall->getIntrinsicID();
if (IID == Intrinsic::eh_exceptionpointer ||
IID == Intrinsic::eh_exceptioncode)
return true;
}
}
return false;
}
Factor out EH landing pad code into a separate function, and constify a bunch of stuff to support it. llvm-svn: 101273
2010-04-14 21:53:31 +02:00
/// PrepareEHLandingPad - Emit an EH_LABEL, set up live-in registers, and
/// do other setup for EH landing-pad blocks.
Re-land r235154-r235156 under the existing -sehprepare flag Keep the old SEH fan-in lowering on by default for now, since projects rely on it. This will make it easy to test this change with a simple flag flip. llvm-svn: 235399
2015-04-21 20:23:57 +02:00
bool SelectionDAGISel::PrepareEHLandingPad() {
Small refactoring. Cache the FunctionInfo->MBB into a local variable. llvm-svn: 141221
2011-10-06 00:16:11 +02:00
MachineBasicBlock *MBB = FuncInfo->MBB;
[WinEH] Update exception pointer registers Summary: The CLR's personality routine passes these in rdx/edx, not rax/eax. Make getExceptionPointerRegister a virtual method parameterized by personality function to allow making this distinction. Similarly make getExceptionSelectorRegister a virtual method parameterized by personality function, for symmetry. Reviewers: pgavlin, majnemer, rnk Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D14344 llvm-svn: 252383
2015-11-07 02:11:31 +01:00
const Constant *PersonalityFn = FuncInfo->Fn->getPersonalityFn();
[SEH] Add llvm.eh.exceptioncode intrinsic This will support the Clang __exception_code intrinsic. llvm-svn: 249492
2015-10-07 02:27:33 +02:00
const BasicBlock *LLVMBB = MBB->getBasicBlock();
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
const TargetRegisterClass *PtrRC =
TLI->getRegClassFor(TLI->getPointerTy(CurDAG->getDataLayout()));
CodeGen support for x86_64 SEH catch handlers in LLVM This adds handling for ExceptionHandling::MSVC, used by the x86_64-pc-windows-msvc triple. It assumes that filter functions have already been outlined in either the frontend or the backend. Filter functions are used in place of the landingpad catch clause type info operands. In catch clause order, the first filter to return true will catch the exception. The C specific handler table expects the landing pad to be split into one block per handler, but LLVM IR uses a single landing pad for all possible unwind actions. This patch papers over the mismatch by synthesizing single instruction BBs for every catch clause to fill in the EH selector that the landing pad block expects. Missing functionality: - Accessing data in the parent frame from outlined filters - Cleanups (from __finally) are unsupported, as they will require outlining and parent frame access - Filter clauses are unsupported, as there's no clear analogue in SEH In other words, this is the minimal set of changes needed to write IR to catch arbitrary exceptions and resume normal execution. Reviewers: majnemer Differential Revision: http://reviews.llvm.org/D6300 llvm-svn: 225904
2015-01-14 02:05:27 +01:00
[SEH] Add llvm.eh.exceptioncode intrinsic This will support the Clang __exception_code intrinsic. llvm-svn: 249492
2015-10-07 02:27:33 +02:00
// Catchpads have one live-in register, which typically holds the exception
// pointer or code.
if (const auto *CPI = dyn_cast<CatchPadInst>(LLVMBB->getFirstNonPHI())) {
if (hasExceptionPointerOrCodeUser(CPI)) {
// Get or create the virtual register to hold the pointer or code. Mark
// the live in physreg and copy into the vreg.
[WinEH] Update exception pointer registers Summary: The CLR's personality routine passes these in rdx/edx, not rax/eax. Make getExceptionPointerRegister a virtual method parameterized by personality function to allow making this distinction. Similarly make getExceptionSelectorRegister a virtual method parameterized by personality function, for symmetry. Reviewers: pgavlin, majnemer, rnk Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D14344 llvm-svn: 252383
2015-11-07 02:11:31 +01:00
MCPhysReg EHPhysReg = TLI->getExceptionPointerRegister(PersonalityFn);
[SEH] Add llvm.eh.exceptioncode intrinsic This will support the Clang __exception_code intrinsic. llvm-svn: 249492
2015-10-07 02:27:33 +02:00
assert(EHPhysReg && "target lacks exception pointer register");
[SEH] Fix llvm.eh.exceptioncode fast register allocation assertion I called the wrong MachineBasicBlock::addLiveIn() overload. llvm-svn: 249786
2015-10-09 02:15:13 +02:00
MBB->addLiveIn(EHPhysReg);
[SEH] Add llvm.eh.exceptioncode intrinsic This will support the Clang __exception_code intrinsic. llvm-svn: 249492
2015-10-07 02:27:33 +02:00
unsigned VReg = FuncInfo->getCatchPadExceptionPointerVReg(CPI, PtrRC);
BuildMI(*MBB, FuncInfo->InsertPt, SDB->getCurDebugLoc(),
TII->get(TargetOpcode::COPY), VReg)
.addReg(EHPhysReg, RegState::Kill);
}
return true;
}
if (!LLVMBB->isLandingPad())
return true;
Factor out EH landing pad code into a separate function, and constify a bunch of stuff to support it. llvm-svn: 101273
2010-04-14 21:53:31 +02:00
// Add a label to mark the beginning of the landing pad. Deletion of the
// landing pad can thus be detected via the MachineModuleInfo.
Move most EH from MachineModuleInfo to MachineFunction Recommitting r288293 with some extra fixes for GlobalISel code. Most of the exception handling members in MachineModuleInfo is actually per function data (talks about the "current function") so it is better to keep it at the function instead of the module. This is a necessary step to have machine module passes work properly. Also: - Rename TidyLandingPads() to tidyLandingPads() - Use doxygen member groups instead of "//===- EH ---"... so it is clear where a group ends. - I had to add an ugly const_cast at two places in the AsmPrinter because the available MachineFunction pointers are const, but the code wants to call tidyLandingPads() in between (markFunctionEnd()/endFunction()). Differential Revision: https://reviews.llvm.org/D27227 llvm-svn: 288405
2016-12-01 20:32:15 +01:00
MCSymbol *Label = MF->addLandingPad(MBB);
Factor out EH landing pad code into a separate function, and constify a bunch of stuff to support it. llvm-svn: 101273
2010-04-14 21:53:31 +02:00
Modify the mapping from landing pad to call sites to accept more than one call site. llvm-svn: 141226
2011-10-06 00:24:35 +02:00
// Assign the call site to the landing pad's begin label.
Move most EH from MachineModuleInfo to MachineFunction Recommitting r288293 with some extra fixes for GlobalISel code. Most of the exception handling members in MachineModuleInfo is actually per function data (talks about the "current function") so it is better to keep it at the function instead of the module. This is a necessary step to have machine module passes work properly. Also: - Rename TidyLandingPads() to tidyLandingPads() - Use doxygen member groups instead of "//===- EH ---"... so it is clear where a group ends. - I had to add an ugly const_cast at two places in the AsmPrinter because the available MachineFunction pointers are const, but the code wants to call tidyLandingPads() in between (markFunctionEnd()/endFunction()). Differential Revision: https://reviews.llvm.org/D27227 llvm-svn: 288405
2016-12-01 20:32:15 +01:00
MF->setCallSiteLandingPad(Label, SDB->LPadToCallSiteMap[MBB]);
whitespace llvm-svn: 152175
2012-03-07 01:18:15 +01:00
Cache SelectionDAGISel TargetInstrInfo lookups on the class and propagate. Also use the TargetSubtargetInfo and the MachineFunction and move TargetRegisterInfo query closer to uses. llvm-svn: 219273
2014-10-08 03:58:03 +02:00
const MCInstrDesc &II = TII->get(TargetOpcode::EH_LABEL);
Small refactoring. Cache the FunctionInfo->MBB into a local variable. llvm-svn: 141221
2011-10-06 00:16:11 +02:00
BuildMI(*MBB, FuncInfo->InsertPt, SDB->getCurDebugLoc(), II)
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
.addSym(Label);
Factor out EH landing pad code into a separate function, and constify a bunch of stuff to support it. llvm-svn: 101273
2010-04-14 21:53:31 +02:00
// Mark exception register as live in.
[WinEH] Update exception pointer registers Summary: The CLR's personality routine passes these in rdx/edx, not rax/eax. Make getExceptionPointerRegister a virtual method parameterized by personality function to allow making this distinction. Similarly make getExceptionSelectorRegister a virtual method parameterized by personality function, for symmetry. Reviewers: pgavlin, majnemer, rnk Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D14344 llvm-svn: 252383
2015-11-07 02:11:31 +01:00
if (unsigned Reg = TLI->getExceptionPointerRegister(PersonalityFn))
Simplify landing pad lowering. Stop using the ISD::EXCEPTIONADDR and ISD::EHSELECTION when lowering landing pad arguments. These nodes were previously legalized into CopyFromReg nodes, but that never worked properly because the CopyFromReg node weren't guaranteed to be scheduled at the top of the basic block. This meant the exception pointer and selector registers could be clobbered before being copied to a virtual register. This patch copies the two physical registers to virtual registers at the beginning of the basic block, and lowers the landingpad instruction directly to two CopyFromReg nodes reading the *virtual* registers. This is safe because virtual registers don't get clobbered. A future patch will remove the ISD::EXCEPTIONADDR and ISD::EHSELECTION nodes. llvm-svn: 185617
2013-07-04 06:53:45 +02:00
FuncInfo->ExceptionPointerVirtReg = MBB->addLiveIn(Reg, PtrRC);
Factor out EH landing pad code into a separate function, and constify a bunch of stuff to support it. llvm-svn: 101273
2010-04-14 21:53:31 +02:00
// Mark exception selector register as live in.
[WinEH] Update exception pointer registers Summary: The CLR's personality routine passes these in rdx/edx, not rax/eax. Make getExceptionPointerRegister a virtual method parameterized by personality function to allow making this distinction. Similarly make getExceptionSelectorRegister a virtual method parameterized by personality function, for symmetry. Reviewers: pgavlin, majnemer, rnk Subscribers: jyknight, dsanders, llvm-commits Differential Revision: http://reviews.llvm.org/D14344 llvm-svn: 252383
2015-11-07 02:11:31 +01:00
if (unsigned Reg = TLI->getExceptionSelectorRegister(PersonalityFn))
Simplify landing pad lowering. Stop using the ISD::EXCEPTIONADDR and ISD::EHSELECTION when lowering landing pad arguments. These nodes were previously legalized into CopyFromReg nodes, but that never worked properly because the CopyFromReg node weren't guaranteed to be scheduled at the top of the basic block. This meant the exception pointer and selector registers could be clobbered before being copied to a virtual register. This patch copies the two physical registers to virtual registers at the beginning of the basic block, and lowers the landingpad instruction directly to two CopyFromReg nodes reading the *virtual* registers. This is safe because virtual registers don't get clobbered. A future patch will remove the ISD::EXCEPTIONADDR and ISD::EHSELECTION nodes. llvm-svn: 185617
2013-07-04 06:53:45 +02:00
FuncInfo->ExceptionSelectorVirtReg = MBB->addLiveIn(Reg, PtrRC);
Re-land r235154-r235156 under the existing -sehprepare flag Keep the old SEH fan-in lowering on by default for now, since projects rely on it. This will make it easy to test this change with a simple flag flip. llvm-svn: 235399
2015-04-21 20:23:57 +02:00
return true;
Factor out EH landing pad code into a separate function, and constify a bunch of stuff to support it. llvm-svn: 101273
2010-04-14 21:53:31 +02:00
}
Sink InstructionSelect() out of each target into SDISel, and rename it DoInstructionSelection. Inline "SelectRoot" into it from DAGISelHeader. Sink some other stuff out of DAGISelHeader into SDISel. Eliminate the various 'Indent' stuff from various targets, which dates to when isel was recursive. 17 files changed, 114 insertions(+), 430 deletions(-) llvm-svn: 97555
2010-03-02 07:34:30 +01:00
split a complex predicate out to a helper function. Simplify two for loops, which don't need to check for falling off the end of a block *and* end of phi nodes, since terminators are never phis. llvm-svn: 129655
2011-04-17 08:03:19 +02:00
/// isFoldedOrDeadInstruction - Return true if the specified instruction is
/// side-effect free and is either dead or folded into a generated instruction.
/// Return false if it needs to be emitted.
static bool isFoldedOrDeadInstruction(const Instruction *I,
FunctionLoweringInfo *FuncInfo) {
Recommit the fix for rdar://9289512 with a couple tweaks to fix bugs exposed by the gcc dejagnu testsuite: 1. The load may actually be used by a dead instruction, which would cause an assert. 2. The load may not be used by the current chain of instructions, and we could move it past a side-effecting instruction. Change how we process uses to define the problem away. llvm-svn: 130018
2011-04-22 23:59:37 +02:00
return !I->mayWriteToMemory() && // Side-effecting instructions aren't folded.
[WinEH] Add some support for code generating catchpad We can now run 32-bit programs with empty catch bodies. The next step is to change PEI so that we get funclet prologues and epilogues. llvm-svn: 246235
2015-08-28 01:27:47 +02:00
!isa<TerminatorInst>(I) && // Terminators aren't folded.
Recommit the fix for rdar://9289512 with a couple tweaks to fix bugs exposed by the gcc dejagnu testsuite: 1. The load may actually be used by a dead instruction, which would cause an assert. 2. The load may not be used by the current chain of instructions, and we could move it past a side-effecting instruction. Change how we process uses to define the problem away. llvm-svn: 130018
2011-04-22 23:59:37 +02:00
!isa<DbgInfoIntrinsic>(I) && // Debug instructions aren't folded.
[WinEH] Add some support for code generating catchpad We can now run 32-bit programs with empty catch bodies. The next step is to change PEI so that we get funclet prologues and epilogues. llvm-svn: 246235
2015-08-28 01:27:47 +02:00
!I->isEHPad() && // EH pad instructions aren't folded.
Recommit the fix for rdar://9289512 with a couple tweaks to fix bugs exposed by the gcc dejagnu testsuite: 1. The load may actually be used by a dead instruction, which would cause an assert. 2. The load may not be used by the current chain of instructions, and we could move it past a side-effecting instruction. Change how we process uses to define the problem away. llvm-svn: 130018
2011-04-22 23:59:37 +02:00
!FuncInfo->isExportedInst(I); // Exported instrs must be computed.
split a complex predicate out to a helper function. Simplify two for loops, which don't need to check for falling off the end of a block *and* end of phi nodes, since terminators are never phis. llvm-svn: 129655
2011-04-17 08:03:19 +02:00
}
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
#ifndef NDEBUG
Add comment. llvm-svn: 147696
2012-01-07 00:45:47 +01:00
// Collect per Instruction statistics for fast-isel misses. Only those
// instructions that cause the bail are accounted for. It does not account for
// instructions higher in the block. Thus, summing the per instructions stats
// will not add up to what is reported by NumFastIselFailures.
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
static void collectFailStats(const Instruction *I) {
switch (I->getOpcode()) {
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
default: assert(false && "<Invalid operator>");
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
// Terminators
case Instruction::Ret: NumFastIselFailRet++; return;
case Instruction::Br: NumFastIselFailBr++; return;
case Instruction::Switch: NumFastIselFailSwitch++; return;
case Instruction::IndirectBr: NumFastIselFailIndirectBr++; return;
case Instruction::Invoke: NumFastIselFailInvoke++; return;
case Instruction::Resume: NumFastIselFailResume++; return;
case Instruction::Unreachable: NumFastIselFailUnreachable++; return;
// Standard binary operators...
case Instruction::Add: NumFastIselFailAdd++; return;
case Instruction::FAdd: NumFastIselFailFAdd++; return;
case Instruction::Sub: NumFastIselFailSub++; return;
case Instruction::FSub: NumFastIselFailFSub++; return;
case Instruction::Mul: NumFastIselFailMul++; return;
case Instruction::FMul: NumFastIselFailFMul++; return;
case Instruction::UDiv: NumFastIselFailUDiv++; return;
case Instruction::SDiv: NumFastIselFailSDiv++; return;
case Instruction::FDiv: NumFastIselFailFDiv++; return;
case Instruction::URem: NumFastIselFailURem++; return;
case Instruction::SRem: NumFastIselFailSRem++; return;
case Instruction::FRem: NumFastIselFailFRem++; return;
// Logical operators...
case Instruction::And: NumFastIselFailAnd++; return;
case Instruction::Or: NumFastIselFailOr++; return;
case Instruction::Xor: NumFastIselFailXor++; return;
// Memory instructions...
case Instruction::Alloca: NumFastIselFailAlloca++; return;
case Instruction::Load: NumFastIselFailLoad++; return;
case Instruction::Store: NumFastIselFailStore++; return;
case Instruction::AtomicCmpXchg: NumFastIselFailAtomicCmpXchg++; return;
case Instruction::AtomicRMW: NumFastIselFailAtomicRMW++; return;
case Instruction::Fence: NumFastIselFailFence++; return;
case Instruction::GetElementPtr: NumFastIselFailGetElementPtr++; return;
// Convert instructions...
case Instruction::Trunc: NumFastIselFailTrunc++; return;
case Instruction::ZExt: NumFastIselFailZExt++; return;
case Instruction::SExt: NumFastIselFailSExt++; return;
case Instruction::FPTrunc: NumFastIselFailFPTrunc++; return;
case Instruction::FPExt: NumFastIselFailFPExt++; return;
case Instruction::FPToUI: NumFastIselFailFPToUI++; return;
case Instruction::FPToSI: NumFastIselFailFPToSI++; return;
case Instruction::UIToFP: NumFastIselFailUIToFP++; return;
case Instruction::SIToFP: NumFastIselFailSIToFP++; return;
whitespace llvm-svn: 152175
2012-03-07 01:18:15 +01:00
case Instruction::IntToPtr: NumFastIselFailIntToPtr++; return;
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
case Instruction::PtrToInt: NumFastIselFailPtrToInt++; return;
whitespace llvm-svn: 152175
2012-03-07 01:18:15 +01:00
case Instruction::BitCast: NumFastIselFailBitCast++; return;
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
// Other instructions...
case Instruction::ICmp: NumFastIselFailICmp++; return;
case Instruction::FCmp: NumFastIselFailFCmp++; return;
case Instruction::PHI: NumFastIselFailPHI++; return;
case Instruction::Select: NumFastIselFailSelect++; return;
[FastISel] Collect statistics about failing intrinsic calls. Add more instruction-specific statistics about failing intrinsic calls during FastISel. llvm-svn: 210556
2014-06-10 20:17:00 +02:00
case Instruction::Call: {
if (auto const *Intrinsic = dyn_cast<IntrinsicInst>(I)) {
switch (Intrinsic->getIntrinsicID()) {
default:
NumFastIselFailIntrinsicCall++; return;
case Intrinsic::sadd_with_overflow:
NumFastIselFailSAddWithOverflow++; return;
case Intrinsic::uadd_with_overflow:
NumFastIselFailUAddWithOverflow++; return;
case Intrinsic::ssub_with_overflow:
NumFastIselFailSSubWithOverflow++; return;
case Intrinsic::usub_with_overflow:
NumFastIselFailUSubWithOverflow++; return;
case Intrinsic::smul_with_overflow:
NumFastIselFailSMulWithOverflow++; return;
case Intrinsic::umul_with_overflow:
NumFastIselFailUMulWithOverflow++; return;
case Intrinsic::frameaddress:
NumFastIselFailFrameaddress++; return;
case Intrinsic::sqrt:
NumFastIselFailSqrt++; return;
case Intrinsic::experimental_stackmap:
NumFastIselFailStackMap++; return;
case Intrinsic::experimental_patchpoint_void: // fall-through
case Intrinsic::experimental_patchpoint_i64:
NumFastIselFailPatchPoint++; return;
}
}
NumFastIselFailCall++;
return;
}
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
case Instruction::Shl: NumFastIselFailShl++; return;
case Instruction::LShr: NumFastIselFailLShr++; return;
case Instruction::AShr: NumFastIselFailAShr++; return;
case Instruction::VAArg: NumFastIselFailVAArg++; return;
case Instruction::ExtractElement: NumFastIselFailExtractElement++; return;
case Instruction::InsertElement: NumFastIselFailInsertElement++; return;
case Instruction::ShuffleVector: NumFastIselFailShuffleVector++; return;
case Instruction::ExtractValue: NumFastIselFailExtractValue++; return;
case Instruction::InsertValue: NumFastIselFailInsertValue++; return;
case Instruction::LandingPad: NumFastIselFailLandingPad++; return;
}
}
Fix Clang-tidy readability-redundant-control-flow warnings; other minor fixes. Differential revision: http://reviews.llvm.org/D16793 llvm-svn: 259539
2016-02-02 19:20:45 +01:00
#endif // NDEBUG
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
/// Set up SwiftErrorVals by going through the function. If the function has
/// swifterror argument, it will be the first entry.
static void setupSwiftErrorVals(const Function &Fn, const TargetLowering *TLI,
FunctionLoweringInfo *FuncInfo) {
if (!TLI->supportSwiftError())
return;
FuncInfo->SwiftErrorVals.clear();
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
FuncInfo->SwiftErrorVRegDefMap.clear();
FuncInfo->SwiftErrorVRegUpwardsUse.clear();
FuncInfo->SwiftErrorArg = nullptr;
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
// Check if function has a swifterror argument.
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
bool HaveSeenSwiftErrorArg = false;
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
for (Function::const_arg_iterator AI = Fn.arg_begin(), AE = Fn.arg_end();
AI != AE; ++AI)
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
if (AI->hasSwiftErrorAttr()) {
assert(!HaveSeenSwiftErrorArg &&
"Must have only one swifterror parameter");
Silence -Wunused-but-set-variable warning llvm-svn: 283927
2016-10-11 21:49:29 +02:00
(void)HaveSeenSwiftErrorArg; // silence warning.
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
HaveSeenSwiftErrorArg = true;
FuncInfo->SwiftErrorArg = &*AI;
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
FuncInfo->SwiftErrorVals.push_back(&*AI);
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
}
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
for (const auto &LLVMBB : Fn)
for (const auto &Inst : LLVMBB) {
if (const AllocaInst *Alloca = dyn_cast<AllocaInst>(&Inst))
if (Alloca->isSwiftError())
FuncInfo->SwiftErrorVals.push_back(Alloca);
}
}
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
static void createSwiftErrorEntriesInEntryBlock(FunctionLoweringInfo *FuncInfo,
const TargetLowering *TLI,
const TargetInstrInfo *TII,
SelectionDAGBuilder *SDB) {
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
if (!TLI->supportSwiftError())
return;
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
// We only need to do this when we have swifterror parameter or swifterror
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
// alloc.
if (FuncInfo->SwiftErrorVals.empty())
return;
[SDAGISel] Simplify some SDAGISel code, NFC Hoist entry block code for arguments and swift error values out of the basic block instruction selection loop. Lowering arguments once up front seems much more readable than doing it conditionally inside the loop. It also makes it clear that argument lowering can update StaticAllocaMap because no instructions have been selected yet. Also use range-based for loops where possible. llvm-svn: 294329
2017-02-07 19:42:53 +01:00
assert(FuncInfo->MBB == &*FuncInfo->MF->begin() &&
"expected to insert into entry block");
auto &DL = FuncInfo->MF->getDataLayout();
auto const *RC = TLI->getRegClassFor(TLI->getPointerTy(DL));
for (const auto *SwiftErrorVal : FuncInfo->SwiftErrorVals) {
// We will always generate a copy from the argument. It is always used at
// least by the 'return' of the swifterror.
if (FuncInfo->SwiftErrorArg && FuncInfo->SwiftErrorArg == SwiftErrorVal)
continue;
unsigned VReg = FuncInfo->MF->getRegInfo().createVirtualRegister(RC);
// Assign Undef to Vreg. We construct MI directly to make sure it works
// with FastISel.
BuildMI(*FuncInfo->MBB, FuncInfo->MBB->getFirstNonPHI(),
SDB->getCurDebugLoc(), TII->get(TargetOpcode::IMPLICIT_DEF),
VReg);
FuncInfo->setCurrentSwiftErrorVReg(FuncInfo->MBB, SwiftErrorVal, VReg);
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
}
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
}
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
/// Propagate swifterror values through the machine function CFG.
static void propagateSwiftErrorVRegs(FunctionLoweringInfo *FuncInfo) {
auto *TLI = FuncInfo->TLI;
if (!TLI->supportSwiftError())
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
return;
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
// We only need to do this when we have swifterror parameter or swifterror
// alloc.
if (FuncInfo->SwiftErrorVals.empty())
return;
// For each machine basic block in reverse post order.
ReversePostOrderTraversal<MachineFunction *> RPOT(FuncInfo->MF);
for (ReversePostOrderTraversal<MachineFunction *>::rpo_iterator
It = RPOT.begin(),
E = RPOT.end();
It != E; ++It) {
MachineBasicBlock *MBB = *It;
// For each swifterror value in the function.
for(const auto *SwiftErrorVal : FuncInfo->SwiftErrorVals) {
auto Key = std::make_pair(MBB, SwiftErrorVal);
auto UUseIt = FuncInfo->SwiftErrorVRegUpwardsUse.find(Key);
auto VRegDefIt = FuncInfo->SwiftErrorVRegDefMap.find(Key);
bool UpwardsUse = UUseIt != FuncInfo->SwiftErrorVRegUpwardsUse.end();
unsigned UUseVReg = UpwardsUse ? UUseIt->second : 0;
bool DownwardDef = VRegDefIt != FuncInfo->SwiftErrorVRegDefMap.end();
assert(!(UpwardsUse && !DownwardDef) &&
"We can't have an upwards use but no downwards def");
// If there is no upwards exposed use and an entry for the swifterror in
// the def map for this value we don't need to do anything: We already
// have a downward def for this basic block.
if (!UpwardsUse && DownwardDef)
continue;
// Otherwise we either have an upwards exposed use vreg that we need to
// materialize or need to forward the downward def from predecessors.
// Check whether we have a single vreg def from all predecessors.
// Otherwise we need a phi.
SmallVector<std::pair<MachineBasicBlock *, unsigned>, 4> VRegs;
SmallSet<const MachineBasicBlock*, 8> Visited;
for (auto *Pred : MBB->predecessors()) {
if (!Visited.insert(Pred).second)
continue;
VRegs.push_back(std::make_pair(
Pred, FuncInfo->getOrCreateSwiftErrorVReg(Pred, SwiftErrorVal)));
if (Pred != MBB)
continue;
// We have a self-edge.
// If there was no upwards use in this basic block there is now one: the
// phi needs to use it self.
if (!UpwardsUse) {
UpwardsUse = true;
UUseIt = FuncInfo->SwiftErrorVRegUpwardsUse.find(Key);
assert(UUseIt != FuncInfo->SwiftErrorVRegUpwardsUse.end());
UUseVReg = UUseIt->second;
}
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
}
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
// We need a phi node if we have more than one predecessor with different
// downward defs.
bool needPHI =
VRegs.size() >= 1 &&
std::find_if(
VRegs.begin(), VRegs.end(),
[&](const std::pair<const MachineBasicBlock *, unsigned> &V)
-> bool { return V.second != VRegs[0].second; }) !=
VRegs.end();
// If there is no upwards exposed used and we don't need a phi just
// forward the swifterror vreg from the predecessor(s).
if (!UpwardsUse && !needPHI) {
assert(!VRegs.empty() &&
"No predecessors? The entry block should bail out earlier");
// Just forward the swifterror vreg from the predecessor(s).
FuncInfo->setCurrentSwiftErrorVReg(MBB, SwiftErrorVal, VRegs[0].second);
continue;
}
auto DLoc = isa<Instruction>(SwiftErrorVal)
? dyn_cast<Instruction>(SwiftErrorVal)->getDebugLoc()
: DebugLoc();
const auto *TII = FuncInfo->MF->getSubtarget().getInstrInfo();
// If we don't need a phi create a copy to the upward exposed vreg.
if (!needPHI) {
assert(UpwardsUse);
unsigned DestReg = UUseVReg;
BuildMI(*MBB, MBB->getFirstNonPHI(), DLoc, TII->get(TargetOpcode::COPY),
DestReg)
.addReg(VRegs[0].second);
continue;
}
// We need a phi: if there is an upwards exposed use we already have a
// destination virtual register number otherwise we generate a new one.
auto &DL = FuncInfo->MF->getDataLayout();
auto const *RC = TLI->getRegClassFor(TLI->getPointerTy(DL));
unsigned PHIVReg =
UpwardsUse ? UUseVReg
: FuncInfo->MF->getRegInfo().createVirtualRegister(RC);
MachineInstrBuilder SwiftErrorPHI =
BuildMI(*MBB, MBB->getFirstNonPHI(), DLoc,
TII->get(TargetOpcode::PHI), PHIVReg);
for (auto BBRegPair : VRegs) {
SwiftErrorPHI.addReg(BBRegPair.second).addMBB(BBRegPair.first);
}
// We did not have a definition in this block before: store the phi's vreg
// as this block downward exposed def.
if (!UpwardsUse)
FuncInfo->setCurrentSwiftErrorVReg(MBB, SwiftErrorVal, PHIVReg);
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
}
}
}
Add const qualifiers to CodeGen's use of LLVM IR constructs. llvm-svn: 101334
2010-04-15 03:51:59 +02:00
void SelectionDAGISel::SelectAllBasicBlocks(const Function &Fn) {
[FastISel] Add a diagnostic to warm on fallback. This is consistent with what we do for GlobalISel. That way, it is easy to see whether or not FastISel is able to fully select a function. At some point we may want to switch that to an optimization remark. llvm-svn: 294970
2017-02-13 18:38:59 +01:00
FastISelFailed = false;
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
// Initialize the Fast-ISel state, if needed.
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
FastISel *FastIS = nullptr;
Revert r195317 (and r195333), "Teach ISel not to optimize 'optnone' functions." It broke, at least, i686 target. It is reproducible with "llc -mtriple=i686-unknown". FYI, it didn't appear to add either "-O0" or "-fast-isel". llvm-svn: 195339
2013-11-21 11:55:15 +01:00
if (TM.Options.EnableFastISel)
Cache TargetLowering on SelectionDAGISel and update previous calls to getTargetLowering() with the cached variable. llvm-svn: 219284
2014-10-08 09:32:17 +02:00
FastIS = TLI->createFastISel(*FuncInfo, LibInfo);
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
Swift Calling Convention: swifterror target-independent change. At IR level, the swifterror argument is an input argument with type ErrorObject**. For targets that support swifterror, we want to optimize it to behave as an inout value with type ErrorObject*; it will be passed in a fixed physical register. The main idea is to track the virtual registers for each swifterror value. We define swifterror values as AllocaInsts with swifterror attribute or a function argument with swifterror attribute. In SelectionDAGISel.cpp, we set up swifterror values (SwiftErrorVals) before handling the basic blocks. When iterating over all basic blocks in RPO, before actually visiting the basic block, we call mergeIncomingSwiftErrors to merge incoming swifterror values when there are multiple predecessors or to simply propagate them. There, we create a virtual register for each swifterror value in the entry block. For predecessors that are not yet visited, we create virtual registers to hold the swifterror values at the end of the predecessor. The assignments are saved in SwiftErrorWorklist and will be materialized at the end of visiting the basic block. When visiting a load from a swifterror value, we copy from the current virtual register assignment. When visiting a store to a swifterror value, we create a virtual register to hold the swifterror value and update SwiftErrorMap to track the current virtual register assignment. Differential Revision: http://reviews.llvm.org/D18108 llvm-svn: 265433
2016-04-05 20:13:16 +02:00
setupSwiftErrorVals(Fn, TLI, FuncInfo);
Have isel visit blocks in reverse postorder rather than an undefined order. This allows for the information propagated across basic blocks to be merged at phis. llvm-svn: 126375
2011-02-24 11:00:04 +01:00
ReversePostOrderTraversal<const Function*> RPOT(&Fn);
Track blocks visited in reverse postorder. llvm-svn: 126377
2011-02-24 11:00:13 +01:00
[SDAGISel] Simplify some SDAGISel code, NFC Hoist entry block code for arguments and swift error values out of the basic block instruction selection loop. Lowering arguments once up front seems much more readable than doing it conditionally inside the loop. It also makes it clear that argument lowering can update StaticAllocaMap because no instructions have been selected yet. Also use range-based for loops where possible. llvm-svn: 294329
2017-02-07 19:42:53 +01:00
// Lower arguments up front. An RPO iteration always visits the entry block
// first.
assert(*RPOT.begin() == &Fn.getEntryBlock());
++NumEntryBlocks;
// Set up FuncInfo for ISel. Entry blocks never have PHIs.
FuncInfo->MBB = FuncInfo->MBBMap[&Fn.getEntryBlock()];
FuncInfo->InsertPt = FuncInfo->MBB->begin();
if (!FastIS) {
LowerArguments(Fn);
} else {
// See if fast isel can lower the arguments.
FastIS->startNewBlock();
if (!FastIS->lowerArguments()) {
[FastISel] Add a diagnostic to warm on fallback. This is consistent with what we do for GlobalISel. That way, it is easy to see whether or not FastISel is able to fully select a function. At some point we may want to switch that to an optimization remark. llvm-svn: 294970
2017-02-13 18:38:59 +01:00
FastISelFailed = true;
[SDAGISel] Simplify some SDAGISel code, NFC Hoist entry block code for arguments and swift error values out of the basic block instruction selection loop. Lowering arguments once up front seems much more readable than doing it conditionally inside the loop. It also makes it clear that argument lowering can update StaticAllocaMap because no instructions have been selected yet. Also use range-based for loops where possible. llvm-svn: 294329
2017-02-07 19:42:53 +01:00
// Fast isel failed to lower these arguments
++NumFastIselFailLowerArguments;
if (EnableFastISelAbort > 1)
report_fatal_error("FastISel didn't lower all arguments");
// Use SelectionDAG argument lowering
LowerArguments(Fn);
CurDAG->setRoot(SDB->getControlRoot());
SDB->clear();
CodeGenAndEmitDAG();
}
// If we inserted any instructions at the beginning, make a note of
// where they are, so we can be sure to emit subsequent instructions
// after them.
if (FuncInfo->InsertPt != FuncInfo->MBB->begin())
FastIS->setLastLocalValue(&*std::prev(FuncInfo->InsertPt));
else
FastIS->setLastLocalValue(nullptr);
}
createSwiftErrorEntriesInEntryBlock(FuncInfo, TLI, TII, SDB);
// Iterate over all basic blocks in the function.
for (const BasicBlock *LLVMBB : RPOT) {
Add a mechanism for invalidating the LiveOutInfo of a PHI, and use it whenever a block is visited before all of its predecessors. llvm-svn: 126378
2011-02-24 11:00:16 +01:00
if (OptLevel != CodeGenOpt::None) {
bool AllPredsVisited = true;
Revert "[C++11] Add predecessors(BasicBlock *) / successors(BasicBlock *) iterator ranges." This reverts commit r213474 (and r213475), which causes a miscompile on a stage2 LTO build. I'll reply on the list in a moment. llvm-svn: 213562
2014-07-21 19:06:51 +02:00
for (const_pred_iterator PI = pred_begin(LLVMBB), PE = pred_end(LLVMBB);
PI != PE; ++PI) {
if (!FuncInfo->VisitedBBs.count(*PI)) {
Add a mechanism for invalidating the LiveOutInfo of a PHI, and use it whenever a block is visited before all of its predecessors. llvm-svn: 126378
2011-02-24 11:00:16 +01:00
AllPredsVisited = false;
break;
}
}
Merge information about the number of zero, one, and sign bits of live-out registers at phis. This enables us to eliminate a lot of pointless zexts during the DAGCombine phase. This fixes <rdar://problem/8760114>. llvm-svn: 126380
2011-02-24 11:00:25 +01:00
if (AllPredsVisited) {
split a complex predicate out to a helper function. Simplify two for loops, which don't need to check for falling off the end of a block *and* end of phi nodes, since terminators are never phis. llvm-svn: 129655
2011-04-17 08:03:19 +02:00
for (BasicBlock::const_iterator I = LLVMBB->begin();
Use dyn_cast instead of isa and cast. No functionality change. llvm-svn: 169196
2012-12-04 01:50:06 +01:00
const PHINode *PN = dyn_cast<PHINode>(I); ++I)
FuncInfo->ComputePHILiveOutRegInfo(PN);
Merge information about the number of zero, one, and sign bits of live-out registers at phis. This enables us to eliminate a lot of pointless zexts during the DAGCombine phase. This fixes <rdar://problem/8760114>. llvm-svn: 126380
2011-02-24 11:00:25 +01:00
} else {
split a complex predicate out to a helper function. Simplify two for loops, which don't need to check for falling off the end of a block *and* end of phi nodes, since terminators are never phis. llvm-svn: 129655
2011-04-17 08:03:19 +02:00
for (BasicBlock::const_iterator I = LLVMBB->begin();
Use dyn_cast instead of isa and cast. No functionality change. llvm-svn: 169196
2012-12-04 01:50:06 +01:00
const PHINode *PN = dyn_cast<PHINode>(I); ++I)
FuncInfo->InvalidatePHILiveOutRegInfo(PN);
Add a mechanism for invalidating the LiveOutInfo of a PHI, and use it whenever a block is visited before all of its predecessors. llvm-svn: 126378
2011-02-24 11:00:16 +01:00
}
Track blocks visited in reverse postorder. llvm-svn: 126377
2011-02-24 11:00:13 +01:00
FuncInfo->VisitedBBs.insert(LLVMBB);
Add a mechanism for invalidating the LiveOutInfo of a PHI, and use it whenever a block is visited before all of its predecessors. llvm-svn: 126378
2011-02-24 11:00:16 +01:00
}
Track blocks visited in reverse postorder. llvm-svn: 126377
2011-02-24 11:00:13 +01:00
SelectionDAG: Remove implicit ilist iterator conversions, NFC llvm-svn: 250214
2015-10-13 21:47:46 +02:00
BasicBlock::const_iterator const Begin =
LLVMBB->getFirstNonPHI()->getIterator();
Add const qualifiers to CodeGen's use of LLVM IR constructs. llvm-svn: 101334
2010-04-15 03:51:59 +02:00
BasicBlock::const_iterator const End = LLVMBB->end();
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
BasicBlock::const_iterator BI = End;
Fix a FastISel bug where the instructions from lowering the arguments were being emitted after the first instructions of the entry block. llvm-svn: 55496
2008-08-28 22:28:56 +02:00
Remove duplicate line and move another closer to its actual use llvm-svn: 176391
2013-03-02 00:32:40 +01:00
FuncInfo->MBB = FuncInfo->MBBMap[LLVMBB];
[WinEH] Avoid creating MBBs for LLVM BBs that cannot contain code Typically these are catchpads, which hold data used to decide whether to catch the exception or continue unwinding. We also shouldn't create MBBs for catchendpads, cleanupendpads, or terminatepads, since no real code can live in them. This fixes a problem where MI passes (like the register allocator) would try to put code into catchpad blocks, which are not executed by the runtime. In the new world, blocks ending in invokes now have many possible successors. llvm-svn: 247102
2015-09-09 01:28:38 +02:00
if (!FuncInfo->MBB)
continue; // Some blocks like catchpads have no code or MBB.
[SDAGISel] Simplify some SDAGISel code, NFC Hoist entry block code for arguments and swift error values out of the basic block instruction selection loop. Lowering arguments once up front seems much more readable than doing it conditionally inside the loop. It also makes it clear that argument lowering can update StaticAllocaMap because no instructions have been selected yet. Also use range-based for loops where possible. llvm-svn: 294329
2017-02-07 19:42:53 +01:00
// Insert new instructions after any phi or argument setup code.
FuncInfo->InsertPt = FuncInfo->MBB->end();
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
// Setup an EH landing-pad block.
Simplify landing pad lowering. Stop using the ISD::EXCEPTIONADDR and ISD::EHSELECTION when lowering landing pad arguments. These nodes were previously legalized into CopyFromReg nodes, but that never worked properly because the CopyFromReg node weren't guaranteed to be scheduled at the top of the basic block. This meant the exception pointer and selector registers could be clobbered before being copied to a virtual register. This patch copies the two physical registers to virtual registers at the beginning of the basic block, and lowers the landingpad instruction directly to two CopyFromReg nodes reading the *virtual* registers. This is safe because virtual registers don't get clobbered. A future patch will remove the ISD::EXCEPTIONADDR and ISD::EHSELECTION nodes. llvm-svn: 185617
2013-07-04 06:53:45 +02:00
FuncInfo->ExceptionPointerVirtReg = 0;
FuncInfo->ExceptionSelectorVirtReg = 0;
Don't call PrepareEHLandingPad on non EH pads This was a minor bug in r249492. Calling PrepareEHLandingPad on a non-landingpad was a no-op, but it attempted to get the generic pointer register class, which apparently doesn't exist for some targets. llvm-svn: 250068
2015-10-12 19:42:32 +02:00
if (LLVMBB->isEHPad())
if (!PrepareEHLandingPad())
continue;
[WinEH] Add some support for code generating catchpad We can now run 32-bit programs with empty catch bodies. The next step is to change PEI so that we get funclet prologues and epilogues. llvm-svn: 246235
2015-08-28 01:27:47 +02:00
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
// Before doing SelectionDAG ISel, see if FastISel has been requested.
Remove the code for special-casing byval for fast-isel. SelectionDAG handles argument lowering anyway, so there's no need for special casing here. llvm-svn: 102828
2010-05-01 04:44:23 +02:00
if (FastIS) {
[SDAGISel] Simplify some SDAGISel code, NFC Hoist entry block code for arguments and swift error values out of the basic block instruction selection loop. Lowering arguments once up front seems much more readable than doing it conditionally inside the loop. It also makes it clear that argument lowering can update StaticAllocaMap because no instructions have been selected yet. Also use range-based for loops where possible. llvm-svn: 294329
2017-02-07 19:42:53 +01:00
if (LLVMBB != &Fn.getEntryBlock())
FastIS->startNewBlock();
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
Silence the unused variable warning. llvm-svn: 176218
2013-02-27 23:52:54 +01:00
unsigned NumFastIselRemaining = std::distance(Begin, End);
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
// Do FastISel on as many instructions as possible.
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
for (; BI != Begin; --BI) {
SelectionDAG: Remove implicit ilist iterator conversions, NFC llvm-svn: 250214
2015-10-13 21:47:46 +02:00
const Instruction *Inst = &*std::prev(BI);
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
// If we no longer require this instruction, skip it.
Fix the stats collection for fast-isel. The failed count was only accounting for a single miss and not all predecessor instructions that get selected by the selection DAG instruction selector. This is still not exact (e.g., over states misses when folded/dead instructions are present), but it is a step in the right direction. llvm-svn: 144832
2011-11-16 22:02:08 +01:00
if (isFoldedOrDeadInstruction(Inst, FuncInfo)) {
--NumFastIselRemaining;
Enable on-demand fast-isel. llvm-svn: 107377
2010-07-01 04:58:57 +02:00
continue;
Fix the stats collection for fast-isel. The failed count was only accounting for a single miss and not all predecessor instructions that get selected by the selection DAG instruction selector. This is still not exact (e.g., over states misses when folded/dead instructions are present), but it is a step in the right direction. llvm-svn: 144832
2011-11-16 22:02:08 +01:00
}
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
// Bottom-up: reset the insert pos at the top, after any local-value
// instructions.
FastIS->recomputeInsertPt();
Enable on-demand fast-isel. llvm-svn: 107377
2010-07-01 04:58:57 +02:00
Update a partially obsolete comment. llvm-svn: 93228
2010-01-12 05:32:35 +01:00
// Try to select the instruction with FastISel.
[FastISel] Rename public visible FastISel functions. NFC. This commit renames the following public FastISel functions: LowerArguments -> lowerArguments SelectInstruction -> selectInstruction TargetSelectInstruction -> fastSelectInstruction FastLowerArguments -> fastLowerArguments FastLowerCall -> fastLowerCall FastLowerIntrinsicCall -> fastLowerIntrinsicCall FastEmitZExtFromI1 -> fastEmitZExtFromI1 FastEmitBranch -> fastEmitBranch UpdateValueMap -> updateValueMap TargetMaterializeConstant -> fastMaterializeConstant TargetMaterializeAlloca -> fastMaterializeAlloca TargetMaterializeFloatZero -> fastMaterializeFloatZero LowerCallTo -> lowerCallTo Reviewed by Eric llvm-svn: 217074
2014-09-03 22:56:52 +02:00
if (FastIS->selectInstruction(Inst)) {
[FastISel] Add a diagnostic to warm on fallback. This is consistent with what we do for GlobalISel. That way, it is easy to see whether or not FastISel is able to fully select a function. At some point we may want to switch that to an optimization remark. llvm-svn: 294970
2017-02-13 18:38:59 +01:00
FastISelFailed = true;
Fix the stats collection for fast-isel. The failed count was only accounting for a single miss and not all predecessor instructions that get selected by the selection DAG instruction selector. This is still not exact (e.g., over states misses when folded/dead instructions are present), but it is a step in the right direction. llvm-svn: 144832
2011-11-16 22:02:08 +01:00
--NumFastIselRemaining;
Disable statistics on Release builds and move tests that depend on -stats. Summary: Statistics are still available in Release+Asserts (any +Asserts builds), and stats can also be turned on with LLVM_ENABLE_STATS. Move some of the FastISel stats that were moved under DEBUG() back out of DEBUG(), since stats are disabled across the board now. Many tests depend on grepping "-stats" output. Move those into a orig_dir/Stats/. so that they can be marked as unsupported when building without statistics. Differential Revision: http://llvm-reviews.chandlerc.com/D486 llvm-svn: 176733
2013-03-08 23:56:31 +01:00
++NumFastIselSuccess;
Recommit the fix for rdar://9289512 with a couple tweaks to fix bugs exposed by the gcc dejagnu testsuite: 1. The load may actually be used by a dead instruction, which would cause an assert. 2. The load may not be used by the current chain of instructions, and we could move it past a side-effecting instruction. Change how we process uses to define the problem away. llvm-svn: 130018
2011-04-22 23:59:37 +02:00
// If fast isel succeeded, skip over all the folded instructions, and
// then see if there is a load right before the selected instructions.
// Try to fold the load if so.
const Instruction *BeforeInst = Inst;
SelectionDAG: Remove implicit ilist iterator conversions, NFC llvm-svn: 250214
2015-10-13 21:47:46 +02:00
while (BeforeInst != &*Begin) {
BeforeInst = &*std::prev(BasicBlock::const_iterator(BeforeInst));
Recommit the fix for rdar://9289512 with a couple tweaks to fix bugs exposed by the gcc dejagnu testsuite: 1. The load may actually be used by a dead instruction, which would cause an assert. 2. The load may not be used by the current chain of instructions, and we could move it past a side-effecting instruction. Change how we process uses to define the problem away. llvm-svn: 130018
2011-04-22 23:59:37 +02:00
if (!isFoldedOrDeadInstruction(BeforeInst, FuncInfo))
break;
}
if (BeforeInst != Inst && isa<LoadInst>(BeforeInst) &&
BeforeInst->hasOneUse() &&
Move TryToFoldFastISelLoad to FastISel, where it belongs. In general, I'm trying to move as much FastISel logic as possible out of the main path in SelectionDAGISel - intermixing them just adds confusion. llvm-svn: 179902
2013-04-20 00:29:18 +02:00
FastIS->tryToFoldLoad(cast<LoadInst>(BeforeInst), Inst)) {
Recommit the fix for rdar://9289512 with a couple tweaks to fix bugs exposed by the gcc dejagnu testsuite: 1. The load may actually be used by a dead instruction, which would cause an assert. 2. The load may not be used by the current chain of instructions, and we could move it past a side-effecting instruction. Change how we process uses to define the problem away. llvm-svn: 130018
2011-04-22 23:59:37 +02:00
// If we succeeded, don't re-select the load.
[C++11] Replace llvm::next and llvm::prior with std::next and std::prev. Remove the old functions. llvm-svn: 202636
2014-03-02 13:27:27 +01:00
BI = std::next(BasicBlock::const_iterator(BeforeInst));
Fix the stats collection for fast-isel. The failed count was only accounting for a single miss and not all predecessor instructions that get selected by the selection DAG instruction selector. This is still not exact (e.g., over states misses when folded/dead instructions are present), but it is a step in the right direction. llvm-svn: 144832
2011-11-16 22:02:08 +01:00
--NumFastIselRemaining;
Disable statistics on Release builds and move tests that depend on -stats. Summary: Statistics are still available in Release+Asserts (any +Asserts builds), and stats can also be turned on with LLVM_ENABLE_STATS. Move some of the FastISel stats that were moved under DEBUG() back out of DEBUG(), since stats are disabled across the board now. Many tests depend on grepping "-stats" output. Move those into a orig_dir/Stats/. so that they can be marked as unsupported when building without statistics. Differential Revision: http://llvm-reviews.chandlerc.com/D486 llvm-svn: 176733
2013-03-08 23:56:31 +01:00
++NumFastIselSuccess;
Fix the stats collection for fast-isel. The failed count was only accounting for a single miss and not all predecessor instructions that get selected by the selection DAG instruction selector. This is still not exact (e.g., over states misses when folded/dead instructions are present), but it is a step in the right direction. llvm-svn: 144832
2011-11-16 22:02:08 +01:00
}
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
continue;
implement rdar://6653118 - fastisel should fold loads where possible. Since mem2reg isn't run at -O0, we get a ton of reloads from the stack, for example, before, this code: int foo(int x, int y, int z) { return x+y+z; } used to compile into: _foo: ## @foo subq $12, %rsp movl %edi, 8(%rsp) movl %esi, 4(%rsp) movl %edx, (%rsp) movl 8(%rsp), %edx movl 4(%rsp), %esi addl %edx, %esi movl (%rsp), %edx addl %esi, %edx movl %edx, %eax addq $12, %rsp ret Now we produce: _foo: ## @foo subq $12, %rsp movl %edi, 8(%rsp) movl %esi, 4(%rsp) movl %edx, (%rsp) movl 8(%rsp), %edx addl 4(%rsp), %edx ## Folded load addl (%rsp), %edx ## Folded load movl %edx, %eax addq $12, %rsp ret Fewer instructions and less register use = faster compiles. llvm-svn: 113102
2010-09-05 04:18:34 +02:00
}
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
#ifndef NDEBUG
[fast-isel] Guard "exhastive" fast-isel output with -fast-isel-verbose2. llvm-svn: 146453
2011-12-13 01:05:11 +01:00
if (EnableFastISelVerbose2)
collectFailStats(Inst);
Add rather verbose stats for fast-isel failures. llvm-svn: 146186
2011-12-08 22:37:10 +01:00
#endif
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
// Then handle certain instructions as single-LLVM-Instruction blocks.
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
if (isa<CallInst>(Inst)) {
Fix the stats collection for fast-isel. The failed count was only accounting for a single miss and not all predecessor instructions that get selected by the selection DAG instruction selector. This is still not exact (e.g., over states misses when folded/dead instructions are present), but it is a step in the right direction. llvm-svn: 144832
2011-11-16 22:02:08 +01:00
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
if (EnableFastISelVerbose || EnableFastISelAbort) {
Change errs() to dbgs(). llvm-svn: 92597
2010-01-05 02:26:11 +01:00
dbgs() << "FastISel missed call: ";
Cleanup dump() functions. We had various variants of defining dump() functions in LLVM. Normalize them (this should just consistently implement the things discussed in http://lists.llvm.org/pipermail/cfe-dev/2014-January/034323.html For reference: - Public headers should just declare the dump() method but not use LLVM_DUMP_METHOD or #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) - The definition of a dump method should look like this: #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) LLVM_DUMP_METHOD void MyClass::dump() { // print stuff to dbgs()... } #endif llvm-svn: 293359
2017-01-28 03:02:38 +01:00
Inst->print(dbgs());
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
}
Change the fast-isel-abort option from bool to int to enable "levels" Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
2015-02-27 19:32:11 +01:00
if (EnableFastISelAbort > 2)
// FastISel selector couldn't handle something and bailed.
// For the purpose of debugging, just abort.
Use report_fatal_error instead of unreachable for -fast-isel-abort Suggestion by Andrea Di Biagio From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 231201
2015-03-04 02:48:39 +01:00
report_fatal_error("FastISel didn't select the entire block");
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
[WinEH] Add some support for code generating catchpad We can now run 32-bit programs with empty catch bodies. The next step is to change PEI so that we get funclet prologues and epilogues. llvm-svn: 246235
2015-08-28 01:27:47 +02:00
if (!Inst->getType()->isVoidTy() && !Inst->getType()->isTokenTy() &&
!Inst->use_empty()) {
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
unsigned &R = FuncInfo->ValueMap[Inst];
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
if (!R)
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
R = FuncInfo->CreateRegs(Inst->getType());
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
}
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
Fix fast-isel to avoid selecting the return instruction if a tail call has been encountered. llvm-svn: 89444
2009-11-20 03:51:26 +01:00
bool HadTailCall = false;
Fall back to the selection dag isel to select tail calls. This shouldn't affect codegen for -O0 compiles as tail call markers are not emitted in unoptimized compiles. Testing with the external/internal nightly test suite reveals no change in compile time performance. Testing with -O1, -O2 and -O3 with fast-isel enabled did not cause any compile-time or execution-time failures. All tests were performed on my x86 machine. I'll monitor our arm testers to ensure no regressions occur there. In an upcoming clang patch I will be marking the objc_autoreleaseReturnValue and objc_retainAutoreleaseReturnValue as tail calls unconditionally. While it's theoretically true that this is just an optimization, it's an optimization that we very much want to happen even at -O0, or else ARC applications become substantially harder to debug. Part of rdar://12553082 llvm-svn: 169796
2012-12-11 01:18:02 +01:00
MachineBasicBlock::iterator SavedInsertPt = FuncInfo->InsertPt;
SelectionDAG: Remove implicit ilist iterator conversions, NFC llvm-svn: 250214
2015-10-13 21:47:46 +02:00
SelectBasicBlock(Inst->getIterator(), BI, HadTailCall);
Fix fast-isel to avoid selecting the return instruction if a tail call has been encountered. llvm-svn: 89444
2009-11-20 03:51:26 +01:00
// If the call was emitted as a tail call, we're done with the block.
Fall back to the selection dag isel to select tail calls. This shouldn't affect codegen for -O0 compiles as tail call markers are not emitted in unoptimized compiles. Testing with the external/internal nightly test suite reveals no change in compile time performance. Testing with -O1, -O2 and -O3 with fast-isel enabled did not cause any compile-time or execution-time failures. All tests were performed on my x86 machine. I'll monitor our arm testers to ensure no regressions occur there. In an upcoming clang patch I will be marking the objc_autoreleaseReturnValue and objc_retainAutoreleaseReturnValue as tail calls unconditionally. While it's theoretically true that this is just an optimization, it's an optimization that we very much want to happen even at -O0, or else ARC applications become substantially harder to debug. Part of rdar://12553082 llvm-svn: 169796
2012-12-11 01:18:02 +01:00
// We also need to delete any previously emitted instructions.
Fix fast-isel to avoid selecting the return instruction if a tail call has been encountered. llvm-svn: 89444
2009-11-20 03:51:26 +01:00
if (HadTailCall) {
Fall back to the selection dag isel to select tail calls. This shouldn't affect codegen for -O0 compiles as tail call markers are not emitted in unoptimized compiles. Testing with the external/internal nightly test suite reveals no change in compile time performance. Testing with -O1, -O2 and -O3 with fast-isel enabled did not cause any compile-time or execution-time failures. All tests were performed on my x86 machine. I'll monitor our arm testers to ensure no regressions occur there. In an upcoming clang patch I will be marking the objc_autoreleaseReturnValue and objc_retainAutoreleaseReturnValue as tail calls unconditionally. While it's theoretically true that this is just an optimization, it's an optimization that we very much want to happen even at -O0, or else ARC applications become substantially harder to debug. Part of rdar://12553082 llvm-svn: 169796
2012-12-11 01:18:02 +01:00
FastIS->removeDeadCode(SavedInsertPt, FuncInfo->MBB->end());
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
--BI;
Fix fast-isel to avoid selecting the return instruction if a tail call has been encountered. llvm-svn: 89444
2009-11-20 03:51:26 +01:00
break;
}
Fall back to the selection dag isel to select tail calls. This shouldn't affect codegen for -O0 compiles as tail call markers are not emitted in unoptimized compiles. Testing with the external/internal nightly test suite reveals no change in compile time performance. Testing with -O1, -O2 and -O3 with fast-isel enabled did not cause any compile-time or execution-time failures. All tests were performed on my x86 machine. I'll monitor our arm testers to ensure no regressions occur there. In an upcoming clang patch I will be marking the objc_autoreleaseReturnValue and objc_retainAutoreleaseReturnValue as tail calls unconditionally. While it's theoretically true that this is just an optimization, it's an optimization that we very much want to happen even at -O0, or else ARC applications become substantially harder to debug. Part of rdar://12553082 llvm-svn: 169796
2012-12-11 01:18:02 +01:00
// Recompute NumFastIselRemaining as Selection DAG instruction
// selection may have handled the call, input args, etc.
unsigned RemainingNow = std::distance(Begin, BI);
Disable statistics on Release builds and move tests that depend on -stats. Summary: Statistics are still available in Release+Asserts (any +Asserts builds), and stats can also be turned on with LLVM_ENABLE_STATS. Move some of the FastISel stats that were moved under DEBUG() back out of DEBUG(), since stats are disabled across the board now. Many tests depend on grepping "-stats" output. Move those into a orig_dir/Stats/. so that they can be marked as unsupported when building without statistics. Differential Revision: http://llvm-reviews.chandlerc.com/D486 llvm-svn: 176733
2013-03-08 23:56:31 +01:00
NumFastIselFailures += NumFastIselRemaining - RemainingNow;
NumFastIselRemaining = RemainingNow;
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
continue;
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
}
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
Fixup for recent -fast-isel-abort change: code didn't match description Level 1 should abort for all instructions but call/terminators/args. Instead it was aborting only if the level was > 2 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230861
2015-02-28 20:34:54 +01:00
bool ShouldAbort = EnableFastISelAbort;
Change the fast-isel-abort option from bool to int to enable "levels" Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
2015-02-27 19:32:11 +01:00
if (EnableFastISelVerbose || EnableFastISelAbort) {
if (isa<TerminatorInst>(Inst)) {
// Use a different message for terminator misses.
Make fast-isel miss counting in -stats and -fast-isel-verbose take terminators into account; since there are many fewer isel misses with recent changes, misses caused by terminators are more significant. llvm-svn: 131502
2011-05-18 01:02:10 +02:00
dbgs() << "FastISel missed terminator: ";
Fixup for recent -fast-isel-abort change: code didn't match description Level 1 should abort for all instructions but call/terminators/args. Instead it was aborting only if the level was > 2 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230861
2015-02-28 20:34:54 +01:00
// Don't abort unless for terminator unless the level is really high
ShouldAbort = (EnableFastISelAbort > 2);
Change the fast-isel-abort option from bool to int to enable "levels" Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
2015-02-27 19:32:11 +01:00
} else {
Change errs() to dbgs(). llvm-svn: 92597
2010-01-05 02:26:11 +01:00
dbgs() << "FastISel miss: ";
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
}
Cleanup dump() functions. We had various variants of defining dump() functions in LLVM. Normalize them (this should just consistently implement the things discussed in http://lists.llvm.org/pipermail/cfe-dev/2014-January/034323.html For reference: - Public headers should just declare the dump() method but not use LLVM_DUMP_METHOD or #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) - The definition of a dump method should look like this: #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP) LLVM_DUMP_METHOD void MyClass::dump() { // print stuff to dbgs()... } #endif llvm-svn: 293359
2017-01-28 03:02:38 +01:00
Inst->print(dbgs());
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
}
Fixup for recent -fast-isel-abort change: code didn't match description Level 1 should abort for all instructions but call/terminators/args. Instead it was aborting only if the level was > 2 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230861
2015-02-28 20:34:54 +01:00
if (ShouldAbort)
Change the fast-isel-abort option from bool to int to enable "levels" Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
2015-02-27 19:32:11 +01:00
// FastISel selector couldn't handle something and bailed.
// For the purpose of debugging, just abort.
Use report_fatal_error instead of unreachable for -fast-isel-abort Suggestion by Andrea Di Biagio From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 231201
2015-03-04 02:48:39 +01:00
report_fatal_error("FastISel didn't select the entire block");
Change the fast-isel-abort option from bool to int to enable "levels" Summary: Currently fast-isel-abort will only abort for regular instructions, and just warn for function calls, terminators, function arguments. There is already fast-isel-abort-args but nothing for calls and terminators. This change turns the fast-isel-abort options into an integer option, so that multiple levels of strictness can be defined. This will help no being surprised when the "abort" option indeed does not abort, and enables the possibility to write test that verifies that no intrinsics are forgotten by fast-isel. Reviewers: resistor, echristo Subscribers: jfb, llvm-commits Differential Revision: http://reviews.llvm.org/D7941 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 230775
2015-02-27 19:32:11 +01:00
NumFastIselFailures += NumFastIselRemaining;
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
break;
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
}
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FastIS->recomputeInsertPt();
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
}
[SDAGISel] Simplify some SDAGISel code, NFC Hoist entry block code for arguments and swift error values out of the basic block instruction selection loop. Lowering arguments once up front seems much more readable than doing it conditionally inside the loop. It also makes it clear that argument lowering can update StaticAllocaMap because no instructions have been selected yet. Also use range-based for loops where possible. llvm-svn: 294329
2017-02-07 19:42:53 +01:00
[stack-protection] Add support for MSVC buffer security check Summary: This patch is adding support for the MSVC buffer security check implementation The buffer security check is turned on with the '/GS' compiler switch. * https://msdn.microsoft.com/en-us/library/8dbf701c.aspx * To be added to clang here: http://reviews.llvm.org/D20347 Some overview of buffer security check feature and implementation: * https://msdn.microsoft.com/en-us/library/aa290051(VS.71).aspx * http://www.ksyash.com/2011/01/buffer-overflow-protection-3/ * http://blog.osom.info/2012/02/understanding-vs-c-compilers-buffer.html For the following example: ``` int example(int offset, int index) { char buffer[10]; memset(buffer, 0xCC, index); return buffer[index]; } ``` The MSVC compiler is adding these instructions to perform stack integrity check: ``` push ebp mov ebp,esp sub esp,50h [1] mov eax,dword ptr [__security_cookie (01068024h)] [2] xor eax,ebp [3] mov dword ptr [ebp-4],eax push ebx push esi push edi mov eax,dword ptr [index] push eax push 0CCh lea ecx,[buffer] push ecx call _memset (010610B9h) add esp,0Ch mov eax,dword ptr [index] movsx eax,byte ptr buffer[eax] pop edi pop esi pop ebx [4] mov ecx,dword ptr [ebp-4] [5] xor ecx,ebp [6] call @__security_check_cookie@4 (01061276h) mov esp,ebp pop ebp ret ``` The instrumentation above is: * [1] is loading the global security canary, * [3] is storing the local computed ([2]) canary to the guard slot, * [4] is loading the guard slot and ([5]) re-compute the global canary, * [6] is validating the resulting canary with the '__security_check_cookie' and performs error handling. Overview of the current stack-protection implementation: * lib/CodeGen/StackProtector.cpp * There is a default stack-protection implementation applied on intermediate representation. * The target can overload 'getIRStackGuard' method if it has a standard location for the stack protector cookie. * An intrinsic 'Intrinsic::stackprotector' is added to the prologue. It will be expanded by the instruction selection pass (DAG or Fast). * Basic Blocks are added to every instrumented function to receive the code for handling stack guard validation and errors handling. * Guard manipulation and comparison are added directly to the intermediate representation. * lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp * lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp * There is an implementation that adds instrumentation during instruction selection (for better handling of sibbling calls). * see long comment above 'class StackProtectorDescriptor' declaration. * The target needs to override 'getSDagStackGuard' to activate SDAG stack protection generation. (note: getIRStackGuard MUST be nullptr). * 'getSDagStackGuard' returns the appropriate stack guard (security cookie) * The code is generated by 'SelectionDAGBuilder.cpp' and 'SelectionDAGISel.cpp'. * include/llvm/Target/TargetLowering.h * Contains function to retrieve the default Guard 'Value'; should be overriden by each target to select which implementation is used and provide Guard 'Value'. * lib/Target/X86/X86ISelLowering.cpp * Contains the x86 specialisation; Guard 'Value' used by the SelectionDAG algorithm. Function-based Instrumentation: * The MSVC doesn't inline the stack guard comparison in every function. Instead, a call to '__security_check_cookie' is added to the epilogue before every return instructions. * To support function-based instrumentation, this patch is * adding a function to get the function-based check (llvm 'Value', see include/llvm/Target/TargetLowering.h), * If provided, the stack protection instrumentation won't be inlined and a call to that function will be added to the prologue. * modifying (SelectionDAGISel.cpp) do avoid producing basic blocks used for inline instrumentation, * generating the function-based instrumentation during the ISEL pass (SelectionDAGBuilder.cpp), * if FastISEL (not SelectionDAG), using the fallback which rely on the same function-based implemented over intermediate representation (StackProtector.cpp). Modifications * adding support for MSVC (lib/Target/X86/X86ISelLowering.cpp) * adding support function-based instrumentation (lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp, .h) Results * IR generated instrumentation: ``` clang-cl /GS test.cc /Od /c -mllvm -print-isel-input ``` ``` *** Final LLVM Code input to ISel *** ; Function Attrs: nounwind sspstrong define i32 @"\01?example@@YAHHH@Z"(i32 %offset, i32 %index) #0 { entry: %StackGuardSlot = alloca i8* <<<-- Allocated guard slot %0 = call i8* @llvm.stackguard() <<<-- Loading Stack Guard value call void @llvm.stackprotector(i8* %0, i8** %StackGuardSlot) <<<-- Prologue intrinsic call (store to Guard slot) %index.addr = alloca i32, align 4 %offset.addr = alloca i32, align 4 %buffer = alloca [10 x i8], align 1 store i32 %index, i32* %index.addr, align 4 store i32 %offset, i32* %offset.addr, align 4 %arraydecay = getelementptr inbounds [10 x i8], [10 x i8]* %buffer, i32 0, i32 0 %1 = load i32, i32* %index.addr, align 4 call void @llvm.memset.p0i8.i32(i8* %arraydecay, i8 -52, i32 %1, i32 1, i1 false) %2 = load i32, i32* %index.addr, align 4 %arrayidx = getelementptr inbounds [10 x i8], [10 x i8]* %buffer, i32 0, i32 %2 %3 = load i8, i8* %arrayidx, align 1 %conv = sext i8 %3 to i32 %4 = load volatile i8*, i8** %StackGuardSlot <<<-- Loading Guard slot call void @__security_check_cookie(i8* %4) <<<-- Epilogue function-based check ret i32 %conv } ``` * SelectionDAG generated instrumentation: ``` clang-cl /GS test.cc /O1 /c /FA ``` ``` "?example@@YAHHH@Z": # @"\01?example@@YAHHH@Z" # BB#0: # %entry pushl %esi subl $16, %esp movl ___security_cookie, %eax <<<-- Loading Stack Guard value movl 28(%esp), %esi movl %eax, 12(%esp) <<<-- Store to Guard slot leal 2(%esp), %eax pushl %esi pushl $204 pushl %eax calll _memset addl $12, %esp movsbl 2(%esp,%esi), %esi movl 12(%esp), %ecx <<<-- Loading Guard slot calll @__security_check_cookie@4 <<<-- Epilogue function-based check movl %esi, %eax addl $16, %esp popl %esi retl ``` Reviewers: kcc, pcc, eugenis, rnk Subscribers: majnemer, llvm-commits, hans, thakis, rnk Differential Revision: http://reviews.llvm.org/D20346 llvm-svn: 272053
2016-06-07 22:15:35 +02:00
if (getAnalysis<StackProtector>().shouldEmitSDCheck(*LLVMBB)) {
bool FunctionBasedInstrumentation =
TLI->getSSPStackGuardCheck(*Fn.getParent());
SDB->SPDescriptor.initialize(LLVMBB, FuncInfo->MBBMap[LLVMBB],
FunctionBasedInstrumentation);
}
Move the point at which FastISel taps into the SelectionDAGISel process up to a higher level. This allows FastISel to leverage more of SelectionDAGISel's infastructure, such as updating Machine PHI nodes. Also, implement transitioning from SDISel back to FastISel in the middle of a block, so it's now possible to go back and forth. This allows FastISel to hand individual CallInsts and other complicated things off to SDISel to handle, while handling the rest of the block itself. To help support this, reorganize the SelectionDAG class so that it is allocated once and reused throughout a function, instead of being completely reallocated for each block. llvm-svn: 55219
2008-08-23 04:25:05 +02:00
Simplify. Do not count use of sdisel for single call instruction. llvm-svn: 117316
2010-10-25 23:31:46 +02:00
if (Begin != BI)
++NumDAGBlocks;
else
++NumFastIselBlocks;
SelectBasicBlock is rather slow even when it doesn't do anything; skip the unnecessary work where possible. llvm-svn: 129763
2011-04-19 19:01:08 +02:00
if (Begin != BI) {
// Run SelectionDAG instruction selection on the remainder of the block
// not handled by FastISel. If FastISel is not run, this is the entire
// block.
bool HadTailCall;
SelectBasicBlock(Begin, BI, HadTailCall);
}
Add skeleton of simple basic block instruction selector. llvm-svn: 54522
2008-08-08 09:27:28 +02:00
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FinishBasicBlock();
Move PHINodesToUpdate out of SelectionDAGBuilder and into FunctionLoweringInfo, as it isn't SelectionDAG-specific. This isn't completely natural, as PHI node state is not per-function but rather per-basic-block, however there's currently no other convenient per-basic-block state to group it with. llvm-svn: 102109
2010-04-22 21:55:20 +02:00
FuncInfo->PHINodesToUpdate.clear();
Factor code that finalize PHI nodes, jump tables, etc. out of SelectBasicBlock. No functionality changes. llvm-svn: 54438
2008-08-07 02:43:25 +02:00
}
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
swifterror: Don't compute swifterror vregs during instruction selection The code used llvm basic block predecessors to decided where to insert phi nodes. Instruction selection can and will liberally insert new machine basic block predecessors. There is not a guaranteed one-to-one mapping from pred. llvm basic blocks and machine basic blocks. Therefore the current approach does not work as it assumes we can mark predecessor machine basic block as needing a copy, and needs to know the set of all predecessor machine basic blocks to decide when to insert phis. Instead of computing the swifterror vregs as we select instructions, propagate them at the end of instruction selection when the MBB CFG is complete. When an instruction needs a swifterror vreg and we don't know the value yet, generate a new vreg and remember this "upward exposed" use, and reconcile this at the end of instruction selection. This will only happen if the target supports promoting swifterror parameters to registers and the swifterror attribute is used. rdar://28300923 llvm-svn: 283617
2016-10-08 00:06:55 +02:00
propagateSwiftErrorVRegs(FuncInfo);
Fix FastISel to not initialize the PIC-base register multiple times in functions with PIC references from more than one basic block. llvm-svn: 56807
2008-09-29 23:55:50 +02:00
delete FastIS;
Preserve debug info during iSel by keeping DanglingDebugInfoMap live until end of function. Patch by Micah Villmow llvm-svn: 131908
2011-05-23 19:44:13 +02:00
SDB->clearDanglingDebugInfo();
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
SDB->SPDescriptor.resetPerFunctionState();
}
[stackprotector] When finding the split point to splice off the end of a parentmbb into a successmbb, include any DBG_VALUE MI. Fix for PR16954. llvm-svn: 188987
2013-08-22 07:40:50 +02:00
/// Given that the input MI is before a partial terminator sequence TSeq, return
/// true if M + TSeq also a partial terminator sequence.
///
/// A Terminator sequence is a sequence of MachineInstrs which at this point in
/// lowering copy vregs into physical registers, which are then passed into
/// terminator instructors so we can satisfy ABI constraints. A partial
/// terminator sequence is an improper subset of a terminator sequence (i.e. it
/// may be the whole terminator sequence).
SelectionDAG: Avoid implicit iterator conversions in SelectionDAGISel, NFC llvm-svn: 274904
2016-07-08 21:11:40 +02:00
static bool MIIsInTerminatorSequence(const MachineInstr &MI) {
[stackprotector] When finding the split point to splice off the end of a parentmbb into a successmbb, include any DBG_VALUE MI. Fix for PR16954. llvm-svn: 188987
2013-08-22 07:40:50 +02:00
// If we do not have a copy or an implicit def, we return true if and only if
// MI is a debug value.
SelectionDAG: Avoid implicit iterator conversions in SelectionDAGISel, NFC llvm-svn: 274904
2016-07-08 21:11:40 +02:00
if (!MI.isCopy() && !MI.isImplicitDef())
[stackprotector] When finding the split point to splice off the end of a parentmbb into a successmbb, include any DBG_VALUE MI. Fix for PR16954. llvm-svn: 188987
2013-08-22 07:40:50 +02:00
// Sometimes DBG_VALUE MI sneak in between the copies from the vregs to the
// physical registers if there is debug info associated with the terminator
// of our mbb. We want to include said debug info in our terminator
// sequence, so we return true in that case.
SelectionDAG: Avoid implicit iterator conversions in SelectionDAGISel, NFC llvm-svn: 274904
2016-07-08 21:11:40 +02:00
return MI.isDebugValue();
[stackprotector] When finding the split point to splice off the end of a parentmbb into a successmbb, include any DBG_VALUE MI. Fix for PR16954. llvm-svn: 188987
2013-08-22 07:40:50 +02:00
[stackprotector] Allow for copies from vreg -> vreg to be in a terminator sequence. Sometimes a copy from a vreg -> vreg sneaks into the middle of a terminator sequence. It is safe to slice this into the stack protector success bb. This fixes PR16979. llvm-svn: 191260
2013-09-24 03:50:26 +02:00
// We have left the terminator sequence if we are not doing one of the
// following:
//
// 1. Copying a vreg into a physical register.
// 2. Copying a vreg into a vreg.
// 3. Defining a register via an implicit def.
// OPI should always be a register definition...
SelectionDAG: Avoid implicit iterator conversions in SelectionDAGISel, NFC llvm-svn: 274904
2016-07-08 21:11:40 +02:00
MachineInstr::const_mop_iterator OPI = MI.operands_begin();
whitespace llvm-svn: 191422
2013-09-26 07:53:31 +02:00
if (!OPI->isReg() || !OPI->isDef())
[stackprotector] Allow for copies from vreg -> vreg to be in a terminator sequence. Sometimes a copy from a vreg -> vreg sneaks into the middle of a terminator sequence. It is safe to slice this into the stack protector success bb. This fixes PR16979. llvm-svn: 191260
2013-09-24 03:50:26 +02:00
return false;
// Defining any register via an implicit def is always ok.
SelectionDAG: Avoid implicit iterator conversions in SelectionDAGISel, NFC llvm-svn: 274904
2016-07-08 21:11:40 +02:00
if (MI.isImplicitDef())
[stackprotector] Allow for copies from vreg -> vreg to be in a terminator sequence. Sometimes a copy from a vreg -> vreg sneaks into the middle of a terminator sequence. It is safe to slice this into the stack protector success bb. This fixes PR16979. llvm-svn: 191260
2013-09-24 03:50:26 +02:00
return true;
// Grab the copy source...
MachineInstr::const_mop_iterator OPI2 = OPI;
++OPI2;
SelectionDAG: Avoid implicit iterator conversions in SelectionDAGISel, NFC llvm-svn: 274904
2016-07-08 21:11:40 +02:00
assert(OPI2 != MI.operands_end()
[stackprotector] Allow for copies from vreg -> vreg to be in a terminator sequence. Sometimes a copy from a vreg -> vreg sneaks into the middle of a terminator sequence. It is safe to slice this into the stack protector success bb. This fixes PR16979. llvm-svn: 191260
2013-09-24 03:50:26 +02:00
&& "Should have a copy implying we should have 2 arguments.");
// Make sure that the copy dest is not a vreg when the copy source is a
// physical register.
if (!OPI2->isReg() ||
[stackprotector] When finding the split point to splice off the end of a parentmbb into a successmbb, include any DBG_VALUE MI. Fix for PR16954. llvm-svn: 188987
2013-08-22 07:40:50 +02:00
(!TargetRegisterInfo::isPhysicalRegister(OPI->getReg()) &&
[stackprotector] Allow for copies from vreg -> vreg to be in a terminator sequence. Sometimes a copy from a vreg -> vreg sneaks into the middle of a terminator sequence. It is safe to slice this into the stack protector success bb. This fixes PR16979. llvm-svn: 191260
2013-09-24 03:50:26 +02:00
TargetRegisterInfo::isPhysicalRegister(OPI2->getReg())))
[stackprotector] When finding the split point to splice off the end of a parentmbb into a successmbb, include any DBG_VALUE MI. Fix for PR16954. llvm-svn: 188987
2013-08-22 07:40:50 +02:00
return false;
return true;
}
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
/// Find the split point at which to splice the end of BB into its success stack
/// protector check machine basic block.
[stackprotector] When finding the split point to splice off the end of a parentmbb into a successmbb, include any DBG_VALUE MI. Fix for PR16954. llvm-svn: 188987
2013-08-22 07:40:50 +02:00
///
/// On many platforms, due to ABI constraints, terminators, even before register
/// allocation, use physical registers. This creates an issue for us since
/// physical registers at this point can not travel across basic
/// blocks. Luckily, selectiondag always moves physical registers into vregs
/// when they enter functions and moves them through a sequence of copies back
/// into the physical registers right before the terminator creating a
/// ``Terminator Sequence''. This function is searching for the beginning of the
/// terminator sequence so that we can ensure that we splice off not just the
/// terminator, but additionally the copies that move the vregs into the
/// physical registers.
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
static MachineBasicBlock::iterator
[stack-protection] Add support for MSVC buffer security check Summary: This patch is adding support for the MSVC buffer security check implementation The buffer security check is turned on with the '/GS' compiler switch. * https://msdn.microsoft.com/en-us/library/8dbf701c.aspx * To be added to clang here: http://reviews.llvm.org/D20347 Some overview of buffer security check feature and implementation: * https://msdn.microsoft.com/en-us/library/aa290051(VS.71).aspx * http://www.ksyash.com/2011/01/buffer-overflow-protection-3/ * http://blog.osom.info/2012/02/understanding-vs-c-compilers-buffer.html For the following example: ``` int example(int offset, int index) { char buffer[10]; memset(buffer, 0xCC, index); return buffer[index]; } ``` The MSVC compiler is adding these instructions to perform stack integrity check: ``` push ebp mov ebp,esp sub esp,50h [1] mov eax,dword ptr [__security_cookie (01068024h)] [2] xor eax,ebp [3] mov dword ptr [ebp-4],eax push ebx push esi push edi mov eax,dword ptr [index] push eax push 0CCh lea ecx,[buffer] push ecx call _memset (010610B9h) add esp,0Ch mov eax,dword ptr [index] movsx eax,byte ptr buffer[eax] pop edi pop esi pop ebx [4] mov ecx,dword ptr [ebp-4] [5] xor ecx,ebp [6] call @__security_check_cookie@4 (01061276h) mov esp,ebp pop ebp ret ``` The instrumentation above is: * [1] is loading the global security canary, * [3] is storing the local computed ([2]) canary to the guard slot, * [4] is loading the guard slot and ([5]) re-compute the global canary, * [6] is validating the resulting canary with the '__security_check_cookie' and performs error handling. Overview of the current stack-protection implementation: * lib/CodeGen/StackProtector.cpp * There is a default stack-protection implementation applied on intermediate representation. * The target can overload 'getIRStackGuard' method if it has a standard location for the stack protector cookie. * An intrinsic 'Intrinsic::stackprotector' is added to the prologue. It will be expanded by the instruction selection pass (DAG or Fast). * Basic Blocks are added to every instrumented function to receive the code for handling stack guard validation and errors handling. * Guard manipulation and comparison are added directly to the intermediate representation. * lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp * lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp * There is an implementation that adds instrumentation during instruction selection (for better handling of sibbling calls). * see long comment above 'class StackProtectorDescriptor' declaration. * The target needs to override 'getSDagStackGuard' to activate SDAG stack protection generation. (note: getIRStackGuard MUST be nullptr). * 'getSDagStackGuard' returns the appropriate stack guard (security cookie) * The code is generated by 'SelectionDAGBuilder.cpp' and 'SelectionDAGISel.cpp'. * include/llvm/Target/TargetLowering.h * Contains function to retrieve the default Guard 'Value'; should be overriden by each target to select which implementation is used and provide Guard 'Value'. * lib/Target/X86/X86ISelLowering.cpp * Contains the x86 specialisation; Guard 'Value' used by the SelectionDAG algorithm. Function-based Instrumentation: * The MSVC doesn't inline the stack guard comparison in every function. Instead, a call to '__security_check_cookie' is added to the epilogue before every return instructions. * To support function-based instrumentation, this patch is * adding a function to get the function-based check (llvm 'Value', see include/llvm/Target/TargetLowering.h), * If provided, the stack protection instrumentation won't be inlined and a call to that function will be added to the prologue. * modifying (SelectionDAGISel.cpp) do avoid producing basic blocks used for inline instrumentation, * generating the function-based instrumentation during the ISEL pass (SelectionDAGBuilder.cpp), * if FastISEL (not SelectionDAG), using the fallback which rely on the same function-based implemented over intermediate representation (StackProtector.cpp). Modifications * adding support for MSVC (lib/Target/X86/X86ISelLowering.cpp) * adding support function-based instrumentation (lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp, .h) Results * IR generated instrumentation: ``` clang-cl /GS test.cc /Od /c -mllvm -print-isel-input ``` ``` *** Final LLVM Code input to ISel *** ; Function Attrs: nounwind sspstrong define i32 @"\01?example@@YAHHH@Z"(i32 %offset, i32 %index) #0 { entry: %StackGuardSlot = alloca i8* <<<-- Allocated guard slot %0 = call i8* @llvm.stackguard() <<<-- Loading Stack Guard value call void @llvm.stackprotector(i8* %0, i8** %StackGuardSlot) <<<-- Prologue intrinsic call (store to Guard slot) %index.addr = alloca i32, align 4 %offset.addr = alloca i32, align 4 %buffer = alloca [10 x i8], align 1 store i32 %index, i32* %index.addr, align 4 store i32 %offset, i32* %offset.addr, align 4 %arraydecay = getelementptr inbounds [10 x i8], [10 x i8]* %buffer, i32 0, i32 0 %1 = load i32, i32* %index.addr, align 4 call void @llvm.memset.p0i8.i32(i8* %arraydecay, i8 -52, i32 %1, i32 1, i1 false) %2 = load i32, i32* %index.addr, align 4 %arrayidx = getelementptr inbounds [10 x i8], [10 x i8]* %buffer, i32 0, i32 %2 %3 = load i8, i8* %arrayidx, align 1 %conv = sext i8 %3 to i32 %4 = load volatile i8*, i8** %StackGuardSlot <<<-- Loading Guard slot call void @__security_check_cookie(i8* %4) <<<-- Epilogue function-based check ret i32 %conv } ``` * SelectionDAG generated instrumentation: ``` clang-cl /GS test.cc /O1 /c /FA ``` ``` "?example@@YAHHH@Z": # @"\01?example@@YAHHH@Z" # BB#0: # %entry pushl %esi subl $16, %esp movl ___security_cookie, %eax <<<-- Loading Stack Guard value movl 28(%esp), %esi movl %eax, 12(%esp) <<<-- Store to Guard slot leal 2(%esp), %eax pushl %esi pushl $204 pushl %eax calll _memset addl $12, %esp movsbl 2(%esp,%esi), %esi movl 12(%esp), %ecx <<<-- Loading Guard slot calll @__security_check_cookie@4 <<<-- Epilogue function-based check movl %esi, %eax addl $16, %esp popl %esi retl ``` Reviewers: kcc, pcc, eugenis, rnk Subscribers: majnemer, llvm-commits, hans, thakis, rnk Differential Revision: http://reviews.llvm.org/D20346 llvm-svn: 272053
2016-06-07 22:15:35 +02:00
FindSplitPointForStackProtector(MachineBasicBlock *BB) {
whitespace llvm-svn: 191422
2013-09-26 07:53:31 +02:00
MachineBasicBlock::iterator SplitPoint = BB->getFirstTerminator();
[stackprotector] When finding the split point to splice off the end of a parentmbb into a successmbb, include any DBG_VALUE MI. Fix for PR16954. llvm-svn: 188987
2013-08-22 07:40:50 +02:00
//
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
if (SplitPoint == BB->begin())
return SplitPoint;
MachineBasicBlock::iterator Start = BB->begin();
MachineBasicBlock::iterator Previous = SplitPoint;
--Previous;
SelectionDAG: Avoid implicit iterator conversions in SelectionDAGISel, NFC llvm-svn: 274904
2016-07-08 21:11:40 +02:00
while (MIIsInTerminatorSequence(*Previous)) {
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
SplitPoint = Previous;
if (Previous == Start)
break;
--Previous;
}
return SplitPoint;
Pool-allocation for SDNodes. The pool is allocated once for each function, and reused across SelectionDAGs. This drastically reduces the number of calls to malloc/free made during instruction selection, and improves memory locality. llvm-svn: 53211
2008-07-08 01:02:41 +02:00
}
Fold the useful features of alist and alist_node into ilist, and a new ilist_node class, and remove them. Unlike alist_node, ilist_node doesn't attempt to manage storage itself, so it avoids the associated problems, including being opaque in gdb. Adjust the Recycler class so that it doesn't depend on alist_node. Also, change it to use explicit Size and Align parameters, allowing it to work when the largest-sized node doesn't have the greatest alignment requirement. Change MachineInstr's MachineMemOperand list from a pool-backed alist to a std::list for now. llvm-svn: 54146
2008-07-28 23:51:04 +02:00
void
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
SelectionDAGISel::FinishBasicBlock() {
Change errs() to dbgs(). llvm-svn: 92597
2010-01-05 02:26:11 +01:00
DEBUG(dbgs() << "Total amount of phi nodes to update: "
Minor code simplifications. llvm-svn: 106286
2010-06-18 18:00:29 +02:00
<< FuncInfo->PHINodesToUpdate.size() << "\n";
for (unsigned i = 0, e = FuncInfo->PHINodesToUpdate.size(); i != e; ++i)
Change errs() to dbgs(). llvm-svn: 92597
2010-01-05 02:26:11 +01:00
dbgs() << "Node " << i << " : ("
Move PHINodesToUpdate out of SelectionDAGBuilder and into FunctionLoweringInfo, as it isn't SelectionDAG-specific. This isn't completely natural, as PHI node state is not per-function but rather per-basic-block, however there's currently no other convenient per-basic-block state to group it with. llvm-svn: 102109
2010-04-22 21:55:20 +02:00
<< FuncInfo->PHINodesToUpdate[i].first
<< ", " << FuncInfo->PHINodesToUpdate[i].second << ")\n");
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Instead of setting up the CFG edges at selectiondag construction time, set them up after the code has been emitted. This allows targets to select one mbb as multiple mbb's as needed. llvm-svn: 20937
2005-03-30 03:10:47 +02:00
// Next, now that we know what the last MBB the LLVM BB expanded is, update
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
// PHI nodes in successors.
Re-commit r235560: Switch lowering: extract jump tables and bit tests before building binary tree (PR22262) Third time's the charm. The previous commit was reverted as a reverse for-loop in SelectionDAGBuilder::lowerWorkItem did 'I--' on an iterator at the beginning of a vector, causing asserts when using debugging iterators. This commit fixes that. llvm-svn: 235608
2015-04-23 18:45:24 +02:00
for (unsigned i = 0, e = FuncInfo->PHINodesToUpdate.size(); i != e; ++i) {
MachineInstrBuilder PHI(*MF, FuncInfo->PHINodesToUpdate[i].first);
assert(PHI->isPHI() &&
"This is not a machine PHI node that we are updating!");
if (!FuncInfo->MBB->isSuccessor(PHI->getParent()))
continue;
PHI.addReg(FuncInfo->PHINodesToUpdate[i].second).addMBB(FuncInfo->MBB);
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
}
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
// Handle stack protector.
[stack-protection] Add support for MSVC buffer security check Summary: This patch is adding support for the MSVC buffer security check implementation The buffer security check is turned on with the '/GS' compiler switch. * https://msdn.microsoft.com/en-us/library/8dbf701c.aspx * To be added to clang here: http://reviews.llvm.org/D20347 Some overview of buffer security check feature and implementation: * https://msdn.microsoft.com/en-us/library/aa290051(VS.71).aspx * http://www.ksyash.com/2011/01/buffer-overflow-protection-3/ * http://blog.osom.info/2012/02/understanding-vs-c-compilers-buffer.html For the following example: ``` int example(int offset, int index) { char buffer[10]; memset(buffer, 0xCC, index); return buffer[index]; } ``` The MSVC compiler is adding these instructions to perform stack integrity check: ``` push ebp mov ebp,esp sub esp,50h [1] mov eax,dword ptr [__security_cookie (01068024h)] [2] xor eax,ebp [3] mov dword ptr [ebp-4],eax push ebx push esi push edi mov eax,dword ptr [index] push eax push 0CCh lea ecx,[buffer] push ecx call _memset (010610B9h) add esp,0Ch mov eax,dword ptr [index] movsx eax,byte ptr buffer[eax] pop edi pop esi pop ebx [4] mov ecx,dword ptr [ebp-4] [5] xor ecx,ebp [6] call @__security_check_cookie@4 (01061276h) mov esp,ebp pop ebp ret ``` The instrumentation above is: * [1] is loading the global security canary, * [3] is storing the local computed ([2]) canary to the guard slot, * [4] is loading the guard slot and ([5]) re-compute the global canary, * [6] is validating the resulting canary with the '__security_check_cookie' and performs error handling. Overview of the current stack-protection implementation: * lib/CodeGen/StackProtector.cpp * There is a default stack-protection implementation applied on intermediate representation. * The target can overload 'getIRStackGuard' method if it has a standard location for the stack protector cookie. * An intrinsic 'Intrinsic::stackprotector' is added to the prologue. It will be expanded by the instruction selection pass (DAG or Fast). * Basic Blocks are added to every instrumented function to receive the code for handling stack guard validation and errors handling. * Guard manipulation and comparison are added directly to the intermediate representation. * lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp * lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp * There is an implementation that adds instrumentation during instruction selection (for better handling of sibbling calls). * see long comment above 'class StackProtectorDescriptor' declaration. * The target needs to override 'getSDagStackGuard' to activate SDAG stack protection generation. (note: getIRStackGuard MUST be nullptr). * 'getSDagStackGuard' returns the appropriate stack guard (security cookie) * The code is generated by 'SelectionDAGBuilder.cpp' and 'SelectionDAGISel.cpp'. * include/llvm/Target/TargetLowering.h * Contains function to retrieve the default Guard 'Value'; should be overriden by each target to select which implementation is used and provide Guard 'Value'. * lib/Target/X86/X86ISelLowering.cpp * Contains the x86 specialisation; Guard 'Value' used by the SelectionDAG algorithm. Function-based Instrumentation: * The MSVC doesn't inline the stack guard comparison in every function. Instead, a call to '__security_check_cookie' is added to the epilogue before every return instructions. * To support function-based instrumentation, this patch is * adding a function to get the function-based check (llvm 'Value', see include/llvm/Target/TargetLowering.h), * If provided, the stack protection instrumentation won't be inlined and a call to that function will be added to the prologue. * modifying (SelectionDAGISel.cpp) do avoid producing basic blocks used for inline instrumentation, * generating the function-based instrumentation during the ISEL pass (SelectionDAGBuilder.cpp), * if FastISEL (not SelectionDAG), using the fallback which rely on the same function-based implemented over intermediate representation (StackProtector.cpp). Modifications * adding support for MSVC (lib/Target/X86/X86ISelLowering.cpp) * adding support function-based instrumentation (lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp, .h) Results * IR generated instrumentation: ``` clang-cl /GS test.cc /Od /c -mllvm -print-isel-input ``` ``` *** Final LLVM Code input to ISel *** ; Function Attrs: nounwind sspstrong define i32 @"\01?example@@YAHHH@Z"(i32 %offset, i32 %index) #0 { entry: %StackGuardSlot = alloca i8* <<<-- Allocated guard slot %0 = call i8* @llvm.stackguard() <<<-- Loading Stack Guard value call void @llvm.stackprotector(i8* %0, i8** %StackGuardSlot) <<<-- Prologue intrinsic call (store to Guard slot) %index.addr = alloca i32, align 4 %offset.addr = alloca i32, align 4 %buffer = alloca [10 x i8], align 1 store i32 %index, i32* %index.addr, align 4 store i32 %offset, i32* %offset.addr, align 4 %arraydecay = getelementptr inbounds [10 x i8], [10 x i8]* %buffer, i32 0, i32 0 %1 = load i32, i32* %index.addr, align 4 call void @llvm.memset.p0i8.i32(i8* %arraydecay, i8 -52, i32 %1, i32 1, i1 false) %2 = load i32, i32* %index.addr, align 4 %arrayidx = getelementptr inbounds [10 x i8], [10 x i8]* %buffer, i32 0, i32 %2 %3 = load i8, i8* %arrayidx, align 1 %conv = sext i8 %3 to i32 %4 = load volatile i8*, i8** %StackGuardSlot <<<-- Loading Guard slot call void @__security_check_cookie(i8* %4) <<<-- Epilogue function-based check ret i32 %conv } ``` * SelectionDAG generated instrumentation: ``` clang-cl /GS test.cc /O1 /c /FA ``` ``` "?example@@YAHHH@Z": # @"\01?example@@YAHHH@Z" # BB#0: # %entry pushl %esi subl $16, %esp movl ___security_cookie, %eax <<<-- Loading Stack Guard value movl 28(%esp), %esi movl %eax, 12(%esp) <<<-- Store to Guard slot leal 2(%esp), %eax pushl %esi pushl $204 pushl %eax calll _memset addl $12, %esp movsbl 2(%esp,%esi), %esi movl 12(%esp), %ecx <<<-- Loading Guard slot calll @__security_check_cookie@4 <<<-- Epilogue function-based check movl %esi, %eax addl $16, %esp popl %esi retl ``` Reviewers: kcc, pcc, eugenis, rnk Subscribers: majnemer, llvm-commits, hans, thakis, rnk Differential Revision: http://reviews.llvm.org/D20346 llvm-svn: 272053
2016-06-07 22:15:35 +02:00
if (SDB->SPDescriptor.shouldEmitFunctionBasedCheckStackProtector()) {
// The target provides a guard check function. There is no need to
// generate error handling code or to split current basic block.
MachineBasicBlock *ParentMBB = SDB->SPDescriptor.getParentMBB();
// Add load and check to the basicblock.
FuncInfo->MBB = ParentMBB;
FuncInfo->InsertPt =
FindSplitPointForStackProtector(ParentMBB);
SDB->visitSPDescriptorParent(SDB->SPDescriptor, ParentMBB);
CurDAG->setRoot(SDB->getRoot());
SDB->clear();
CodeGenAndEmitDAG();
// Clear the Per-BB State.
SDB->SPDescriptor.resetPerBBState();
} else if (SDB->SPDescriptor.shouldEmitStackProtector()) {
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
MachineBasicBlock *ParentMBB = SDB->SPDescriptor.getParentMBB();
MachineBasicBlock *SuccessMBB = SDB->SPDescriptor.getSuccessMBB();
// Find the split point to split the parent mbb. At the same time copy all
// physical registers used in the tail of parent mbb into virtual registers
// before the split point and back into physical registers after the split
// point. This prevents us needing to deal with Live-ins and many other
// register allocation issues caused by us splitting the parent mbb. The
// register allocator will clean up said virtual copies later on.
MachineBasicBlock::iterator SplitPoint =
[stack-protection] Add support for MSVC buffer security check Summary: This patch is adding support for the MSVC buffer security check implementation The buffer security check is turned on with the '/GS' compiler switch. * https://msdn.microsoft.com/en-us/library/8dbf701c.aspx * To be added to clang here: http://reviews.llvm.org/D20347 Some overview of buffer security check feature and implementation: * https://msdn.microsoft.com/en-us/library/aa290051(VS.71).aspx * http://www.ksyash.com/2011/01/buffer-overflow-protection-3/ * http://blog.osom.info/2012/02/understanding-vs-c-compilers-buffer.html For the following example: ``` int example(int offset, int index) { char buffer[10]; memset(buffer, 0xCC, index); return buffer[index]; } ``` The MSVC compiler is adding these instructions to perform stack integrity check: ``` push ebp mov ebp,esp sub esp,50h [1] mov eax,dword ptr [__security_cookie (01068024h)] [2] xor eax,ebp [3] mov dword ptr [ebp-4],eax push ebx push esi push edi mov eax,dword ptr [index] push eax push 0CCh lea ecx,[buffer] push ecx call _memset (010610B9h) add esp,0Ch mov eax,dword ptr [index] movsx eax,byte ptr buffer[eax] pop edi pop esi pop ebx [4] mov ecx,dword ptr [ebp-4] [5] xor ecx,ebp [6] call @__security_check_cookie@4 (01061276h) mov esp,ebp pop ebp ret ``` The instrumentation above is: * [1] is loading the global security canary, * [3] is storing the local computed ([2]) canary to the guard slot, * [4] is loading the guard slot and ([5]) re-compute the global canary, * [6] is validating the resulting canary with the '__security_check_cookie' and performs error handling. Overview of the current stack-protection implementation: * lib/CodeGen/StackProtector.cpp * There is a default stack-protection implementation applied on intermediate representation. * The target can overload 'getIRStackGuard' method if it has a standard location for the stack protector cookie. * An intrinsic 'Intrinsic::stackprotector' is added to the prologue. It will be expanded by the instruction selection pass (DAG or Fast). * Basic Blocks are added to every instrumented function to receive the code for handling stack guard validation and errors handling. * Guard manipulation and comparison are added directly to the intermediate representation. * lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp * lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp * There is an implementation that adds instrumentation during instruction selection (for better handling of sibbling calls). * see long comment above 'class StackProtectorDescriptor' declaration. * The target needs to override 'getSDagStackGuard' to activate SDAG stack protection generation. (note: getIRStackGuard MUST be nullptr). * 'getSDagStackGuard' returns the appropriate stack guard (security cookie) * The code is generated by 'SelectionDAGBuilder.cpp' and 'SelectionDAGISel.cpp'. * include/llvm/Target/TargetLowering.h * Contains function to retrieve the default Guard 'Value'; should be overriden by each target to select which implementation is used and provide Guard 'Value'. * lib/Target/X86/X86ISelLowering.cpp * Contains the x86 specialisation; Guard 'Value' used by the SelectionDAG algorithm. Function-based Instrumentation: * The MSVC doesn't inline the stack guard comparison in every function. Instead, a call to '__security_check_cookie' is added to the epilogue before every return instructions. * To support function-based instrumentation, this patch is * adding a function to get the function-based check (llvm 'Value', see include/llvm/Target/TargetLowering.h), * If provided, the stack protection instrumentation won't be inlined and a call to that function will be added to the prologue. * modifying (SelectionDAGISel.cpp) do avoid producing basic blocks used for inline instrumentation, * generating the function-based instrumentation during the ISEL pass (SelectionDAGBuilder.cpp), * if FastISEL (not SelectionDAG), using the fallback which rely on the same function-based implemented over intermediate representation (StackProtector.cpp). Modifications * adding support for MSVC (lib/Target/X86/X86ISelLowering.cpp) * adding support function-based instrumentation (lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp, .h) Results * IR generated instrumentation: ``` clang-cl /GS test.cc /Od /c -mllvm -print-isel-input ``` ``` *** Final LLVM Code input to ISel *** ; Function Attrs: nounwind sspstrong define i32 @"\01?example@@YAHHH@Z"(i32 %offset, i32 %index) #0 { entry: %StackGuardSlot = alloca i8* <<<-- Allocated guard slot %0 = call i8* @llvm.stackguard() <<<-- Loading Stack Guard value call void @llvm.stackprotector(i8* %0, i8** %StackGuardSlot) <<<-- Prologue intrinsic call (store to Guard slot) %index.addr = alloca i32, align 4 %offset.addr = alloca i32, align 4 %buffer = alloca [10 x i8], align 1 store i32 %index, i32* %index.addr, align 4 store i32 %offset, i32* %offset.addr, align 4 %arraydecay = getelementptr inbounds [10 x i8], [10 x i8]* %buffer, i32 0, i32 0 %1 = load i32, i32* %index.addr, align 4 call void @llvm.memset.p0i8.i32(i8* %arraydecay, i8 -52, i32 %1, i32 1, i1 false) %2 = load i32, i32* %index.addr, align 4 %arrayidx = getelementptr inbounds [10 x i8], [10 x i8]* %buffer, i32 0, i32 %2 %3 = load i8, i8* %arrayidx, align 1 %conv = sext i8 %3 to i32 %4 = load volatile i8*, i8** %StackGuardSlot <<<-- Loading Guard slot call void @__security_check_cookie(i8* %4) <<<-- Epilogue function-based check ret i32 %conv } ``` * SelectionDAG generated instrumentation: ``` clang-cl /GS test.cc /O1 /c /FA ``` ``` "?example@@YAHHH@Z": # @"\01?example@@YAHHH@Z" # BB#0: # %entry pushl %esi subl $16, %esp movl ___security_cookie, %eax <<<-- Loading Stack Guard value movl 28(%esp), %esi movl %eax, 12(%esp) <<<-- Store to Guard slot leal 2(%esp), %eax pushl %esi pushl $204 pushl %eax calll _memset addl $12, %esp movsbl 2(%esp,%esi), %esi movl 12(%esp), %ecx <<<-- Loading Guard slot calll @__security_check_cookie@4 <<<-- Epilogue function-based check movl %esi, %eax addl $16, %esp popl %esi retl ``` Reviewers: kcc, pcc, eugenis, rnk Subscribers: majnemer, llvm-commits, hans, thakis, rnk Differential Revision: http://reviews.llvm.org/D20346 llvm-svn: 272053
2016-06-07 22:15:35 +02:00
FindSplitPointForStackProtector(ParentMBB);
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
// Splice the terminator of ParentMBB into SuccessMBB.
SuccessMBB->splice(SuccessMBB->end(), ParentMBB,
SplitPoint,
ParentMBB->end());
// Add compare/jump on neq/jump to the parent BB.
FuncInfo->MBB = ParentMBB;
FuncInfo->InsertPt = ParentMBB->end();
SDB->visitSPDescriptorParent(SDB->SPDescriptor, ParentMBB);
CurDAG->setRoot(SDB->getRoot());
SDB->clear();
CodeGenAndEmitDAG();
// CodeGen Failure MBB if we have not codegened it yet.
MachineBasicBlock *FailureMBB = SDB->SPDescriptor.getFailureMBB();
Don't check if a list is empty with ilist::size. ilist::size() is O(n) while ilist::empty() is O(1) llvm-svn: 258636
2016-01-23 21:58:09 +01:00
if (FailureMBB->empty()) {
Teach selectiondag how to handle the stackprotectorcheck intrinsic. Previously, generation of stack protectors was done exclusively in the pre-SelectionDAG Codegen LLVM IR Pass "Stack Protector". This necessitated splitting basic blocks at the IR level to create the success/failure basic blocks in the tail of the basic block in question. As a result of this, calls that would have qualified for the sibling call optimization were no longer eligible for optimization since said calls were no longer right in the "tail position" (i.e. the immediate predecessor of a ReturnInst instruction). Then it was noticed that since the sibling call optimization causes the callee to reuse the caller's stack, if we could delay the generation of the stack protector check until later in CodeGen after the sibling call decision was made, we get both the tail call optimization and the stack protector check! A few goals in solving this problem were: 1. Preserve the architecture independence of stack protector generation. 2. Preserve the normal IR level stack protector check for platforms like OpenBSD for which we support platform specific stack protector generation. The main problem that guided the present solution is that one can not solve this problem in an architecture independent manner at the IR level only. This is because: 1. The decision on whether or not to perform a sibling call on certain platforms (for instance i386) requires lower level information related to available registers that can not be known at the IR level. 2. Even if the previous point were not true, the decision on whether to perform a tail call is done in LowerCallTo in SelectionDAG which occurs after the Stack Protector Pass. As a result, one would need to put the relevant callinst into the stack protector check success basic block (where the return inst is placed) and then move it back later at SelectionDAG/MI time before the stack protector check if the tail call optimization failed. The MI level option was nixed immediately since it would require platform specific pattern matching. The SelectionDAG level option was nixed because SelectionDAG only processes one IR level basic block at a time implying one could not create a DAG Combine to move the callinst. To get around this problem a few things were realized: 1. While one can not handle multiple IR level basic blocks at the SelectionDAG Level, one can generate multiple machine basic blocks for one IR level basic block. This is how we handle bit tests and switches. 2. At the MI level, tail calls are represented via a special return MIInst called "tcreturn". Thus if we know the basic block in which we wish to insert the stack protector check, we get the correct behavior by always inserting the stack protector check right before the return statement. This is a "magical transformation" since no matter where the stack protector check intrinsic is, we always insert the stack protector check code at the end of the BB. Given the aforementioned constraints, the following solution was devised: 1. On platforms that do not support SelectionDAG stack protector check generation, allow for the normal IR level stack protector check generation to continue. 2. On platforms that do support SelectionDAG stack protector check generation: a. Use the IR level stack protector pass to decide if a stack protector is required/which BB we insert the stack protector check in by reusing the logic already therein. If we wish to generate a stack protector check in a basic block, we place a special IR intrinsic called llvm.stackprotectorcheck right before the BB's returninst or if there is a callinst that could potentially be sibling call optimized, before the call inst. b. Then when a BB with said intrinsic is processed, we codegen the BB normally via SelectBasicBlock. In said process, when we visit the stack protector check, we do not actually emit anything into the BB. Instead, we just initialize the stack protector descriptor class (which involves stashing information/creating the success mbbb and the failure mbb if we have not created one for this function yet) and export the guard variable that we are going to compare. c. After we finish selecting the basic block, in FinishBasicBlock if the StackProtectorDescriptor attached to the SelectionDAGBuilder is initialized, we first find a splice point in the parent basic block before the terminator and then splice the terminator of said basic block into the success basic block. Then we code-gen a new tail for the parent basic block consisting of the two loads, the comparison, and finally two branches to the success/failure basic blocks. We conclude by code-gening the failure basic block if we have not code-gened it already (all stack protector checks we generate in the same function, use the same failure basic block). llvm-svn: 188755
2013-08-20 09:00:16 +02:00
FuncInfo->MBB = FailureMBB;
FuncInfo->InsertPt = FailureMBB->end();
SDB->visitSPDescriptorFailure(SDB->SPDescriptor);
CurDAG->setRoot(SDB->getRoot());
SDB->clear();
CodeGenAndEmitDAG();
}
// Clear the Per-BB State.
SDB->SPDescriptor.resetPerBBState();
}
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
// Lower each BitTestBlock.
for (auto &BTB : SDB->BitTestCases) {
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
// Lower header first, if it wasn't already lowered
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
if (!BTB.Emitted) {
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
// Set the current basic block to the mbb we wish to insert the code into
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
FuncInfo->MBB = BTB.Parent;
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FuncInfo->InsertPt = FuncInfo->MBB->end();
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
// Emit the code
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
SDB->visitBitTestHeader(BTB, FuncInfo->MBB);
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
CurDAG->setRoot(SDB->getRoot());
SDB->clear();
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
CodeGenAndEmitDAG();
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
}
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
BranchProbability UnhandledProb = BTB.Prob;
for (unsigned j = 0, ej = BTB.Cases.size(); j != ej; ++j) {
UnhandledProb -= BTB.Cases[j].ExtraProb;
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
// Set the current basic block to the mbb we wish to insert the code into
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
FuncInfo->MBB = BTB.Cases[j].ThisBB;
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FuncInfo->InsertPt = FuncInfo->MBB->end();
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
// Emit the code
Remove the final bit test during lowering switch statement if all cases in bit test cover a contiguous range. When lowering switch statement, if bit tests are used then LLVM will always generates a jump to the default statement in the last bit test. However, this is not necessary when all cases in bit tests cover a contiguous range. This is because when generating the bit tests header MBB, there is a range check that guarantees cases in bit tests won't go outside of [low, high], where low and high are minimum and maximum case values in the bit tests. This patch checks if this is the case and then doesn't emit jump to default statement and hence saves a bit test and a branch. Differential Revision: http://reviews.llvm.org/D12249 llvm-svn: 245976
2015-08-25 23:34:38 +02:00
// If all cases cover a contiguous range, it is not necessary to jump to
// the default block after the last bit test fails. This is because the
// range check during bit test header creation has guaranteed that every
Switch lowering: don't add incoming PHI values from skipped bit test MBB's (PR27135) After r245976, LLVM will skip the last bit test case if knows it will always be true. However, we would still erroneously update PHI nodes with incoming values from the MBB that would perform the final bit test, causing -verify-machineinstrs to fail. llvm-svn: 266479
2016-04-15 23:45:30 +02:00
// case here doesn't go outside the range. In this case, there is no need
// to perform the last bit test, as it will always be true. Instead, make
// the second-to-last bit-test fall through to the target of the last bit
// test, and delete the last bit test.
Remove the final bit test during lowering switch statement if all cases in bit test cover a contiguous range. When lowering switch statement, if bit tests are used then LLVM will always generates a jump to the default statement in the last bit test. However, this is not necessary when all cases in bit tests cover a contiguous range. This is because when generating the bit tests header MBB, there is a range check that guarantees cases in bit tests won't go outside of [low, high], where low and high are minimum and maximum case values in the bit tests. This patch checks if this is the case and then doesn't emit jump to default statement and hence saves a bit test and a branch. Differential Revision: http://reviews.llvm.org/D12249 llvm-svn: 245976
2015-08-25 23:34:38 +02:00
MachineBasicBlock *NextMBB;
Switch lowering: don't add incoming PHI values from skipped bit test MBB's (PR27135) After r245976, LLVM will skip the last bit test case if knows it will always be true. However, we would still erroneously update PHI nodes with incoming values from the MBB that would perform the final bit test, causing -verify-machineinstrs to fail. llvm-svn: 266479
2016-04-15 23:45:30 +02:00
if (BTB.ContiguousRange && j + 2 == ej) {
// Second-to-last bit-test with contiguous range: fall through to the
// target of the final bit test.
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
NextMBB = BTB.Cases[j + 1].TargetBB;
Switch lowering: don't add incoming PHI values from skipped bit test MBB's (PR27135) After r245976, LLVM will skip the last bit test case if knows it will always be true. However, we would still erroneously update PHI nodes with incoming values from the MBB that would perform the final bit test, causing -verify-machineinstrs to fail. llvm-svn: 266479
2016-04-15 23:45:30 +02:00
} else if (j + 1 == ej) {
// For the last bit test, fall through to Default.
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
NextMBB = BTB.Default;
Switch lowering: don't add incoming PHI values from skipped bit test MBB's (PR27135) After r245976, LLVM will skip the last bit test case if knows it will always be true. However, we would still erroneously update PHI nodes with incoming values from the MBB that would perform the final bit test, causing -verify-machineinstrs to fail. llvm-svn: 266479
2016-04-15 23:45:30 +02:00
} else {
// Otherwise, fall through to the next bit test.
NextMBB = BTB.Cases[j + 1].ThisBB;
}
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
SDB->visitBitTestCase(BTB, NextMBB, UnhandledProb, BTB.Reg, BTB.Cases[j],
Remove the final bit test during lowering switch statement if all cases in bit test cover a contiguous range. When lowering switch statement, if bit tests are used then LLVM will always generates a jump to the default statement in the last bit test. However, this is not necessary when all cases in bit tests cover a contiguous range. This is because when generating the bit tests header MBB, there is a range check that guarantees cases in bit tests won't go outside of [low, high], where low and high are minimum and maximum case values in the bit tests. This patch checks if this is the case and then doesn't emit jump to default statement and hence saves a bit test and a branch. Differential Revision: http://reviews.llvm.org/D12249 llvm-svn: 245976
2015-08-25 23:34:38 +02:00
FuncInfo->MBB);
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
CurDAG->setRoot(SDB->getRoot());
SDB->clear();
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
CodeGenAndEmitDAG();
Remove the final bit test during lowering switch statement if all cases in bit test cover a contiguous range. When lowering switch statement, if bit tests are used then LLVM will always generates a jump to the default statement in the last bit test. However, this is not necessary when all cases in bit tests cover a contiguous range. This is because when generating the bit tests header MBB, there is a range check that guarantees cases in bit tests won't go outside of [low, high], where low and high are minimum and maximum case values in the bit tests. This patch checks if this is the case and then doesn't emit jump to default statement and hence saves a bit test and a branch. Differential Revision: http://reviews.llvm.org/D12249 llvm-svn: 245976
2015-08-25 23:34:38 +02:00
Switch lowering: don't add incoming PHI values from skipped bit test MBB's (PR27135) After r245976, LLVM will skip the last bit test case if knows it will always be true. However, we would still erroneously update PHI nodes with incoming values from the MBB that would perform the final bit test, causing -verify-machineinstrs to fail. llvm-svn: 266479
2016-04-15 23:45:30 +02:00
if (BTB.ContiguousRange && j + 2 == ej) {
// Since we're not going to use the final bit test, remove it.
BTB.Cases.pop_back();
Remove the final bit test during lowering switch statement if all cases in bit test cover a contiguous range. When lowering switch statement, if bit tests are used then LLVM will always generates a jump to the default statement in the last bit test. However, this is not necessary when all cases in bit tests cover a contiguous range. This is because when generating the bit tests header MBB, there is a range check that guarantees cases in bit tests won't go outside of [low, high], where low and high are minimum and maximum case values in the bit tests. This patch checks if this is the case and then doesn't emit jump to default statement and hence saves a bit test and a branch. Differential Revision: http://reviews.llvm.org/D12249 llvm-svn: 245976
2015-08-25 23:34:38 +02:00
break;
Switch lowering: don't add incoming PHI values from skipped bit test MBB's (PR27135) After r245976, LLVM will skip the last bit test case if knows it will always be true. However, we would still erroneously update PHI nodes with incoming values from the MBB that would perform the final bit test, causing -verify-machineinstrs to fail. llvm-svn: 266479
2016-04-15 23:45:30 +02:00
}
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
}
// Update PHI Nodes
Move PHINodesToUpdate out of SelectionDAGBuilder and into FunctionLoweringInfo, as it isn't SelectionDAG-specific. This isn't completely natural, as PHI node state is not per-function but rather per-basic-block, however there's currently no other convenient per-basic-block state to group it with. llvm-svn: 102109
2010-04-22 21:55:20 +02:00
for (unsigned pi = 0, pe = FuncInfo->PHINodesToUpdate.size();
pi != pe; ++pi) {
Use MachineInstrBuilder for PHI nodes in SelectionDAGISel. llvm-svn: 170716
2012-12-20 19:46:29 +01:00
MachineInstrBuilder PHI(*MF, FuncInfo->PHINodesToUpdate[pi].first);
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
MachineBasicBlock *PHIBB = PHI->getParent();
move target-independent opcodes out of TargetInstrInfo into TargetOpcodes.h. #include the new TargetOpcodes.h into MachineInstr. Add new inline accessors (like isPHI()) to MachineInstr, and start using them throughout the codebase. llvm-svn: 95687
2010-02-09 20:54:29 +01:00
assert(PHI->isPHI() &&
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
"This is not a machine PHI node that we are updating!");
// This is "default" BB. We have two jumps to it. From "header" BB and
Switch lowering: don't add incoming PHI values from skipped bit test MBB's (PR27135) After r245976, LLVM will skip the last bit test case if knows it will always be true. However, we would still erroneously update PHI nodes with incoming values from the MBB that would perform the final bit test, causing -verify-machineinstrs to fail. llvm-svn: 266479
2016-04-15 23:45:30 +02:00
// from last "case" BB, unless the latter was skipped.
if (PHIBB == BTB.Default) {
PHI.addReg(FuncInfo->PHINodesToUpdate[pi].second).addMBB(BTB.Parent);
if (!BTB.ContiguousRange) {
PHI.addReg(FuncInfo->PHINodesToUpdate[pi].second)
.addMBB(BTB.Cases.back().ThisBB);
}
}
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
// One of "cases" BB.
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
for (unsigned j = 0, ej = BTB.Cases.size();
Reorganize the lifetimes of the major objects SelectionDAGISel works with. SelectionDAG, FunctionLoweringInfo, and SelectionDAGLowering objects now get created once per SelectionDAGISel instance, and can be reused across blocks and across functions. Previously, they were created and destroyed each time they were needed. This reorganization simplifies the handling of PHI nodes, and also SwitchCases, JumpTables, and BitTestBlocks. This simplification has the side effect of fixing a bug in FastISel where successor PHI nodes weren't being updated correctly. This is also a step towards making the transition from FastISel into and out of SelectionDAG faster, and also making plain SelectionDAG faster on code with lots of little blocks. llvm-svn: 55450
2008-08-28 01:52:12 +02:00
j != ej; ++j) {
SelectionDAGISel: rangeify a loop llvm-svn: 266478
2016-04-15 23:45:09 +02:00
MachineBasicBlock* cBB = BTB.Cases[j].ThisBB;
Use MachineInstrBuilder for PHI nodes in SelectionDAGISel. llvm-svn: 170716
2012-12-20 19:46:29 +01:00
if (cBB->isSuccessor(PHIBB))
PHI.addReg(FuncInfo->PHINodesToUpdate[pi].second).addMBB(cBB);
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
}
}
}
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
SDB->BitTestCases.clear();
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
Fix the updating of the machine CFG when a PHI node was in a successor of the jump table's range check block. This re-enables 100% dense jump tables by default on PPC & x86 llvm-svn: 27952
2006-04-23 08:26:20 +02:00
// If the JumpTable record is filled in, then we need to emit a jump table.
// Updating the PHI nodes is tricky in this case, since we need to determine
// whether the PHI is a successor of the range check MBB or the jump table MBB
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
for (unsigned i = 0, e = SDB->JTCases.size(); i != e; ++i) {
First step of switch lowering refactoring: perform worklist-driven strategy, emit JT's where possible. llvm-svn: 35338
2007-03-25 17:07:15 +02:00
// Lower header first, if it wasn't already lowered
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
if (!SDB->JTCases[i].first.Emitted) {
First step of switch lowering refactoring: perform worklist-driven strategy, emit JT's where possible. llvm-svn: 35338
2007-03-25 17:07:15 +02:00
// Set the current basic block to the mbb we wish to insert the code into
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FuncInfo->MBB = SDB->JTCases[i].first.HeaderBB;
FuncInfo->InsertPt = FuncInfo->MBB->end();
First step of switch lowering refactoring: perform worklist-driven strategy, emit JT's where possible. llvm-svn: 35338
2007-03-25 17:07:15 +02:00
// Emit the code
Eliminate the CurMBB member from SelectionDAGBuilder. For places that need it, just pass around the parent block of the current instruction explicitly. llvm-svn: 101822
2010-04-20 00:41:47 +02:00
SDB->visitJumpTableHeader(SDB->JTCases[i].second, SDB->JTCases[i].first,
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FuncInfo->MBB);
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
CurDAG->setRoot(SDB->getRoot());
SDB->clear();
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
CodeGenAndEmitDAG();
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
}
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
JumpTable support! What this represents is working asm and jit support for x86 and ppc for 100% dense switch statements when relocations are non-PIC. This support will be extended and enhanced in the coming days to support PIC, and less dense forms of jump tables. llvm-svn: 27947
2006-04-22 20:53:45 +02:00
// Set the current basic block to the mbb we wish to insert the code into
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FuncInfo->MBB = SDB->JTCases[i].second.MBB;
FuncInfo->InsertPt = FuncInfo->MBB->end();
JumpTable support! What this represents is working asm and jit support for x86 and ppc for 100% dense switch statements when relocations are non-PIC. This support will be extended and enhanced in the coming days to support PIC, and less dense forms of jump tables. llvm-svn: 27947
2006-04-22 20:53:45 +02:00
// Emit the code
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
SDB->visitJumpTable(SDB->JTCases[i].second);
CurDAG->setRoot(SDB->getRoot());
SDB->clear();
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
CodeGenAndEmitDAG();
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
JumpTable support! What this represents is working asm and jit support for x86 and ppc for 100% dense switch statements when relocations are non-PIC. This support will be extended and enhanced in the coming days to support PIC, and less dense forms of jump tables. llvm-svn: 27947
2006-04-22 20:53:45 +02:00
// Update PHI Nodes
Move PHINodesToUpdate out of SelectionDAGBuilder and into FunctionLoweringInfo, as it isn't SelectionDAG-specific. This isn't completely natural, as PHI node state is not per-function but rather per-basic-block, however there's currently no other convenient per-basic-block state to group it with. llvm-svn: 102109
2010-04-22 21:55:20 +02:00
for (unsigned pi = 0, pe = FuncInfo->PHINodesToUpdate.size();
pi != pe; ++pi) {
Use MachineInstrBuilder for PHI nodes in SelectionDAGISel. llvm-svn: 170716
2012-12-20 19:46:29 +01:00
MachineInstrBuilder PHI(*MF, FuncInfo->PHINodesToUpdate[pi].first);
JumpTable support! What this represents is working asm and jit support for x86 and ppc for 100% dense switch statements when relocations are non-PIC. This support will be extended and enhanced in the coming days to support PIC, and less dense forms of jump tables. llvm-svn: 27947
2006-04-22 20:53:45 +02:00
MachineBasicBlock *PHIBB = PHI->getParent();
move target-independent opcodes out of TargetInstrInfo into TargetOpcodes.h. #include the new TargetOpcodes.h into MachineInstr. Add new inline accessors (like isPHI()) to MachineInstr, and start using them throughout the codebase. llvm-svn: 95687
2010-02-09 20:54:29 +01:00
assert(PHI->isPHI() &&
JumpTable support! What this represents is working asm and jit support for x86 and ppc for 100% dense switch statements when relocations are non-PIC. This support will be extended and enhanced in the coming days to support PIC, and less dense forms of jump tables. llvm-svn: 27947
2006-04-22 20:53:45 +02:00
"This is not a machine PHI node that we are updating!");
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
// "default" BB. We can go there only from header BB.
Use MachineInstrBuilder for PHI nodes in SelectionDAGISel. llvm-svn: 170716
2012-12-20 19:46:29 +01:00
if (PHIBB == SDB->JTCases[i].second.Default)
PHI.addReg(FuncInfo->PHINodesToUpdate[pi].second)
.addMBB(SDB->JTCases[i].first.HeaderBB);
Next stage into switch lowering refactoring 1. Fix some bugs in the jump table lowering threshold 2. Implement much better metric for optimal pivot selection 3. Tune thresholds for different lowering methods 4. Implement shift-and trick for lowering small (<machine word length) cases with few destinations. Good testcase will follow. llvm-svn: 35816
2007-04-09 14:31:58 +02:00
// JT BB. Just iterate over successors here
Use MachineInstrBuilder for PHI nodes in SelectionDAGISel. llvm-svn: 170716
2012-12-20 19:46:29 +01:00
if (FuncInfo->MBB->isSuccessor(PHIBB))
PHI.addReg(FuncInfo->PHINodesToUpdate[pi].second).addMBB(FuncInfo->MBB);
JumpTable support! What this represents is working asm and jit support for x86 and ppc for 100% dense switch statements when relocations are non-PIC. This support will be extended and enhanced in the coming days to support PIC, and less dense forms of jump tables. llvm-svn: 27947
2006-04-22 20:53:45 +02:00
}
}
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
SDB->JTCases.clear();
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
SelectionDAGISel can now natively handle Switch instructions, in the same manner that the LowerSwitch LLVM to LLVM pass does: emitting a binary search tree of basic blocks. The new approach has several advantages: it is faster, it generates significantly smaller code in many cases, and it paves the way for implementing dense switch tables as a jump table by handling switches directly in the instruction selector. This functionality is currently only enabled on x86, but should be safe for every target. In anticipation of making it the default, the cfg is now properly updated in the x86, ppc, and sparc select lowering code. llvm-svn: 27156
2006-03-27 03:32:24 +02:00
// If we generated any switch lowering information, build and codegen any
// additional DAGs necessary.
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
for (unsigned i = 0, e = SDB->SwitchCases.size(); i != e; ++i) {
SelectionDAGISel can now natively handle Switch instructions, in the same manner that the LowerSwitch LLVM to LLVM pass does: emitting a binary search tree of basic blocks. The new approach has several advantages: it is faster, it generates significantly smaller code in many cases, and it paves the way for implementing dense switch tables as a jump table by handling switches directly in the instruction selector. This functionality is currently only enabled on x86, but should be safe for every target. In anticipation of making it the default, the cfg is now properly updated in the x86, ppc, and sparc select lowering code. llvm-svn: 27156
2006-03-27 03:32:24 +02:00
// Set the current basic block to the mbb we wish to insert the code into
Delete an assignment to ThisBB which isn't needed, and tidy up some comments. llvm-svn: 123479
2011-01-14 23:26:16 +01:00
FuncInfo->MBB = SDB->SwitchCases[i].ThisBB;
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FuncInfo->InsertPt = FuncInfo->MBB->end();
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Reorgnaize more switch code lowering to clean up some tricky code, and to eliminate the need for the SelectionDAGBuilder state to be live during CodeGenAndEmitDAG calls. Call SDB->clear() before CodeGenAndEmitDAG calls instead of before it, and move the CurDAG->clear() out of SelectionDAGBuilder, which doesn't own the DAG, and into CodeGenAndEmitDAG. llvm-svn: 102814
2010-05-01 02:25:44 +02:00
// Determine the unique successors.
SmallVector<MachineBasicBlock *, 2> Succs;
Succs.push_back(SDB->SwitchCases[i].TrueBB);
if (SDB->SwitchCases[i].TrueBB != SDB->SwitchCases[i].FalseBB)
Succs.push_back(SDB->SwitchCases[i].FalseBB);
Delete an assignment to ThisBB which isn't needed, and tidy up some comments. llvm-svn: 123479
2011-01-14 23:26:16 +01:00
// Emit the code. Note that this could result in FuncInfo->MBB being split.
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
SDB->visitSwitchCase(SDB->SwitchCases[i], FuncInfo->MBB);
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
CurDAG->setRoot(SDB->getRoot());
Reorgnaize more switch code lowering to clean up some tricky code, and to eliminate the need for the SelectionDAGBuilder state to be live during CodeGenAndEmitDAG calls. Call SDB->clear() before CodeGenAndEmitDAG calls instead of before it, and move the CurDAG->clear() out of SelectionDAGBuilder, which doesn't own the DAG, and into CodeGenAndEmitDAG. llvm-svn: 102814
2010-05-01 02:25:44 +02:00
SDB->clear();
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
CodeGenAndEmitDAG();
Delete an assignment to ThisBB which isn't needed, and tidy up some comments. llvm-svn: 123479
2011-01-14 23:26:16 +01:00
// Remember the last block, now that any splitting is done, for use in
// populating PHI nodes in successors.
MachineBasicBlock *ThisBB = FuncInfo->MBB;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Fix CodeGen/Generic/2006-09-06-SwitchLowering.ll, a bug where SDIsel inserted too many phi operands when lowering a switch to branches in some cases. llvm-svn: 30142
2006-09-07 03:59:34 +02:00
// Handle any PHI nodes in successors of this chunk, as if we were coming
// from the original BB before switch expansion. Note that PHI nodes can
// occur multiple times in PHINodesToUpdate. We have to be very careful to
// handle them the right number of times.
Reorgnaize more switch code lowering to clean up some tricky code, and to eliminate the need for the SelectionDAGBuilder state to be live during CodeGenAndEmitDAG calls. Call SDB->clear() before CodeGenAndEmitDAG calls instead of before it, and move the CurDAG->clear() out of SelectionDAGBuilder, which doesn't own the DAG, and into CodeGenAndEmitDAG. llvm-svn: 102814
2010-05-01 02:25:44 +02:00
for (unsigned i = 0, e = Succs.size(); i != e; ++i) {
Reapply bottom-up fast-isel, with several fixes for x86-32: - Check getBytesToPopOnReturn(). - Eschew ST0 and ST1 for return values. - Fix the PIC base register initialization so that it doesn't ever fail to end up the top of the entry block. llvm-svn: 108039
2010-07-10 11:00:22 +02:00
FuncInfo->MBB = Succs[i];
FuncInfo->InsertPt = FuncInfo->MBB->end();
// FuncInfo->MBB may have been removed from the CFG if a branch was
// constant folded.
if (ThisBB->isSuccessor(FuncInfo->MBB)) {
Use MachineInstrBuilder for PHI nodes in SelectionDAGISel. llvm-svn: 170716
2012-12-20 19:46:29 +01:00
for (MachineBasicBlock::iterator
MBBI = FuncInfo->MBB->begin(), MBBE = FuncInfo->MBB->end();
MBBI != MBBE && MBBI->isPHI(); ++MBBI) {
MachineInstrBuilder PHI(*MF, MBBI);
Avoid adding PHI arguments for a predecessor that has gone away when a BRCOND was constant folded. This fixes PR5980. llvm-svn: 93184
2010-01-11 22:02:33 +01:00
// This value for this PHI node is recorded in PHINodesToUpdate.
for (unsigned pn = 0; ; ++pn) {
Move PHINodesToUpdate out of SelectionDAGBuilder and into FunctionLoweringInfo, as it isn't SelectionDAG-specific. This isn't completely natural, as PHI node state is not per-function but rather per-basic-block, however there's currently no other convenient per-basic-block state to group it with. llvm-svn: 102109
2010-04-22 21:55:20 +02:00
assert(pn != FuncInfo->PHINodesToUpdate.size() &&
Avoid adding PHI arguments for a predecessor that has gone away when a BRCOND was constant folded. This fixes PR5980. llvm-svn: 93184
2010-01-11 22:02:33 +01:00
"Didn't find PHI entry!");
Use MachineInstrBuilder for PHI nodes in SelectionDAGISel. llvm-svn: 170716
2012-12-20 19:46:29 +01:00
if (FuncInfo->PHINodesToUpdate[pn].first == PHI) {
PHI.addReg(FuncInfo->PHINodesToUpdate[pn].second).addMBB(ThisBB);
Avoid adding PHI arguments for a predecessor that has gone away when a BRCOND was constant folded. This fixes PR5980. llvm-svn: 93184
2010-01-11 22:02:33 +01:00
break;
}
Revert r82214. It broke 403.gcc on x86_64 / Darwin. llvm-svn: 82215
2009-09-18 10:26:06 +02:00
}
Fix CodeGen/Generic/2006-09-06-SwitchLowering.ll, a bug where SDIsel inserted too many phi operands when lowering a switch to branches in some cases. llvm-svn: 30142
2006-09-07 03:59:34 +02:00
}
SelectionDAGISel can now natively handle Switch instructions, in the same manner that the LowerSwitch LLVM to LLVM pass does: emitting a binary search tree of basic blocks. The new approach has several advantages: it is faster, it generates significantly smaller code in many cases, and it paves the way for implementing dense switch tables as a jump table by handling switches directly in the instruction selector. This functionality is currently only enabled on x86, but should be safe for every target. In anticipation of making it the default, the cfg is now properly updated in the x86, ppc, and sparc select lowering code. llvm-svn: 27156
2006-03-27 03:32:24 +02:00
}
}
Instead of setting up the CFG edges at selectiondag construction time, set them up after the code has been emitted. This allows targets to select one mbb as multiple mbb's as needed. llvm-svn: 20937
2005-03-30 03:10:47 +02:00
}
Rename SelectionDAGLowering to SelectionDAGBuilder, and rename SelectionDAGBuild.cpp to SelectionDAGBuilder.cpp. llvm-svn: 89681
2009-11-23 19:04:58 +01:00
SDB->SwitchCases.clear();
Initial implementation of the SelectionDAGISel class. This contains most of the code for lowering from LLVM code to a SelectionDAG. llvm-svn: 19331
2005-01-07 08:47:53 +01:00
}
Do some code refactoring on Jim's scheduler in preparation of the new list scheduler. llvm-svn: 25493
2006-01-21 03:32:06 +01:00
Rename SelectionDAGISel::Schedule to SelectionDAGISel::CreateScheduler, and make it just create the scheduler. Leave running the scheduler to the higher-level code. This makes the higher-level code a little more explicit and easier to follow, and will help enable some future refactoring. llvm-svn: 63944
2009-02-06 19:26:51 +01:00
/// Create the scheduler. If a specific scheduler was specified
/// via the SchedulerRegistry, use it, otherwise select the
/// one preferred by the target.
Reapply 53476 and 53480, with a fix so that it properly updates the BB member to the current basic block after emitting instructions. llvm-svn: 53567
2008-07-14 20:19:29 +02:00
///
When scheduling a block in parts, keep track of the overall instruction index across each part. Instruction indices are used to make live range queries, and live ranges can extend beyond scheduling region boundaries. Refactor the ScheduleDAGSDNodes class some more so that it doesn't have to worry about this additional information. llvm-svn: 64288
2009-02-11 05:27:20 +01:00
ScheduleDAGSDNodes *SelectionDAGISel::CreateScheduler() {
Move the Target way of overriding DAG Scheduler to a target hook Summary: The previous way of overriding it was relying on calling "setDefault" on the global registry, which implies global mutable state. Reviewers: echristo, atrick Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11538 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 243388
2015-07-28 08:18:04 +02:00
return ISHeuristic(this, OptLevel);
Do some code refactoring on Jim's scheduler in preparation of the new list scheduler. llvm-svn: 25493
2006-01-21 03:32:06 +01:00
}
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
//===----------------------------------------------------------------------===//
// Helper functions used by the generated instruction selector.
//===----------------------------------------------------------------------===//
// Calls to these methods are generated by tblgen.
/// CheckAndMask - The isel is trying to match something like (and X, 255). If
/// the dag combiner simplified the 255, we still want to match. RHS is the
/// actual value in the DAG on the RHS of an AND, and DesiredMaskS is the value
/// specified in the .td file (e.g. 255).
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
bool SelectionDAGISel::CheckAndMask(SDValue LHS, ConstantSDNode *RHS,
Add const to CanBeFoldedBy, CheckAndMask, and CheckOrMask. llvm-svn: 40480
2007-07-25 01:00:27 +02:00
int64_t DesiredMaskS) const {
Convert MaskedValueIsZero and all its users to use APInt. Also add a SignBitIsZero function to simplify a common use case. llvm-svn: 47561
2008-02-25 22:11:39 +01:00
const APInt &ActualMask = RHS->getAPIntValue();
const APInt &DesiredMask = APInt(LHS.getValueSizeInBits(), DesiredMaskS);
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// If the actual mask exactly matches, success!
if (ActualMask == DesiredMask)
return true;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// If the actual AND mask is allowing unallowed bits, this doesn't match.
Convert MaskedValueIsZero and all its users to use APInt. Also add a SignBitIsZero function to simplify a common use case. llvm-svn: 47561
2008-02-25 22:11:39 +01:00
if (ActualMask.intersects(~DesiredMask))
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
return false;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// Otherwise, the DAG Combiner may have proven that the value coming in is
// either already zero or is not demanded. Check for known zero input bits.
Convert MaskedValueIsZero and all its users to use APInt. Also add a SignBitIsZero function to simplify a common use case. llvm-svn: 47561
2008-02-25 22:11:39 +01:00
APInt NeededMask = DesiredMask & ~ActualMask;
Move ComputeMaskedBits, MaskedValueIsZero, and ComputeNumSignBits from TargetLowering to SelectionDAG so that they have more convenient access to the current DAG, in preparation for the ValueType routines being changed from standalone functions to members of SelectionDAG for the pre-legalize vector type changes. llvm-svn: 37704
2007-06-22 16:59:07 +02:00
if (CurDAG->MaskedValueIsZero(LHS, NeededMask))
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
return true;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// TODO: check to see if missing bits are just not demanded.
// Otherwise, this pattern doesn't match.
return false;
}
/// CheckOrMask - The isel is trying to match something like (or X, 255). If
/// the dag combiner simplified the 255, we still want to match. RHS is the
/// actual value in the DAG on the RHS of an OR, and DesiredMaskS is the value
/// specified in the .td file (e.g. 255).
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
bool SelectionDAGISel::CheckOrMask(SDValue LHS, ConstantSDNode *RHS,
Convert MaskedValueIsZero and all its users to use APInt. Also add a SignBitIsZero function to simplify a common use case. llvm-svn: 47561
2008-02-25 22:11:39 +01:00
int64_t DesiredMaskS) const {
const APInt &ActualMask = RHS->getAPIntValue();
const APInt &DesiredMask = APInt(LHS.getValueSizeInBits(), DesiredMaskS);
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// If the actual mask exactly matches, success!
if (ActualMask == DesiredMask)
return true;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// If the actual AND mask is allowing unallowed bits, this doesn't match.
Convert MaskedValueIsZero and all its users to use APInt. Also add a SignBitIsZero function to simplify a common use case. llvm-svn: 47561
2008-02-25 22:11:39 +01:00
if (ActualMask.intersects(~DesiredMask))
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
return false;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// Otherwise, the DAG Combiner may have proven that the value coming in is
// either already zero or is not demanded. Check for known zero input bits.
Convert MaskedValueIsZero and all its users to use APInt. Also add a SignBitIsZero function to simplify a common use case. llvm-svn: 47561
2008-02-25 22:11:39 +01:00
APInt NeededMask = DesiredMask & ~ActualMask;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Convert MaskedValueIsZero and all its users to use APInt. Also add a SignBitIsZero function to simplify a common use case. llvm-svn: 47561
2008-02-25 22:11:39 +01:00
APInt KnownZero, KnownOne;
Rename ComputeMaskedBits to computeKnownBits. "Masked" has been inappropriate since it lost its Mask parameter in r154011. llvm-svn: 208811
2014-05-14 23:14:37 +02:00
CurDAG->computeKnownBits(LHS, KnownZero, KnownOne);
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// If all the missing bits in the or are already known to be set, match!
if ((NeededMask & KnownOne) == NeededMask)
return true;
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// TODO: check to see if missing bits are just not demanded.
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
add two helper methods. llvm-svn: 30869
2006-10-11 05:58:02 +02:00
// Otherwise, this pattern doesn't match.
return false;
}
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
/// SelectInlineAsmMemoryOperands - Calls to this are automatically generated
/// by tblgen. Others should not call it.
Pass DebugLoc and SDLoc by const ref. This used to be free, copying and moving DebugLocs became expensive after the metadata rewrite. Passing by reference eliminates a ton of track/untrack operations. No functionality change intended. llvm-svn: 272512
2016-06-12 17:39:02 +02:00
void SelectionDAGISel::SelectInlineAsmMemoryOperands(std::vector<SDValue> &Ops,
const SDLoc &DL) {
Rename SDOperand to SDValue. llvm-svn: 54128
2008-07-27 23:46:04 +02:00
std::vector<SDValue> InOps;
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
std::swap(InOps, Ops);
Three changes: 1. Introduce some enums and accessors in the InlineAsm class that eliminate a ton of magic numbers when handling inline asm SDNode. 2. Add a new MDNodeSDNode selection dag node type that holds a MDNode (shocking!) 3. Add a new argument to ISD::INLINEASM nodes that hold !srcloc metadata, propagating it to the instruction emitter, which drops it. No functionality change. llvm-svn: 100605
2010-04-07 07:20:54 +02:00
Ops.push_back(InOps[InlineAsm::Op_InputChain]); // 0
Ops.push_back(InOps[InlineAsm::Op_AsmString]); // 1
Ops.push_back(InOps[InlineAsm::Op_MDNode]); // 2, !srcloc
Do not model all INLINEASM instructions as having unmodelled side effects. Instead encode llvm IR level property "HasSideEffects" in an operand (shared with IsAlignStack). Added MachineInstrs::hasUnmodeledSideEffects() to check the operand when the instruction is an INLINEASM. This allows memory instructions to be moved around INLINEASM instructions. llvm-svn: 123044
2011-01-08 00:50:32 +01:00
Ops.push_back(InOps[InlineAsm::Op_ExtraInfo]); // 3 (SideEffect, AlignStack)
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
Three changes: 1. Introduce some enums and accessors in the InlineAsm class that eliminate a ton of magic numbers when handling inline asm SDNode. 2. Add a new MDNodeSDNode selection dag node type that holds a MDNode (shocking!) 3. Add a new argument to ISD::INLINEASM nodes that hold !srcloc metadata, propagating it to the instruction emitter, which drops it. No functionality change. llvm-svn: 100605
2010-04-07 07:20:54 +02:00
unsigned i = InlineAsm::Op_FirstOperand, e = InOps.size();
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
if (InOps[e-1].getValueType() == MVT::Glue)
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
--e; // Don't process a glue operand if it is here.
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
while (i != e) {
Rename ConstantSDNode::getValue to getZExtValue, for consistency with ConstantInt. This led to fixing a bug in TargetLowering.cpp using getValue instead of getAPIntValue. llvm-svn: 56159
2008-09-12 18:56:44 +02:00
unsigned Flags = cast<ConstantSDNode>(InOps[i])->getZExtValue();
Three changes: 1. Introduce some enums and accessors in the InlineAsm class that eliminate a ton of magic numbers when handling inline asm SDNode. 2. Add a new MDNodeSDNode selection dag node type that holds a MDNode (shocking!) 3. Add a new argument to ISD::INLINEASM nodes that hold !srcloc metadata, propagating it to the instruction emitter, which drops it. No functionality change. llvm-svn: 100605
2010-04-07 07:20:54 +02:00
if (!InlineAsm::isMemKind(Flags)) {
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
// Just skip over this operand, copying the operands verbatim.
For inline asm output operand that matches an input. Encode the input operand index in the high bits. llvm-svn: 67387
2009-03-20 19:03:34 +01:00
Ops.insert(Ops.end(), InOps.begin()+i,
InOps.begin()+i+InlineAsm::getNumOperandRegisters(Flags) + 1);
i += InlineAsm::getNumOperandRegisters(Flags) + 1;
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
} else {
For inline asm output operand that matches an input. Encode the input operand index in the high bits. llvm-svn: 67387
2009-03-20 19:03:34 +01:00
assert(InlineAsm::getNumOperandRegisters(Flags) == 1 &&
"Memory operand with multiple values?");
Recommit r232027 with PR22883 fixed: Add infrastructure for support of multiple memory constraints. The operand flag word for ISD::INLINEASM nodes now contains a 15-bit memory constraint ID when the operand kind is Kind_Mem. This constraint ID is a numeric equivalent to the constraint code string and is converted with a target specific hook in TargetLowering. This patch maps all memory constraints to InlineAsm::Constraint_m so there is no functional change at this point. It just proves that using these previously unused bits in the encoding of the flag word doesn't break anything. The next patch will make each target preserve the current mapping of everything to Constraint_m for itself while changing the target independent implementation of the hook to return Constraint_Unknown appropriately. Each target will then be adapted in separate patches to use appropriate Constraint_* values. PR22883 was caused the matching operands copying the whole of the operand flags for the matched operand. This included the constraint id which needed to be replaced with the operand number. This has been fixed with a conversion function. Following on from this, matching operands also used the operand number as the constraint id. This has been fixed by looking up the matched operand and taking it from there. llvm-svn: 232165
2015-03-13 13:45:09 +01:00
unsigned TiedToOperand;
if (InlineAsm::isUseOperandTiedToDef(Flags, TiedToOperand)) {
// We need the constraint ID from the operand this is tied to.
unsigned CurOp = InlineAsm::Op_FirstOperand;
Flags = cast<ConstantSDNode>(InOps[CurOp])->getZExtValue();
for (; TiedToOperand; --TiedToOperand) {
CurOp += InlineAsm::getNumOperandRegisters(Flags)+1;
Flags = cast<ConstantSDNode>(InOps[CurOp])->getZExtValue();
}
}
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
// Otherwise, this is a memory operand. Ask the target to select it.
Rename SDOperand to SDValue. llvm-svn: 54128
2008-07-27 23:46:04 +02:00
std::vector<SDValue> SelOps;
[inlineasm] Propagate operand constraints to the backend When SelectionDAGISel transforms a node representing an inline asm block, memory constraint information is not preserved. This can cause constraints to be broken when a memory offset is of the form: offset + frame index when the frame is resolved. By propagating the constraints all the way to the backend, targets can enforce memory operands of inline assembly to conform to their constraints. For MIPSR6, some instructions had their offsets reduced to 9 bits from 16 bits such as ll/sc. This becomes problematic when using inline assembly to perform atomic operations, as an offset can generated that is too big to encode in the instruction. Reviewers: dsanders, vkalintris Differential Review: https://reviews.llvm.org/D21615 llvm-svn: 275786
2016-07-18 15:17:31 +02:00
unsigned ConstraintID = InlineAsm::getMemoryConstraintID(Flags);
if (SelectInlineAsmMemoryOperand(InOps[i+1], ConstraintID, SelOps))
rename llvm::llvm_report_error -> llvm::report_fatal_error llvm-svn: 100709
2010-04-08 00:58:41 +02:00
report_fatal_error("Could not match memory address. Inline asm"
minor tidying. llvm-svn: 100725
2010-04-08 01:50:38 +02:00
" failure!");
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
// Add this to the output node.
Three changes: 1. Introduce some enums and accessors in the InlineAsm class that eliminate a ton of magic numbers when handling inline asm SDNode. 2. Add a new MDNodeSDNode selection dag node type that holds a MDNode (shocking!) 3. Add a new argument to ISD::INLINEASM nodes that hold !srcloc metadata, propagating it to the instruction emitter, which drops it. No functionality change. llvm-svn: 100605
2010-04-07 07:20:54 +02:00
unsigned NewFlags =
InlineAsm::getFlagWord(InlineAsm::Kind_Mem, SelOps.size());
[inlineasm] Propagate operand constraints to the backend When SelectionDAGISel transforms a node representing an inline asm block, memory constraint information is not preserved. This can cause constraints to be broken when a memory offset is of the form: offset + frame index when the frame is resolved. By propagating the constraints all the way to the backend, targets can enforce memory operands of inline assembly to conform to their constraints. For MIPSR6, some instructions had their offsets reduced to 9 bits from 16 bits such as ll/sc. This becomes problematic when using inline assembly to perform atomic operations, as an offset can generated that is too big to encode in the instruction. Reviewers: dsanders, vkalintris Differential Review: https://reviews.llvm.org/D21615 llvm-svn: 275786
2016-07-18 15:17:31 +02:00
NewFlags = InlineAsm::getFlagWordForMem(NewFlags, ConstraintID);
Reapply r235977 "[DebugInfo] Add debug locations to constant SD nodes" [DebugInfo] Add debug locations to constant SD nodes This adds debug location to constant nodes of Selection DAG and updates all places that create constants to pass debug locations (see PR13269). Can't guarantee that all locations are correct, but in a lot of cases choice is obvious, so most of them should be. At least all tests pass. Tests for these changes do not cover everything, instead just check it for SDNodes, ARM and AArch64 where it's easy to get incorrect locations on constants. This is not complete fix as FastISel contains workaround for wrong debug locations, which drops locations from instructions on processing constants, but there isn't currently a way to use debug locations from constants there as llvm::Constant doesn't cache it (yet). Although this is a bit different issue, not directly related to these changes. Differential Revision: http://reviews.llvm.org/D9084 llvm-svn: 235989
2015-04-28 16:05:47 +02:00
Ops.push_back(CurDAG->getTargetConstant(NewFlags, DL, MVT::i32));
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
Ops.insert(Ops.end(), SelOps.begin(), SelOps.end());
i += 2;
}
}
Tabs -> spaces, and remove trailing whitespace. llvm-svn: 82355
2009-09-20 04:20:51 +02:00
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// Add the glue input back if present.
Implement (most of) selection of inline asm memory operands. llvm-svn: 26350
2006-02-24 03:13:54 +01:00
if (e != InOps.size())
Ops.push_back(InOps.back());
}
Do not use typeinfo to identify pass in pass manager. llvm-svn: 36632
2007-05-01 23:15:47 +02:00
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
/// findGlueUse - Return use of MVT::Glue value produced by the specified
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
/// SDNode.
///
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
static SDNode *findGlueUse(SDNode *N) {
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
unsigned FlagResNo = N->getNumValues()-1;
for (SDNode::use_iterator I = N->use_begin(), E = N->use_end(); I != E; ++I) {
SDUse &Use = I.getUse();
if (Use.getResNo() == FlagResNo)
return Use.getUser();
}
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
return nullptr;
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
}
/// findNonImmUse - Return true if "Use" is a non-immediate use of "Def".
/// This function recursively traverses up the operand chain, ignoring
/// certain nodes.
static bool findNonImmUse(SDNode *Use, SDNode* Def, SDNode *ImmedUse,
Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid needing to mention the size. llvm-svn: 216158
2014-08-21 07:55:13 +02:00
SDNode *Root, SmallPtrSetImpl<SDNode*> &Visited,
Fix some issues in WalkChainUsers dealing with CopyToReg/CopyFromReg/INLINEASM. These are annoying because they have the same opcode before an after isel. Fix this by setting their NodeID to -1 to indicate that they are selected, just like what automatically happens when selecting things that end up being machine nodes. With that done, give IsLegalToFold a new flag that causes it to ignore chains. This lets the HandleMergeInputChains routine be the one place that validates chains after a match is successful, enabling the new hotness in chain processing. This smarter chain processing eliminates the need for "PreprocessRMW" in the X86 and MSP430 backends and enables MSP to start matching it's multiple mem operand instructions more aggressively. I currently #if out the dead code in the X86 backend and MSP backend, I'll remove it for real in a follow-on patch. The testcase changes are: test/CodeGen/X86/sse3.ll: we generate better code test/CodeGen/X86/store_op_load_fold2.ll: PreprocessRMW was miscompiling this before, we now generate correct code Convert it to filecheck while I'm at it. test/CodeGen/MSP430/Inst16mm.ll: Add a testcase for mem/mem folding to make anton happy. :) llvm-svn: 97596
2010-03-02 23:20:06 +01:00
bool IgnoreChains) {
fix a bug in findNonImmUse (used by IsLegalToFold) where nodes with no id's would cause early exit allowing IsLegalToFold to return true instead of false, producing a cyclic dag. This was striking the new isel because it isn't using SelectNodeTo yet, which theoretically is just an optimization. llvm-svn: 96972
2010-02-23 20:32:27 +01:00
// The NodeID's are given uniques ID's where a node ID is guaranteed to be
// greater than all of its (recursive) operands. If we scan to a point where
// 'use' is smaller than the node we're scanning for, then we know we will
// never find it.
//
// The Use may be -1 (unassigned) if it is a newly allocated node. This can
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// happen because we scan down to newly selected nodes in the case of glue
fix a bug in findNonImmUse (used by IsLegalToFold) where nodes with no id's would cause early exit allowing IsLegalToFold to return true instead of false, producing a cyclic dag. This was striking the new isel because it isn't using SelectNodeTo yet, which theoretically is just an optimization. llvm-svn: 96972
2010-02-23 20:32:27 +01:00
// uses.
if ((Use->getNodeId() < Def->getNodeId() && Use->getNodeId() != -1))
return false;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
fix a bug in findNonImmUse (used by IsLegalToFold) where nodes with no id's would cause early exit allowing IsLegalToFold to return true instead of false, producing a cyclic dag. This was striking the new isel because it isn't using SelectNodeTo yet, which theoretically is just an optimization. llvm-svn: 96972
2010-02-23 20:32:27 +01:00
// Don't revisit nodes if we already scanned it and didn't fail, we know we
// won't fail if we scan it again.
Update SetVector to rely on the underlying set's insert to return a pair<iterator, bool> This is to be consistent with StringSet and ultimately with the standard library's associative container insert function. This lead to updating SmallSet::insert to return pair<iterator, bool>, and then to update SmallPtrSet::insert to return pair<iterator, bool>, and then to update all the existing users of those functions... llvm-svn: 222334
2014-11-19 08:49:26 +01:00
if (!Visited.insert(Use).second)
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
return false;
Convert a bunch of loops to foreach. NFC. This uses the new SDNode::op_values() iterator range committed in r240805. llvm-svn: 240822
2015-06-26 21:37:02 +02:00
for (const SDValue &Op : Use->op_values()) {
Fix some issues in WalkChainUsers dealing with CopyToReg/CopyFromReg/INLINEASM. These are annoying because they have the same opcode before an after isel. Fix this by setting their NodeID to -1 to indicate that they are selected, just like what automatically happens when selecting things that end up being machine nodes. With that done, give IsLegalToFold a new flag that causes it to ignore chains. This lets the HandleMergeInputChains routine be the one place that validates chains after a match is successful, enabling the new hotness in chain processing. This smarter chain processing eliminates the need for "PreprocessRMW" in the X86 and MSP430 backends and enables MSP to start matching it's multiple mem operand instructions more aggressively. I currently #if out the dead code in the X86 backend and MSP backend, I'll remove it for real in a follow-on patch. The testcase changes are: test/CodeGen/X86/sse3.ll: we generate better code test/CodeGen/X86/store_op_load_fold2.ll: PreprocessRMW was miscompiling this before, we now generate correct code Convert it to filecheck while I'm at it. test/CodeGen/MSP430/Inst16mm.ll: Add a testcase for mem/mem folding to make anton happy. :) llvm-svn: 97596
2010-03-02 23:20:06 +01:00
// Ignore chain uses, they are validated by HandleMergeInputChains.
Convert a bunch of loops to foreach. NFC. This uses the new SDNode::op_values() iterator range committed in r240805. llvm-svn: 240822
2015-06-26 21:37:02 +02:00
if (Op.getValueType() == MVT::Other && IgnoreChains)
Fix some issues in WalkChainUsers dealing with CopyToReg/CopyFromReg/INLINEASM. These are annoying because they have the same opcode before an after isel. Fix this by setting their NodeID to -1 to indicate that they are selected, just like what automatically happens when selecting things that end up being machine nodes. With that done, give IsLegalToFold a new flag that causes it to ignore chains. This lets the HandleMergeInputChains routine be the one place that validates chains after a match is successful, enabling the new hotness in chain processing. This smarter chain processing eliminates the need for "PreprocessRMW" in the X86 and MSP430 backends and enables MSP to start matching it's multiple mem operand instructions more aggressively. I currently #if out the dead code in the X86 backend and MSP backend, I'll remove it for real in a follow-on patch. The testcase changes are: test/CodeGen/X86/sse3.ll: we generate better code test/CodeGen/X86/store_op_load_fold2.ll: PreprocessRMW was miscompiling this before, we now generate correct code Convert it to filecheck while I'm at it. test/CodeGen/MSP430/Inst16mm.ll: Add a testcase for mem/mem folding to make anton happy. :) llvm-svn: 97596
2010-03-02 23:20:06 +01:00
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Convert a bunch of loops to foreach. NFC. This uses the new SDNode::op_values() iterator range committed in r240805. llvm-svn: 240822
2015-06-26 21:37:02 +02:00
SDNode *N = Op.getNode();
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
if (N == Def) {
if (Use == ImmedUse || Use == Root)
continue; // We are not looking for immediate use.
assert(N != Root);
return true;
}
// Traverse up the operand chain.
Fix some issues in WalkChainUsers dealing with CopyToReg/CopyFromReg/INLINEASM. These are annoying because they have the same opcode before an after isel. Fix this by setting their NodeID to -1 to indicate that they are selected, just like what automatically happens when selecting things that end up being machine nodes. With that done, give IsLegalToFold a new flag that causes it to ignore chains. This lets the HandleMergeInputChains routine be the one place that validates chains after a match is successful, enabling the new hotness in chain processing. This smarter chain processing eliminates the need for "PreprocessRMW" in the X86 and MSP430 backends and enables MSP to start matching it's multiple mem operand instructions more aggressively. I currently #if out the dead code in the X86 backend and MSP backend, I'll remove it for real in a follow-on patch. The testcase changes are: test/CodeGen/X86/sse3.ll: we generate better code test/CodeGen/X86/store_op_load_fold2.ll: PreprocessRMW was miscompiling this before, we now generate correct code Convert it to filecheck while I'm at it. test/CodeGen/MSP430/Inst16mm.ll: Add a testcase for mem/mem folding to make anton happy. :) llvm-svn: 97596
2010-03-02 23:20:06 +01:00
if (findNonImmUse(N, Def, ImmedUse, Root, Visited, IgnoreChains))
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
return true;
}
return false;
}
Split SelectionDAGISel::IsLegalAndProfitableToFold to IsLegalToFold and IsProfitableToFold. The generic version of the later simply checks whether the folding candidate has a single use. This allows the target isel routines more flexibility in deciding whether folding makes sense. The specific case we are interested in is folding constant pool loads with multiple uses. llvm-svn: 96255
2010-02-15 20:41:07 +01:00
/// IsProfitableToFold - Returns true if it's profitable to fold the specific
/// operand node N of U during instruction selection that starts at Root.
bool SelectionDAGISel::IsProfitableToFold(SDValue N, SDNode *U,
SDNode *Root) const {
if (OptLevel == CodeGenOpt::None) return false;
return N.hasOneUse();
}
/// IsLegalToFold - Returns true if the specific operand node N of
/// U can be folded during instruction selection that starts at Root.
Fix some issues in WalkChainUsers dealing with CopyToReg/CopyFromReg/INLINEASM. These are annoying because they have the same opcode before an after isel. Fix this by setting their NodeID to -1 to indicate that they are selected, just like what automatically happens when selecting things that end up being machine nodes. With that done, give IsLegalToFold a new flag that causes it to ignore chains. This lets the HandleMergeInputChains routine be the one place that validates chains after a match is successful, enabling the new hotness in chain processing. This smarter chain processing eliminates the need for "PreprocessRMW" in the X86 and MSP430 backends and enables MSP to start matching it's multiple mem operand instructions more aggressively. I currently #if out the dead code in the X86 backend and MSP backend, I'll remove it for real in a follow-on patch. The testcase changes are: test/CodeGen/X86/sse3.ll: we generate better code test/CodeGen/X86/store_op_load_fold2.ll: PreprocessRMW was miscompiling this before, we now generate correct code Convert it to filecheck while I'm at it. test/CodeGen/MSP430/Inst16mm.ll: Add a testcase for mem/mem folding to make anton happy. :) llvm-svn: 97596
2010-03-02 23:20:06 +01:00
bool SelectionDAGISel::IsLegalToFold(SDValue N, SDNode *U, SDNode *Root,
Use const qualifiers with TargetLowering. This eliminates several const_casts, and it reinforces the design of the Target classes being immutable. SelectionDAGISel::IsLegalToFold is now a static member function, because PIC16 uses it in an unconventional way. There is more room for API cleanup here. And PIC16's AsmPrinter no longer uses TargetLowering. llvm-svn: 101635
2010-04-17 17:26:15 +02:00
CodeGenOpt::Level OptLevel,
bool IgnoreChains) {
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
if (OptLevel == CodeGenOpt::None) return false;
// If Root use can somehow reach N through a path that that doesn't contain
// U then folding N would create a cycle. e.g. In the following
// diagram, Root can reach N through X. If N is folded into into Root, then
// X is both a predecessor and a successor of U.
//
// [N*] //
// ^ ^ //
// / \ //
// [U*] [X]? //
// ^ ^ //
// \ / //
// \ / //
// [Root*] //
//
// * indicates nodes to be folded together.
//
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// If Root produces glue, then it gets (even more) interesting. Since it
// will be "glued" together with its glue use in the scheduler, we need to
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
// check if it might reach N.
//
// [N*] //
// ^ ^ //
// / \ //
// [U*] [X]? //
// ^ ^ //
// \ \ //
// \ | //
// [Root*] | //
// ^ | //
// f | //
// | / //
// [Y] / //
// ^ / //
// f / //
// | / //
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// [GU] //
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
//
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// If GU (glue use) indirectly reaches N (the load), and Root folds N
// (call it Fold), then X is a predecessor of GU and a successor of
// Fold. But since Fold and GU are glued together, this will create
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
// a cycle in the scheduling graph.
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// If the node has glue, walk down the graph to the "lowest" node in the
// glueged set.
Rename MVT to EVT, in preparation for splitting SimpleValueType out into its own struct type. llvm-svn: 78610
2009-08-11 00:56:29 +02:00
EVT VT = Root->getValueType(Root->getNumValues()-1);
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
while (VT == MVT::Glue) {
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
SDNode *GU = findGlueUse(Root);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
if (!GU)
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
break;
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
Root = GU;
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
VT = Root->getValueType(Root->getNumValues()-1);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// If our query node has a glue result with a use, we've walked up it. If
Fix PR6497, a bug where we'd fold a load into an addc node which has a flag. That flag in turn was used by an already-selected adde which turned into an ADC32ri8 which used a selected load which was chained to the load we folded. This flag use caused us to form a cycle. Fix this by not ignoring chains in IsLegalToFold even in cases where the isel thinks it can. llvm-svn: 97791
2010-03-05 07:19:13 +01:00
// the user (which has already been selected) has a chain or indirectly uses
// the chain, our WalkChainUsers predicate will not consider it. Because of
// this, we cannot ignore chains in this predicate.
IgnoreChains = false;
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
inline a small function with one call site. llvm-svn: 97789
2010-03-05 06:49:45 +01:00
SmallPtrSet<SDNode*, 16> Visited;
return !findNonImmUse(Root, N.getNode(), U, Root, Visited, IgnoreChains);
Factor out cycle-finder code and make it generic. llvm-svn: 71241
2009-05-08 20:51:58 +02:00
}
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
void SelectionDAGISel::Select_INLINEASM(SDNode *N) {
Reapply r235977 "[DebugInfo] Add debug locations to constant SD nodes" [DebugInfo] Add debug locations to constant SD nodes This adds debug location to constant nodes of Selection DAG and updates all places that create constants to pass debug locations (see PR13269). Can't guarantee that all locations are correct, but in a lot of cases choice is obvious, so most of them should be. At least all tests pass. Tests for these changes do not cover everything, instead just check it for SDNodes, ARM and AArch64 where it's easy to get incorrect locations on constants. This is not complete fix as FastISel contains workaround for wrong debug locations, which drops locations from instructions on processing constants, but there isn't currently a way to use debug locations from constants there as llvm::Constant doesn't cache it (yet). Although this is a bit different issue, not directly related to these changes. Differential Revision: http://reviews.llvm.org/D9084 llvm-svn: 235989
2015-04-28 16:05:47 +02:00
SDLoc DL(N);
Change SelectCode's argument from SDValue to SDNode *, to make it more clear what information these functions are actually using. This is also a micro-optimization, as passing a SDNode * around is simpler than passing a { SDNode *, int } by value or reference. llvm-svn: 92564
2010-01-05 02:24:18 +01:00
std::vector<SDValue> Ops(N->op_begin(), N->op_end());
Reapply r235977 "[DebugInfo] Add debug locations to constant SD nodes" [DebugInfo] Add debug locations to constant SD nodes This adds debug location to constant nodes of Selection DAG and updates all places that create constants to pass debug locations (see PR13269). Can't guarantee that all locations are correct, but in a lot of cases choice is obvious, so most of them should be. At least all tests pass. Tests for these changes do not cover everything, instead just check it for SDNodes, ARM and AArch64 where it's easy to get incorrect locations on constants. This is not complete fix as FastISel contains workaround for wrong debug locations, which drops locations from instructions on processing constants, but there isn't currently a way to use debug locations from constants there as llvm::Constant doesn't cache it (yet). Although this is a bit different issue, not directly related to these changes. Differential Revision: http://reviews.llvm.org/D9084 llvm-svn: 235989
2015-04-28 16:05:47 +02:00
SelectInlineAsmMemoryOperands(Ops, DL);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Make constant arrays that are passed to functions as const. In theory this allows the compiler to skip materializing the array on the stack. In practice clang often fails to do that, but that's a different story. NFC. llvm-svn: 231571
2015-03-07 18:41:00 +01:00
const EVT VTs[] = {MVT::Other, MVT::Glue};
Reapply r235977 "[DebugInfo] Add debug locations to constant SD nodes" [DebugInfo] Add debug locations to constant SD nodes This adds debug location to constant nodes of Selection DAG and updates all places that create constants to pass debug locations (see PR13269). Can't guarantee that all locations are correct, but in a lot of cases choice is obvious, so most of them should be. At least all tests pass. Tests for these changes do not cover everything, instead just check it for SDNodes, ARM and AArch64 where it's easy to get incorrect locations on constants. This is not complete fix as FastISel contains workaround for wrong debug locations, which drops locations from instructions on processing constants, but there isn't currently a way to use debug locations from constants there as llvm::Constant doesn't cache it (yet). Although this is a bit different issue, not directly related to these changes. Differential Revision: http://reviews.llvm.org/D9084 llvm-svn: 235989
2015-04-28 16:05:47 +02:00
SDValue New = CurDAG->getNode(ISD::INLINEASM, DL, VTs, Ops);
Fix some issues in WalkChainUsers dealing with CopyToReg/CopyFromReg/INLINEASM. These are annoying because they have the same opcode before an after isel. Fix this by setting their NodeID to -1 to indicate that they are selected, just like what automatically happens when selecting things that end up being machine nodes. With that done, give IsLegalToFold a new flag that causes it to ignore chains. This lets the HandleMergeInputChains routine be the one place that validates chains after a match is successful, enabling the new hotness in chain processing. This smarter chain processing eliminates the need for "PreprocessRMW" in the X86 and MSP430 backends and enables MSP to start matching it's multiple mem operand instructions more aggressively. I currently #if out the dead code in the X86 backend and MSP backend, I'll remove it for real in a follow-on patch. The testcase changes are: test/CodeGen/X86/sse3.ll: we generate better code test/CodeGen/X86/store_op_load_fold2.ll: PreprocessRMW was miscompiling this before, we now generate correct code Convert it to filecheck while I'm at it. test/CodeGen/MSP430/Inst16mm.ll: Add a testcase for mem/mem folding to make anton happy. :) llvm-svn: 97596
2010-03-02 23:20:06 +01:00
New->setNodeId(-1);
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
ReplaceUses(N, New.getNode());
CurDAG->RemoveDeadNode(N);
Move some code from being emitted as boilerplate duplicated in every *ISelDAGToDAG.cpp to being regular code in SelectionDAGISel.cpp. llvm-svn: 85530
2009-10-29 23:30:23 +01:00
}
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
void SelectionDAGISel::Select_READ_REGISTER(SDNode *Op) {
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
SDLoc dl(Op);
Preserve the order of READ_REGISTER and WRITE_REGISTER At the present time, we don't have a way to represent general dependency relationships, so everything is represented using memory dependency. In order to preserve the data dependency of a READ_REGISTER on WRITE_REGISTER, we need to model WRITE_REGISTER as writing (which we had been doing) and model READ_REGISTER as reading (which we had not been doing). Fix this, and also the way that the chain operands were generated at the SDAG level. Patch by Nicholas Paul Johnson, thanks! Test case by me. llvm-svn: 237584
2015-05-18 18:42:10 +02:00
MDNodeSDNode *MD = dyn_cast<MDNodeSDNode>(Op->getOperand(1));
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
const MDString *RegStr = dyn_cast<MDString>(MD->getMD()->getOperand(0));
Cache TargetLowering on SelectionDAGISel and update previous calls to getTargetLowering() with the cached variable. llvm-svn: 219284
2014-10-08 09:32:17 +02:00
unsigned Reg =
Allow {e,r}bp as the target of {read,write}_register. This patch allows the read_register and write_register intrinsics to read/write the RBP/EBP registers on X86 iff the targeted register is the frame pointer for the containing function. Differential Revision: http://reviews.llvm.org/D10977 llvm-svn: 241827
2015-07-09 19:40:29 +02:00
TLI->getRegisterByName(RegStr->getString().data(), Op->getValueType(0),
*CurDAG);
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
SDValue New = CurDAG->getCopyFromReg(
Preserve the order of READ_REGISTER and WRITE_REGISTER At the present time, we don't have a way to represent general dependency relationships, so everything is represented using memory dependency. In order to preserve the data dependency of a READ_REGISTER on WRITE_REGISTER, we need to model WRITE_REGISTER as writing (which we had been doing) and model READ_REGISTER as reading (which we had not been doing). Fix this, and also the way that the chain operands were generated at the SDAG level. Patch by Nicholas Paul Johnson, thanks! Test case by me. llvm-svn: 237584
2015-05-18 18:42:10 +02:00
Op->getOperand(0), dl, Reg, Op->getValueType(0));
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
New->setNodeId(-1);
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
ReplaceUses(Op, New.getNode());
CurDAG->RemoveDeadNode(Op);
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
}
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
void SelectionDAGISel::Select_WRITE_REGISTER(SDNode *Op) {
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
SDLoc dl(Op);
MDNodeSDNode *MD = dyn_cast<MDNodeSDNode>(Op->getOperand(1));
const MDString *RegStr = dyn_cast<MDString>(MD->getMD()->getOperand(0));
Cache TargetLowering on SelectionDAGISel and update previous calls to getTargetLowering() with the cached variable. llvm-svn: 219284
2014-10-08 09:32:17 +02:00
unsigned Reg = TLI->getRegisterByName(RegStr->getString().data(),
Allow {e,r}bp as the target of {read,write}_register. This patch allows the read_register and write_register intrinsics to read/write the RBP/EBP registers on X86 iff the targeted register is the frame pointer for the containing function. Differential Revision: http://reviews.llvm.org/D10977 llvm-svn: 241827
2015-07-09 19:40:29 +02:00
Op->getOperand(2).getValueType(),
*CurDAG);
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
SDValue New = CurDAG->getCopyToReg(
Preserve the order of READ_REGISTER and WRITE_REGISTER At the present time, we don't have a way to represent general dependency relationships, so everything is represented using memory dependency. In order to preserve the data dependency of a READ_REGISTER on WRITE_REGISTER, we need to model WRITE_REGISTER as writing (which we had been doing) and model READ_REGISTER as reading (which we had not been doing). Fix this, and also the way that the chain operands were generated at the SDAG level. Patch by Nicholas Paul Johnson, thanks! Test case by me. llvm-svn: 237584
2015-05-18 18:42:10 +02:00
Op->getOperand(0), dl, Reg, Op->getOperand(2));
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
New->setNodeId(-1);
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
ReplaceUses(Op, New.getNode());
CurDAG->RemoveDeadNode(Op);
Implememting named register intrinsics This patch implements the infrastructure to use named register constructs in programs that need access to specific registers (bare metal, kernels, etc). So far, only the stack pointer is supported as a technology preview, but as it is, the intrinsic can already support all non-allocatable registers from any architecture. llvm-svn: 208104
2014-05-06 18:51:25 +02:00
}
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
void SelectionDAGISel::Select_UNDEF(SDNode *N) {
SDAG: Have SelectNodeTo replace uses if it CSE's instead of morphing a node It's awkward to force callers of SelectNodeTo to figure out whether the node was morphed or CSE'd. Update uses here instead of requiring callers to (sometimes) do it. llvm-svn: 269235
2016-05-11 23:00:33 +02:00
CurDAG->SelectNodeTo(N, TargetOpcode::IMPLICIT_DEF, N->getValueType(0));
Move some code from being emitted as boilerplate duplicated in every *ISelDAGToDAG.cpp to being regular code in SelectionDAGISel.cpp. llvm-svn: 85530
2009-10-29 23:30:23 +01:00
}
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
/// GetVBR - decode a vbr encoding whose top bit is set.
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline uint64_t
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
GetVBR(uint64_t Val, const unsigned char *MatcherTable, unsigned &Idx) {
assert(Val >= 128 && "Not a VBR");
Val &= 127; // Remove first vbr bit.
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
unsigned Shift = 7;
uint64_t NextBits;
do {
eliminate GetInt1/2 llvm-svn: 97426
2010-02-28 23:38:43 +01:00
NextBits = MatcherTable[Idx++];
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
Val |= (NextBits&127) << Shift;
Shift += 7;
} while (NextBits & 128);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
return Val;
}
SDAG: Remove OPC_MarkGlueResults and associated logic. NFC This opcode never happens in practice, and yet the logic we have in place to handle it would be undefined behaviour if we ever executed it. Remove it rather than trying to refactor code that's never reached. llvm-svn: 268692
2016-05-06 00:37:45 +02:00
/// When a match is complete, this method updates uses of interior chain results
/// to use the new results.
void SelectionDAGISel::UpdateChains(
SDNode *NodeToMatch, SDValue InputChain,
SDAG: Update ChainNodesMatched during UpdateChains if a node is replaced Previously, we would hit UB (or the ISD::DELETED_NODE assert) if we happened to replace a node during UpdateChains, because it would be left in the list we were iterating over. This nulls out the pointer when that happens so that we can avoid the issue. Fixes llvm.org/PR31710 llvm-svn: 293522
2017-01-30 19:29:46 +01:00
SmallVectorImpl<SDNode *> &ChainNodesMatched, bool isMorphNodeTo) {
Fix the xfail I added a couple of patches back. The issue was that we weren't properly handling the case when interior nodes of a matched pattern become dead after updating chain and flag uses. Now we handle this explicitly in UpdateChainsAndFlags. llvm-svn: 97561
2010-03-02 08:50:03 +01:00
SmallVector<SDNode*, 4> NowDeadNodes;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// Now that all the normal results are replaced, we replace the chain and
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// glue results if present.
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
if (!ChainNodesMatched.empty()) {
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
assert(InputChain.getNode() &&
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
"Matched input chains but didn't produce a chain");
// Loop over all of the nodes we matched that produced a chain result.
// Replace all the chain results with the final chain we ended up with.
for (unsigned i = 0, e = ChainNodesMatched.size(); i != e; ++i) {
SDNode *ChainNode = ChainNodesMatched[i];
SDAG: Update ChainNodesMatched during UpdateChains if a node is replaced Previously, we would hit UB (or the ISD::DELETED_NODE assert) if we happened to replace a node during UpdateChains, because it would be left in the list we were iterating over. This nulls out the pointer when that happens so that we can avoid the issue. Fixes llvm.org/PR31710 llvm-svn: 293522
2017-01-30 19:29:46 +01:00
// If ChainNode is null, it's because we replaced it on a previous
// iteration and we cleared it out of the map. Just skip it.
if (!ChainNode)
continue;
Re-apply "SDAG: Update ChainNodesMatched as nodes are deleted" My first attempt at this had an overly aggressive assert - chain nodes will only be removed, but we could hit the assert if a non-chain node was CSE'd (NodeToMatch, for instance). This reapplies r271706 by reverting r271713 and fixing an assert. Original message: Avoid relying on UB by looking into deleted nodes for a marker value. Instead, update the list of chain nodes as we go. llvm-svn: 271733
2016-06-03 22:47:40 +02:00
assert(ChainNode->getOpcode() != ISD::DELETED_NODE &&
"Deleted node left in chain");
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// Don't replace the results of the root node if we're doing a
// MorphNodeTo.
if (ChainNode == NodeToMatch && isMorphNodeTo)
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
SDValue ChainVal = SDValue(ChainNode, ChainNode->getNumValues()-1);
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
if (ChainVal.getValueType() == MVT::Glue)
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
ChainVal = ChainVal.getValue(ChainVal->getNumValues()-2);
assert(ChainVal.getValueType() == MVT::Other && "Not a chain?");
SDAG: Update ChainNodesMatched during UpdateChains if a node is replaced Previously, we would hit UB (or the ISD::DELETED_NODE assert) if we happened to replace a node during UpdateChains, because it would be left in the list we were iterating over. This nulls out the pointer when that happens so that we can avoid the issue. Fixes llvm.org/PR31710 llvm-svn: 293522
2017-01-30 19:29:46 +01:00
SelectionDAG::DAGNodeDeletedListener NDL(
*CurDAG, [&](SDNode *N, SDNode *E) {
std::replace(ChainNodesMatched.begin(), ChainNodesMatched.end(), N,
static_cast<SDNode *>(nullptr));
});
Register DAGUpdateListeners with SelectionDAG. Instead of passing listener pointers to RAUW, let SelectionDAG itself keep a linked list of interested listeners. This makes it possible to have multiple listeners active at once, like RAUWUpdateListener was already doing. It also makes it possible to register listeners up the call stack without controlling all RAUW calls below. DAGUpdateListener uses an RAII pattern to add itself to the SelectionDAG list of active listeners. llvm-svn: 155248
2012-04-21 00:08:46 +02:00
CurDAG->ReplaceAllUsesOfValueWith(ChainVal, InputChain);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
don't add nodes to the now-dead nodes list multiple times, this can cause a crash on crazy situations in msp430 when morph-node-to is disabled. llvm-svn: 99739
2010-03-28 07:28:31 +02:00
// If the node became dead and we haven't already seen it, delete it.
SDAG: Don't leave dangling dead nodes after SelectCodeCommon Relying on the caller to clean up after we've replaced all uses of a node won't work when we've migrated to the `void Select(...)` API. llvm-svn: 268774
2016-05-06 20:42:16 +02:00
if (ChainNode != NodeToMatch && ChainNode->use_empty() &&
don't add nodes to the now-dead nodes list multiple times, this can cause a crash on crazy situations in msp430 when morph-node-to is disabled. llvm-svn: 99739
2010-03-28 07:28:31 +02:00
!std::count(NowDeadNodes.begin(), NowDeadNodes.end(), ChainNode))
Fix the xfail I added a couple of patches back. The issue was that we weren't properly handling the case when interior nodes of a matched pattern become dead after updating chain and flag uses. Now we handle this explicitly in UpdateChainsAndFlags. llvm-svn: 97561
2010-03-02 08:50:03 +01:00
NowDeadNodes.push_back(ChainNode);
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Fix the xfail I added a couple of patches back. The issue was that we weren't properly handling the case when interior nodes of a matched pattern become dead after updating chain and flag uses. Now we handle this explicitly in UpdateChainsAndFlags. llvm-svn: 97561
2010-03-02 08:50:03 +01:00
if (!NowDeadNodes.empty())
Register DAGUpdateListeners with SelectionDAG. Instead of passing listener pointers to RAUW, let SelectionDAG itself keep a linked list of interested listeners. This makes it possible to have multiple listeners active at once, like RAUWUpdateListener was already doing. It also makes it possible to register listeners up the call stack without controlling all RAUW calls below. DAGUpdateListener uses an RAII pattern to add itself to the SelectionDAG list of active listeners. llvm-svn: 155248
2012-04-21 00:08:46 +02:00
CurDAG->RemoveDeadNodes(NowDeadNodes);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << "ISEL: Match complete!\n");
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
enum ChainResult {
CR_Simple,
CR_InducesCycle,
CR_LeadsToInteriorNode
};
/// WalkChainUsers - Walk down the users of the specified chained node that is
/// part of the pattern we're matching, looking at all of the users we find.
/// This determines whether something is an interior node, whether we have a
/// non-pattern node in between two pattern nodes (which prevent folding because
/// it would induce a cycle) and whether we have a TokenFactor node sandwiched
/// between pattern nodes (in which case the TF becomes part of the pattern).
///
/// The walk we do here is guaranteed to be small because we quickly get down to
/// already selected nodes "below" us.
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
static ChainResult
Add some constantness. No functionality change. llvm-svn: 155859
2012-05-01 01:41:30 +02:00
WalkChainUsers(const SDNode *ChainedNode,
[SelectionDAG] Eliminate exponential behavior in WalkChainUsers llvm-svn: 259315
2016-01-31 04:59:34 +01:00
SmallVectorImpl<SDNode *> &ChainedNodesInPattern,
DenseMap<const SDNode *, ChainResult> &TokenFactorResult,
SmallVectorImpl<SDNode *> &InteriorChainedNodes) {
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
ChainResult Result = CR_Simple;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
for (SDNode::use_iterator UI = ChainedNode->use_begin(),
E = ChainedNode->use_end(); UI != E; ++UI) {
// Make sure the use is of the chain, not some other value we produce.
if (UI.getUse().getValueType() != MVT::Other) continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
SDNode *User = *UI;
ISelDAG: spot chain cycles involving MachineNodes Previously, the DAGISel function WalkChainUsers was spotting that it had entered already-selected territory by whether a node was a MachineNode (amongst other things). Since it's fairly common practice to insert MachineNodes during ISelLowering, this was not the correct check. Looking around, it seems that other nodes get their NodeId set to -1 upon selection, so this makes sure the same thing happens to all MachineNodes and uses that characteristic to determine whether we should stop looking for a loop during selection. This should fix PR15840. llvm-svn: 191165
2013-09-22 10:21:56 +02:00
if (User->getOpcode() == ISD::HANDLENODE) // Root of the graph.
continue;
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// If we see an already-selected machine node, then we've gone beyond the
// pattern that we're selecting down into the already selected chunk of the
// DAG.
Add a new optimization pass: Stack Coloring, that merges disjoint static allocations (allocas). Allocas are known to be disjoint if they are marked by disjoint lifetime markers (@llvm.lifetime.XXX intrinsics). llvm-svn: 163299
2012-09-06 11:17:37 +02:00
unsigned UserOpcode = User->getOpcode();
ISelDAG: spot chain cycles involving MachineNodes Previously, the DAGISel function WalkChainUsers was spotting that it had entered already-selected territory by whether a node was a MachineNode (amongst other things). Since it's fairly common practice to insert MachineNodes during ISelLowering, this was not the correct check. Looking around, it seems that other nodes get their NodeId set to -1 upon selection, so this makes sure the same thing happens to all MachineNodes and uses that characteristic to determine whether we should stop looking for a loop during selection. This should fix PR15840. llvm-svn: 191165
2013-09-22 10:21:56 +02:00
if (User->isMachineOpcode() ||
UserOpcode == ISD::CopyToReg ||
Add a new optimization pass: Stack Coloring, that merges disjoint static allocations (allocas). Allocas are known to be disjoint if they are marked by disjoint lifetime markers (@llvm.lifetime.XXX intrinsics). llvm-svn: 163299
2012-09-06 11:17:37 +02:00
UserOpcode == ISD::CopyFromReg ||
UserOpcode == ISD::INLINEASM ||
UserOpcode == ISD::EH_LABEL ||
UserOpcode == ISD::LIFETIME_START ||
UserOpcode == ISD::LIFETIME_END) {
Fix some issues in WalkChainUsers dealing with CopyToReg/CopyFromReg/INLINEASM. These are annoying because they have the same opcode before an after isel. Fix this by setting their NodeID to -1 to indicate that they are selected, just like what automatically happens when selecting things that end up being machine nodes. With that done, give IsLegalToFold a new flag that causes it to ignore chains. This lets the HandleMergeInputChains routine be the one place that validates chains after a match is successful, enabling the new hotness in chain processing. This smarter chain processing eliminates the need for "PreprocessRMW" in the X86 and MSP430 backends and enables MSP to start matching it's multiple mem operand instructions more aggressively. I currently #if out the dead code in the X86 backend and MSP backend, I'll remove it for real in a follow-on patch. The testcase changes are: test/CodeGen/X86/sse3.ll: we generate better code test/CodeGen/X86/store_op_load_fold2.ll: PreprocessRMW was miscompiling this before, we now generate correct code Convert it to filecheck while I'm at it. test/CodeGen/MSP430/Inst16mm.ll: Add a testcase for mem/mem folding to make anton happy. :) llvm-svn: 97596
2010-03-02 23:20:06 +01:00
// If their node ID got reset to -1 then they've already been selected.
// Treat them like a MachineOpcode.
if (User->getNodeId() == -1)
continue;
}
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// If we have a TokenFactor, we handle it specially.
if (User->getOpcode() != ISD::TokenFactor) {
// If the node isn't a token factor and isn't part of our pattern, then it
// must be a random chained node in between two nodes we're selecting.
// This happens when we have something like:
// x = load ptr
// call
// y = x+4
// store y -> ptr
// Because we structurally match the load/store as a read/modify/write,
// but the call is chained between them. We cannot fold in this case
// because it would induce a cycle in the graph.
if (!std::count(ChainedNodesInPattern.begin(),
ChainedNodesInPattern.end(), User))
return CR_InducesCycle;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// Otherwise we found a node that is part of our pattern. For example in:
// x = load ptr
// y = x+4
// store y -> ptr
// This would happen when we're scanning down from the load and see the
// store as a user. Record that there is a use of ChainedNode that is
// part of the pattern and keep scanning uses.
Result = CR_LeadsToInteriorNode;
InteriorChainedNodes.push_back(User);
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// If we found a TokenFactor, there are two cases to consider: first if the
// TokenFactor is just hanging "below" the pattern we're matching (i.e. no
// uses of the TF are in our pattern) we just want to ignore it. Second,
// the TokenFactor can be sandwiched in between two chained nodes, like so:
// [Load chain]
// ^
// |
// [Load]
// ^ ^
// | \ DAG's like cheese
// / \ do you?
// / |
// [TokenFactor] [Op]
// ^ ^
// | |
// \ /
// \ /
// [Store]
//
// In this case, the TokenFactor becomes part of our match and we rewrite it
// as a new TokenFactor.
//
// To distinguish these two cases, do a recursive walk down the uses.
[SelectionDAG] Eliminate exponential behavior in WalkChainUsers llvm-svn: 259315
2016-01-31 04:59:34 +01:00
auto MemoizeResult = TokenFactorResult.find(User);
bool Visited = MemoizeResult != TokenFactorResult.end();
// Recursively walk chain users only if the result is not memoized.
if (!Visited) {
auto Res = WalkChainUsers(User, ChainedNodesInPattern, TokenFactorResult,
InteriorChainedNodes);
MemoizeResult = TokenFactorResult.insert(std::make_pair(User, Res)).first;
}
switch (MemoizeResult->second) {
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
case CR_Simple:
// If the uses of the TokenFactor are just already-selected nodes, ignore
// it, it is "below" our pattern.
continue;
case CR_InducesCycle:
// If the uses of the TokenFactor lead to nodes that are not part of our
// pattern that are not selected, folding would turn this into a cycle,
// bail out now.
return CR_InducesCycle;
case CR_LeadsToInteriorNode:
break; // Otherwise, keep processing.
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// Okay, we know we're in the interesting interior case. The TokenFactor
// is now going to be considered part of the pattern so that we rewrite its
// uses (it may have uses that are not part of the pattern) with the
// ultimate chain result of the generated code. We will also add its chain
// inputs as inputs to the ultimate TokenFactor we create.
Result = CR_LeadsToInteriorNode;
[SelectionDAG] Eliminate exponential behavior in WalkChainUsers llvm-svn: 259315
2016-01-31 04:59:34 +01:00
if (!Visited) {
ChainedNodesInPattern.push_back(User);
InteriorChainedNodes.push_back(User);
}
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
return Result;
}
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
/// HandleMergeInputChains - This implements the OPC_EmitMergeInputChains
run HandleMergeInputChains even if we only have one input chain. llvm-svn: 97581
2010-03-02 20:34:59 +01:00
/// operation for when the pattern matched at least one node with a chains. The
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
/// input vector contains a list of all of the chained nodes that we match. We
/// must determine if this is a valid thing to cover (i.e. matching it won't
/// induce cycles in the DAG) and if so, creating a TokenFactor node. that will
/// be used as the input node chain for the generated nodes.
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
static SDValue
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
HandleMergeInputChains(SmallVectorImpl<SDNode*> &ChainNodesMatched,
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
SelectionDAG *CurDAG) {
[SelectionDAG] Eliminate exponential behavior in WalkChainUsers llvm-svn: 259315
2016-01-31 04:59:34 +01:00
// Used for memoization. Without it WalkChainUsers could take exponential
// time to run.
DenseMap<const SDNode *, ChainResult> TokenFactorResult;
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// Walk all of the chained nodes we've matched, recursively scanning down the
// users of the chain result. This adds any TokenFactor nodes that are caught
// in between chained nodes to the chained and interior nodes list.
SmallVector<SDNode*, 3> InteriorChainedNodes;
for (unsigned i = 0, e = ChainNodesMatched.size(); i != e; ++i) {
if (WalkChainUsers(ChainNodesMatched[i], ChainNodesMatched,
[SelectionDAG] Eliminate exponential behavior in WalkChainUsers llvm-svn: 259315
2016-01-31 04:59:34 +01:00
TokenFactorResult,
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
InteriorChainedNodes) == CR_InducesCycle)
return SDValue(); // Would induce a cycle.
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// Okay, we have walked all the matched nodes and collected TokenFactor nodes
// that we are interested in. Form our input TokenFactor node.
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
SmallVector<SDValue, 3> InputChains;
for (unsigned i = 0, e = ChainNodesMatched.size(); i != e; ++i) {
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// Add the input chain of this node to the InputChains list (which will be
// the operands of the generated TokenFactor) if it's not an interior node.
SDNode *N = ChainNodesMatched[i];
if (N->getOpcode() != ISD::TokenFactor) {
if (std::count(InteriorChainedNodes.begin(),InteriorChainedNodes.end(),N))
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// Otherwise, add the input chain.
SDValue InChain = ChainNodesMatched[i]->getOperand(0);
assert(InChain.getValueType() == MVT::Other && "Not a chain");
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
InputChains.push_back(InChain);
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
// If we have a token factor, we want to add all inputs of the token factor
// that are not part of the pattern we're matching.
Convert a bunch of loops to foreach. NFC. This uses the new SDNode::op_values() iterator range committed in r240805. llvm-svn: 240822
2015-06-26 21:37:02 +02:00
for (const SDValue &Op : N->op_values()) {
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
if (!std::count(ChainNodesMatched.begin(), ChainNodesMatched.end(),
Convert a bunch of loops to foreach. NFC. This uses the new SDNode::op_values() iterator range committed in r240805. llvm-svn: 240822
2015-06-26 21:37:02 +02:00
Op.getNode()))
InputChains.push_back(Op);
Rewrite chain handling validation and input TokenFactor handling stuff now that we don't care about emulating the old broken behavior of the old isel. This eliminates the 'CheckChainCompatible' check (along with IsChainCompatible) which did an incorrect and inefficient scan *up* the chain nodes which happened as the pattern was being formed and does the validation at the end in HandleMergeInputChains when it forms a structural pattern. This scans "down" the graph, which means that it is quickly bounded by nodes already selected. This also handles token factors that get "trapped" in the dag. Removing the CheckChainCompatible nodes also shrinks the generated tables by about 6K for X86 (down to 83K). There are two pieces remaining before I can nuke PreprocessRMW: 1. I xfailed a test because we're now producing worse code in a case that has nothing to do with the change: it turns out that our use of MorphNodeTo will leave dead nodes in the graph which (depending on how the graph is walked) end up causing bogus uses of chains and blocking matches. This is really bad for other reasons, so I'll fix this in a follow-up patch. 2. CheckFoldableChainNode needs to be improved to handle the TF. llvm-svn: 97539
2010-03-02 03:22:10 +01:00
}
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
if (InputChains.size() == 1)
return InputChains[0];
Track IR ordering of SelectionDAG nodes 2/4. Change SelectionDAG::getXXXNode() interfaces as well as call sites of these functions to pass in SDLoc instead of DebugLoc. llvm-svn: 182703
2013-05-25 04:42:55 +02:00
return CurDAG->getNode(ISD::TokenFactor, SDLoc(ChainNodesMatched[0]),
Convert SelectionDAG::getNode methods to use ArrayRef<SDValue>. llvm-svn: 207327
2014-04-26 20:35:24 +02:00
MVT::Other, InputChains);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
}
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
/// MorphNode - Handle morphing a node in place for the selector.
SDNode *SelectionDAGISel::
MorphNode(SDNode *Node, unsigned TargetOpc, SDVTList VTList,
Convert SelectionDAGISel::MorphNode to use ArrayRef. llvm-svn: 207379
2014-04-27 21:21:20 +02:00
ArrayRef<SDValue> Ops, unsigned EmitNodeInfo) {
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
// It is possible we're using MorphNodeTo to replace a node with no
// normal results with one that has a normal result (or we could be
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// adding a chain) and the input could have glue and chains as well.
finally remove the immAllOnesV_bc/immAllZerosV_bc patterns and those derived from them. These are obnoxious because they were written as: PatLeaf<(bitconvert). Not having an argument was foiling adding better type checking for operand count matching up with what was required (in this case, bitconvert always requires an operand!) llvm-svn: 99759
2010-03-28 10:43:23 +02:00
// In this case we need to shift the operands down.
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
// FIXME: This is a horrible hack and broken in obscure cases, no worse
improve -debug-only=isel comments for cases when we don't enter a scope due to obviously false predicate. llvm-svn: 99723
2010-03-27 19:54:50 +01:00
// than the old isel though.
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
int OldGlueResultNo = -1, OldChainResultNo = -1;
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
unsigned NTMNumResults = Node->getNumValues();
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
if (Node->getValueType(NTMNumResults-1) == MVT::Glue) {
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
OldGlueResultNo = NTMNumResults-1;
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
if (NTMNumResults != 1 &&
Node->getValueType(NTMNumResults-2) == MVT::Other)
OldChainResultNo = NTMNumResults-2;
} else if (Node->getValueType(NTMNumResults-1) == MVT::Other)
OldChainResultNo = NTMNumResults-1;
I was confused about this, it turns out that MorphNodeTo *does* delete ex-operands that become dead. llvm-svn: 97559
2010-03-02 08:14:49 +01:00
// Call the underlying SelectionDAG routine to do the transmogrification. Note
// that this deletes operands of the old node that become dead.
Convert SelectionDAGISel::MorphNode to use ArrayRef. llvm-svn: 207379
2014-04-27 21:21:20 +02:00
SDNode *Res = CurDAG->MorphNodeTo(Node, ~TargetOpc, VTList, Ops);
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
// MorphNodeTo can operate in two ways: if an existing node with the
// specified operands exists, it can just return it. Otherwise, it
// updates the node in place to have the requested operands.
if (Res == Node) {
// If we updated the node in place, reset the node ID. To the isel,
// this should be just like a newly allocated machine node.
Res->setNodeId(-1);
}
unsigned ResNumResults = Res->getNumValues();
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// Move the glue if needed.
if ((EmitNodeInfo & OPFL_GlueOutput) && OldGlueResultNo != -1 &&
(unsigned)OldGlueResultNo != ResNumResults-1)
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
CurDAG->ReplaceAllUsesOfValueWith(SDValue(Node, OldGlueResultNo),
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
SDValue(Res, ResNumResults-1));
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
if ((EmitNodeInfo & OPFL_GlueOutput) != 0)
Proper indentation. llvm-svn: 107581
2010-07-04 10:58:43 +02:00
--ResNumResults;
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
// Move the chain reference if needed.
if ((EmitNodeInfo & OPFL_Chain) && OldChainResultNo != -1 &&
(unsigned)OldChainResultNo != ResNumResults-1)
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
CurDAG->ReplaceAllUsesOfValueWith(SDValue(Node, OldChainResultNo),
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
SDValue(Res, ResNumResults-1));
// Otherwise, no replacement happened because the node already exists. Replace
// Uses of the old node with the new one.
SDAG: Drop a redundant replace and move the dead node removal closer. NFC llvm-svn: 271429
2016-06-01 22:55:26 +02:00
if (Res != Node) {
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
CurDAG->ReplaceAllUsesWith(Node, Res);
SDAG: Drop a redundant replace and move the dead node removal closer. NFC llvm-svn: 271429
2016-06-01 22:55:26 +02:00
CurDAG->RemoveDeadNode(Node);
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
factor node morphing out to its own helper method. llvm-svn: 97558
2010-03-02 07:55:04 +01:00
return Res;
}
Fix bad comment. No functional change. llvm-svn: 164000
2012-09-16 18:48:25 +02:00
/// CheckSame - Implements OP_CheckSame.
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
CheckSame(const unsigned char *MatcherTable, unsigned &MatcherIndex,
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
SDValue N,
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
const SmallVectorImpl<std::pair<SDValue, SDNode*>> &RecordedNodes) {
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
// Accept if it is exactly the same as a previously recorded node.
unsigned RecNo = MatcherTable[MatcherIndex++];
assert(RecNo < RecordedNodes.size() && "Invalid CheckSame");
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
return N == RecordedNodes[RecNo].first;
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Add OPC_CheckChildSame0-3 to the DAG isel matcher. This replaces sequences of MoveChild, CheckSame, MoveParent. Saves 846 bytes from the X86 DAG isel matcher, ~300 from ARM, ~840 from Hexagon. llvm-svn: 192026
2013-10-05 07:38:16 +02:00
/// CheckChildSame - Implements OP_CheckChildXSame.
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
Add OPC_CheckChildSame0-3 to the DAG isel matcher. This replaces sequences of MoveChild, CheckSame, MoveParent. Saves 846 bytes from the X86 DAG isel matcher, ~300 from ARM, ~840 from Hexagon. llvm-svn: 192026
2013-10-05 07:38:16 +02:00
CheckChildSame(const unsigned char *MatcherTable, unsigned &MatcherIndex,
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
SDValue N,
const SmallVectorImpl<std::pair<SDValue, SDNode*>> &RecordedNodes,
unsigned ChildNo) {
Add OPC_CheckChildSame0-3 to the DAG isel matcher. This replaces sequences of MoveChild, CheckSame, MoveParent. Saves 846 bytes from the X86 DAG isel matcher, ~300 from ARM, ~840 from Hexagon. llvm-svn: 192026
2013-10-05 07:38:16 +02:00
if (ChildNo >= N.getNumOperands())
return false; // Match fails if out of range child #.
return ::CheckSame(MatcherTable, MatcherIndex, N.getOperand(ChildNo),
RecordedNodes);
}
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
/// CheckPatternPredicate - Implements OP_CheckPatternPredicate.
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
CheckPatternPredicate(const unsigned char *MatcherTable, unsigned &MatcherIndex,
Add some constantness. No functionality change. llvm-svn: 155859
2012-05-01 01:41:30 +02:00
const SelectionDAGISel &SDISel) {
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
return SDISel.CheckPatternPredicate(MatcherTable[MatcherIndex++]);
}
/// CheckNodePredicate - Implements OP_CheckNodePredicate.
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
CheckNodePredicate(const unsigned char *MatcherTable, unsigned &MatcherIndex,
Add some constantness. No functionality change. llvm-svn: 155859
2012-05-01 01:41:30 +02:00
const SelectionDAGISel &SDISel, SDNode *N) {
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
return SDISel.CheckNodePredicate(N, MatcherTable[MatcherIndex++]);
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
CheckOpcode(const unsigned char *MatcherTable, unsigned &MatcherIndex,
SDNode *N) {
Change tblgen to emit FOOISD opcode names as two bytes instead of one byte. This is important because we're running up to too many opcodes to fit in a byte and it is aggrevated by FIRST_TARGET_MEMORY_OPCODE making the numbering sparse. This just bites the bullet and bloats out the table. In practice, this increases the size of the x86 isel table from 74.5K to 76K. I think we'll cope :) This fixes rdar://7791648 llvm-svn: 99494
2010-03-25 07:33:05 +01:00
uint16_t Opc = MatcherTable[MatcherIndex++];
Opc |= (unsigned short)MatcherTable[MatcherIndex++] << 8;
return N->getOpcode() == Opc;
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
CheckType(const unsigned char *MatcherTable, unsigned &MatcherIndex, SDValue N,
const TargetLowering *TLI, const DataLayout &DL) {
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
MVT::SimpleValueType VT = (MVT::SimpleValueType)MatcherTable[MatcherIndex++];
if (N.getValueType() == VT) return true;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
// Handle the case when VT is iPTR.
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
return VT == MVT::iPTR && N.getValueType() == TLI->getPointerTy(DL);
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
CheckChildType(const unsigned char *MatcherTable, unsigned &MatcherIndex,
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
SDValue N, const TargetLowering *TLI, const DataLayout &DL,
unsigned ChildNo) {
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
if (ChildNo >= N.getNumOperands())
return false; // Match fails if out of range child #.
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
return ::CheckType(MatcherTable, MatcherIndex, N.getOperand(ChildNo), TLI,
DL);
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
CheckCondCode(const unsigned char *MatcherTable, unsigned &MatcherIndex,
SDValue N) {
return cast<CondCodeSDNode>(N)->get() ==
(ISD::CondCode)MatcherTable[MatcherIndex++];
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
CheckValueType(const unsigned char *MatcherTable, unsigned &MatcherIndex,
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
SDValue N, const TargetLowering *TLI, const DataLayout &DL) {
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
MVT::SimpleValueType VT = (MVT::SimpleValueType)MatcherTable[MatcherIndex++];
if (cast<VTSDNode>(N)->getVT() == VT)
return true;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
// Handle the case when VT is iPTR.
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
return VT == MVT::iPTR && cast<VTSDNode>(N)->getVT() == TLI->getPointerTy(DL);
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
CheckInteger(const unsigned char *MatcherTable, unsigned &MatcherIndex,
SDValue N) {
int64_t Val = MatcherTable[MatcherIndex++];
if (Val & 128)
Val = GetVBR(Val, MatcherTable, MatcherIndex);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
ConstantSDNode *C = dyn_cast<ConstantSDNode>(N);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
return C && C->getSExtValue() == Val;
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
Add CheckChildInteger to ISelMatcher operations. Removes nearly 2000 bytes from X86 matcher table. llvm-svn: 200821
2014-02-05 06:44:28 +01:00
CheckChildInteger(const unsigned char *MatcherTable, unsigned &MatcherIndex,
SDValue N, unsigned ChildNo) {
if (ChildNo >= N.getNumOperands())
return false; // Match fails if out of range child #.
return ::CheckInteger(MatcherTable, MatcherIndex, N.getOperand(ChildNo));
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
CheckAndImm(const unsigned char *MatcherTable, unsigned &MatcherIndex,
Add some constantness. No functionality change. llvm-svn: 155859
2012-05-01 01:41:30 +02:00
SDValue N, const SelectionDAGISel &SDISel) {
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
int64_t Val = MatcherTable[MatcherIndex++];
if (Val & 128)
Val = GetVBR(Val, MatcherTable, MatcherIndex);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
if (N->getOpcode() != ISD::AND) return false;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
ConstantSDNode *C = dyn_cast<ConstantSDNode>(N->getOperand(1));
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
return C && SDISel.CheckAndMask(N.getOperand(0), C, Val);
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
}
Add an explicit 'inline' specifier to these static functions. GCC is warning on them having always_inline attribute for reasons I don't fully understand -- static functions are just as inlinable as inline functions in terms of linkage. llvm-svn: 247334
2015-09-10 22:34:57 +02:00
LLVM_ATTRIBUTE_ALWAYS_INLINE static inline bool
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
CheckOrImm(const unsigned char *MatcherTable, unsigned &MatcherIndex,
Add some constantness. No functionality change. llvm-svn: 155859
2012-05-01 01:41:30 +02:00
SDValue N, const SelectionDAGISel &SDISel) {
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
int64_t Val = MatcherTable[MatcherIndex++];
if (Val & 128)
Val = GetVBR(Val, MatcherTable, MatcherIndex);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
if (N->getOpcode() != ISD::OR) return false;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
ConstantSDNode *C = dyn_cast<ConstantSDNode>(N->getOperand(1));
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
return C && SDISel.CheckOrMask(N.getOperand(0), C, Val);
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
}
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
/// IsPredicateKnownToFail - If we know how and can do so without pushing a
/// scope, evaluate the current node. If the current predicate is known to
/// fail, set Result=true and return anything. If the current predicate is
/// known to pass, set Result=false and return the MatcherIndex to continue
/// with. If the current predicate is unknown, set Result=false and return the
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
/// MatcherIndex to continue with.
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
static unsigned IsPredicateKnownToFail(const unsigned char *Table,
unsigned Index, SDValue N,
Add some constantness. No functionality change. llvm-svn: 155859
2012-05-01 01:41:30 +02:00
bool &Result,
const SelectionDAGISel &SDISel,
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
SmallVectorImpl<std::pair<SDValue, SDNode*>> &RecordedNodes) {
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
switch (Table[Index++]) {
default:
Result = false;
return Index-1; // Could not evaluate this predicate.
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
case SelectionDAGISel::OPC_CheckSame:
Result = !::CheckSame(Table, Index, N, RecordedNodes);
return Index;
Add OPC_CheckChildSame0-3 to the DAG isel matcher. This replaces sequences of MoveChild, CheckSame, MoveParent. Saves 846 bytes from the X86 DAG isel matcher, ~300 from ARM, ~840 from Hexagon. llvm-svn: 192026
2013-10-05 07:38:16 +02:00
case SelectionDAGISel::OPC_CheckChild0Same:
case SelectionDAGISel::OPC_CheckChild1Same:
case SelectionDAGISel::OPC_CheckChild2Same:
case SelectionDAGISel::OPC_CheckChild3Same:
Result = !::CheckChildSame(Table, Index, N, RecordedNodes,
Table[Index-1] - SelectionDAGISel::OPC_CheckChild0Same);
return Index;
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
case SelectionDAGISel::OPC_CheckPatternPredicate:
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
Result = !::CheckPatternPredicate(Table, Index, SDISel);
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
return Index;
case SelectionDAGISel::OPC_CheckPredicate:
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
Result = !::CheckNodePredicate(Table, Index, SDISel, N.getNode());
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
return Index;
case SelectionDAGISel::OPC_CheckOpcode:
Result = !::CheckOpcode(Table, Index, N.getNode());
return Index;
case SelectionDAGISel::OPC_CheckType:
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
Result = !::CheckType(Table, Index, N, SDISel.TLI,
SDISel.CurDAG->getDataLayout());
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
return Index;
case SelectionDAGISel::OPC_CheckChild0Type:
case SelectionDAGISel::OPC_CheckChild1Type:
case SelectionDAGISel::OPC_CheckChild2Type:
case SelectionDAGISel::OPC_CheckChild3Type:
case SelectionDAGISel::OPC_CheckChild4Type:
case SelectionDAGISel::OPC_CheckChild5Type:
case SelectionDAGISel::OPC_CheckChild6Type:
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
case SelectionDAGISel::OPC_CheckChild7Type:
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
Result = !::CheckChildType(
Table, Index, N, SDISel.TLI, SDISel.CurDAG->getDataLayout(),
Table[Index - 1] - SelectionDAGISel::OPC_CheckChild0Type);
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
return Index;
case SelectionDAGISel::OPC_CheckCondCode:
Result = !::CheckCondCode(Table, Index, N);
return Index;
case SelectionDAGISel::OPC_CheckValueType:
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
Result = !::CheckValueType(Table, Index, N, SDISel.TLI,
SDISel.CurDAG->getDataLayout());
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
return Index;
case SelectionDAGISel::OPC_CheckInteger:
Result = !::CheckInteger(Table, Index, N);
return Index;
Add CheckChildInteger to ISelMatcher operations. Removes nearly 2000 bytes from X86 matcher table. llvm-svn: 200821
2014-02-05 06:44:28 +01:00
case SelectionDAGISel::OPC_CheckChild0Integer:
case SelectionDAGISel::OPC_CheckChild1Integer:
case SelectionDAGISel::OPC_CheckChild2Integer:
case SelectionDAGISel::OPC_CheckChild3Integer:
case SelectionDAGISel::OPC_CheckChild4Integer:
Result = !::CheckChildInteger(Table, Index, N,
Table[Index-1] - SelectionDAGISel::OPC_CheckChild0Integer);
return Index;
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
case SelectionDAGISel::OPC_CheckAndImm:
Result = !::CheckAndImm(Table, Index, N, SDISel);
return Index;
case SelectionDAGISel::OPC_CheckOrImm:
Result = !::CheckOrImm(Table, Index, N, SDISel);
return Index;
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
}
}
Fix a bunch of namespace polution. llvm-svn: 101376
2010-04-15 19:08:50 +02:00
namespace {
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
struct MatchScope {
/// FailIndex - If this match fails, this is the index to continue with.
unsigned FailIndex;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
/// NodeStack - The node stack when the scope was formed.
SmallVector<SDValue, 4> NodeStack;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
/// NumRecordedNodes - The number of recorded nodes when the scope was formed.
unsigned NumRecordedNodes;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
/// NumMatchedMemRefs - The number of matched memref entries.
unsigned NumMatchedMemRefs;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
/// InputChain/InputGlue - The current chain/glue
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
SDValue InputChain, InputGlue;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
/// HasChainNodesMatched - True if the ChainNodesMatched list is non-empty.
SDAG: Remove OPC_MarkGlueResults and associated logic. NFC This opcode never happens in practice, and yet the logic we have in place to handle it would be undefined behaviour if we ever executed it. Remove it rather than trying to refactor code that's never reached. llvm-svn: 268692
2016-05-06 00:37:45 +02:00
bool HasChainNodesMatched;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
};
[ISel] Keep matching state consistent when folding during X86 address match In the X86 backend, matching an address is initiated by the 'addr' complex pattern and its friends. During this process we may reassociate and-of-shift into shift-of-and (FoldMaskedShiftToScaledMask) to allow folding of the shift into the scale of the address. However as demonstrated by the testcase, this can trigger CSE of not only the shift and the AND which the code is prepared for but also the underlying load node. In the testcase this node is sitting in the RecordedNode and MatchScope data structures of the matcher and becomes a deleted node upon CSE. Returning from the complex pattern function, we try to access it again hitting an assert because the node is no longer a load even though this was checked before. Now obviously changing the DAG this late is bending the rules but I think it makes sense somewhat. Outside of addresses we prefer and-of-shift because it may lead to smaller immediates (FoldMaskAndShiftToScale is an even better example because it create a non-canonical node). We currently don't recognize addresses during DAGCombiner where arguably this canonicalization should be performed. On the other hand, having this in the matcher allows us to cover all the cases where an address can be used in an instruction. I've also talked a little bit to Dan Gohman on llvm-dev who added the RAUW for the new shift node in FoldMaskedShiftToScaledMask. This RAUW is responsible for initiating the recursive CSE on users (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-September/076903.html) but it is not strictly necessary since the shift is hooked into the visited user. Of course it's safer to keep the DAG consistent at all times (e.g. for accurate number of uses, etc.). So rather than changing the fundamentals, I've decided to continue along the previous patches and detect the CSE. This patch installs a very targeted DAGUpdateListener for the duration of a complex-pattern match and updates the matching state accordingly. (Previous patches used HandleSDNode to detect the CSE but that's not practical here). The listener is only installed on X86. I tested that there is no measurable overhead due to this while running through the spec2k BC files with llc. The only thing we pay for is the creation of the listener. The callback never ever triggers in spec2k since this is a corner case. Fixes rdar://problem/18206171 llvm-svn: 219009
2014-10-03 22:00:34 +02:00
/// \\brief A DAG update listener to keep the matching state
/// (i.e. RecordedNodes and MatchScope) uptodate if the target is allowed to
/// change the DAG while matching. X86 addressing mode matcher is an example
/// for this.
class MatchStateUpdater : public SelectionDAG::DAGUpdateListener
{
[SelectionDAG] Fix for PR30775: Assertion `NodeToMatch->getOpcode() != ISD::DELETED_NODE && "NodeToMatch was removed partway through selection"' failed. NodeToMatch can be modified during matching, but code does not handle this situation. Differential Revision: https://reviews.llvm.org/D29292 llvm-svn: 294003
2017-02-03 13:28:40 +01:00
SDNode **NodeToMatch;
SmallVectorImpl<std::pair<SDValue, SDNode *>> &RecordedNodes;
SmallVectorImpl<MatchScope> &MatchScopes;
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
[ISel] Keep matching state consistent when folding during X86 address match In the X86 backend, matching an address is initiated by the 'addr' complex pattern and its friends. During this process we may reassociate and-of-shift into shift-of-and (FoldMaskedShiftToScaledMask) to allow folding of the shift into the scale of the address. However as demonstrated by the testcase, this can trigger CSE of not only the shift and the AND which the code is prepared for but also the underlying load node. In the testcase this node is sitting in the RecordedNode and MatchScope data structures of the matcher and becomes a deleted node upon CSE. Returning from the complex pattern function, we try to access it again hitting an assert because the node is no longer a load even though this was checked before. Now obviously changing the DAG this late is bending the rules but I think it makes sense somewhat. Outside of addresses we prefer and-of-shift because it may lead to smaller immediates (FoldMaskAndShiftToScale is an even better example because it create a non-canonical node). We currently don't recognize addresses during DAGCombiner where arguably this canonicalization should be performed. On the other hand, having this in the matcher allows us to cover all the cases where an address can be used in an instruction. I've also talked a little bit to Dan Gohman on llvm-dev who added the RAUW for the new shift node in FoldMaskedShiftToScaledMask. This RAUW is responsible for initiating the recursive CSE on users (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-September/076903.html) but it is not strictly necessary since the shift is hooked into the visited user. Of course it's safer to keep the DAG consistent at all times (e.g. for accurate number of uses, etc.). So rather than changing the fundamentals, I've decided to continue along the previous patches and detect the CSE. This patch installs a very targeted DAGUpdateListener for the duration of a complex-pattern match and updates the matching state accordingly. (Previous patches used HandleSDNode to detect the CSE but that's not practical here). The listener is only installed on X86. I tested that there is no measurable overhead due to this while running through the spec2k BC files with llc. The only thing we pay for is the creation of the listener. The callback never ever triggers in spec2k since this is a corner case. Fixes rdar://problem/18206171 llvm-svn: 219009
2014-10-03 22:00:34 +02:00
public:
[SelectionDAG] Fix for PR30775: Assertion `NodeToMatch->getOpcode() != ISD::DELETED_NODE && "NodeToMatch was removed partway through selection"' failed. NodeToMatch can be modified during matching, but code does not handle this situation. Differential Revision: https://reviews.llvm.org/D29292 llvm-svn: 294003
2017-02-03 13:28:40 +01:00
MatchStateUpdater(SelectionDAG &DAG, SDNode **NodeToMatch,
SmallVectorImpl<std::pair<SDValue, SDNode *>> &RN,
SmallVectorImpl<MatchScope> &MS)
: SelectionDAG::DAGUpdateListener(DAG), NodeToMatch(NodeToMatch),
RecordedNodes(RN), MatchScopes(MS) {}
[ISel] Keep matching state consistent when folding during X86 address match In the X86 backend, matching an address is initiated by the 'addr' complex pattern and its friends. During this process we may reassociate and-of-shift into shift-of-and (FoldMaskedShiftToScaledMask) to allow folding of the shift into the scale of the address. However as demonstrated by the testcase, this can trigger CSE of not only the shift and the AND which the code is prepared for but also the underlying load node. In the testcase this node is sitting in the RecordedNode and MatchScope data structures of the matcher and becomes a deleted node upon CSE. Returning from the complex pattern function, we try to access it again hitting an assert because the node is no longer a load even though this was checked before. Now obviously changing the DAG this late is bending the rules but I think it makes sense somewhat. Outside of addresses we prefer and-of-shift because it may lead to smaller immediates (FoldMaskAndShiftToScale is an even better example because it create a non-canonical node). We currently don't recognize addresses during DAGCombiner where arguably this canonicalization should be performed. On the other hand, having this in the matcher allows us to cover all the cases where an address can be used in an instruction. I've also talked a little bit to Dan Gohman on llvm-dev who added the RAUW for the new shift node in FoldMaskedShiftToScaledMask. This RAUW is responsible for initiating the recursive CSE on users (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-September/076903.html) but it is not strictly necessary since the shift is hooked into the visited user. Of course it's safer to keep the DAG consistent at all times (e.g. for accurate number of uses, etc.). So rather than changing the fundamentals, I've decided to continue along the previous patches and detect the CSE. This patch installs a very targeted DAGUpdateListener for the duration of a complex-pattern match and updates the matching state accordingly. (Previous patches used HandleSDNode to detect the CSE but that's not practical here). The listener is only installed on X86. I tested that there is no measurable overhead due to this while running through the spec2k BC files with llc. The only thing we pay for is the creation of the listener. The callback never ever triggers in spec2k since this is a corner case. Fixes rdar://problem/18206171 llvm-svn: 219009
2014-10-03 22:00:34 +02:00
Use 'override/final' instead of 'virtual' for overridden methods The patch is generated using clang-tidy misc-use-override check. This command was used: tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py \ -checks='-*,misc-use-override' -header-filter='llvm|clang' \ -j=32 -fix -format http://reviews.llvm.org/D8925 llvm-svn: 234679
2015-04-11 04:11:45 +02:00
void NodeDeleted(SDNode *N, SDNode *E) override {
[ISel] Keep matching state consistent when folding during X86 address match In the X86 backend, matching an address is initiated by the 'addr' complex pattern and its friends. During this process we may reassociate and-of-shift into shift-of-and (FoldMaskedShiftToScaledMask) to allow folding of the shift into the scale of the address. However as demonstrated by the testcase, this can trigger CSE of not only the shift and the AND which the code is prepared for but also the underlying load node. In the testcase this node is sitting in the RecordedNode and MatchScope data structures of the matcher and becomes a deleted node upon CSE. Returning from the complex pattern function, we try to access it again hitting an assert because the node is no longer a load even though this was checked before. Now obviously changing the DAG this late is bending the rules but I think it makes sense somewhat. Outside of addresses we prefer and-of-shift because it may lead to smaller immediates (FoldMaskAndShiftToScale is an even better example because it create a non-canonical node). We currently don't recognize addresses during DAGCombiner where arguably this canonicalization should be performed. On the other hand, having this in the matcher allows us to cover all the cases where an address can be used in an instruction. I've also talked a little bit to Dan Gohman on llvm-dev who added the RAUW for the new shift node in FoldMaskedShiftToScaledMask. This RAUW is responsible for initiating the recursive CSE on users (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-September/076903.html) but it is not strictly necessary since the shift is hooked into the visited user. Of course it's safer to keep the DAG consistent at all times (e.g. for accurate number of uses, etc.). So rather than changing the fundamentals, I've decided to continue along the previous patches and detect the CSE. This patch installs a very targeted DAGUpdateListener for the duration of a complex-pattern match and updates the matching state accordingly. (Previous patches used HandleSDNode to detect the CSE but that's not practical here). The listener is only installed on X86. I tested that there is no measurable overhead due to this while running through the spec2k BC files with llc. The only thing we pay for is the creation of the listener. The callback never ever triggers in spec2k since this is a corner case. Fixes rdar://problem/18206171 llvm-svn: 219009
2014-10-03 22:00:34 +02:00
// Some early-returns here to avoid the search if we deleted the node or
// if the update comes from MorphNodeTo (MorphNodeTo is the last thing we
// do, so it's unnecessary to update matching state at that point).
// Neither of these can occur currently because we only install this
// update listener during matching a complex patterns.
if (!E || E->isMachineOpcode())
return;
[SelectionDAG] Fix for PR30775: Assertion `NodeToMatch->getOpcode() != ISD::DELETED_NODE && "NodeToMatch was removed partway through selection"' failed. NodeToMatch can be modified during matching, but code does not handle this situation. Differential Revision: https://reviews.llvm.org/D29292 llvm-svn: 294003
2017-02-03 13:28:40 +01:00
// Check if NodeToMatch was updated.
if (N == *NodeToMatch)
*NodeToMatch = E;
[ISel] Keep matching state consistent when folding during X86 address match In the X86 backend, matching an address is initiated by the 'addr' complex pattern and its friends. During this process we may reassociate and-of-shift into shift-of-and (FoldMaskedShiftToScaledMask) to allow folding of the shift into the scale of the address. However as demonstrated by the testcase, this can trigger CSE of not only the shift and the AND which the code is prepared for but also the underlying load node. In the testcase this node is sitting in the RecordedNode and MatchScope data structures of the matcher and becomes a deleted node upon CSE. Returning from the complex pattern function, we try to access it again hitting an assert because the node is no longer a load even though this was checked before. Now obviously changing the DAG this late is bending the rules but I think it makes sense somewhat. Outside of addresses we prefer and-of-shift because it may lead to smaller immediates (FoldMaskAndShiftToScale is an even better example because it create a non-canonical node). We currently don't recognize addresses during DAGCombiner where arguably this canonicalization should be performed. On the other hand, having this in the matcher allows us to cover all the cases where an address can be used in an instruction. I've also talked a little bit to Dan Gohman on llvm-dev who added the RAUW for the new shift node in FoldMaskedShiftToScaledMask. This RAUW is responsible for initiating the recursive CSE on users (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-September/076903.html) but it is not strictly necessary since the shift is hooked into the visited user. Of course it's safer to keep the DAG consistent at all times (e.g. for accurate number of uses, etc.). So rather than changing the fundamentals, I've decided to continue along the previous patches and detect the CSE. This patch installs a very targeted DAGUpdateListener for the duration of a complex-pattern match and updates the matching state accordingly. (Previous patches used HandleSDNode to detect the CSE but that's not practical here). The listener is only installed on X86. I tested that there is no measurable overhead due to this while running through the spec2k BC files with llc. The only thing we pay for is the creation of the listener. The callback never ever triggers in spec2k since this is a corner case. Fixes rdar://problem/18206171 llvm-svn: 219009
2014-10-03 22:00:34 +02:00
// Performing linear search here does not matter because we almost never
// run this code. You'd have to have a CSE during complex pattern
// matching.
for (auto &I : RecordedNodes)
if (I.first.getNode() == N)
I.first.setNode(E);
for (auto &I : MatchScopes)
for (auto &J : I.NodeStack)
if (J.getNode() == N)
J.setNode(E);
}
};
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
Fix Clang-tidy readability-redundant-control-flow warnings; other minor fixes. Differential revision: http://reviews.llvm.org/D16793 llvm-svn: 259539
2016-02-02 19:20:45 +01:00
} // end anonymous namespace
Fix a bunch of namespace polution. llvm-svn: 101376
2010-04-15 19:08:50 +02:00
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
void SelectionDAGISel::SelectCodeCommon(SDNode *NodeToMatch,
const unsigned char *MatcherTable,
unsigned TableSize) {
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// FIXME: Should these even be selected? Handle these cases in the caller?
switch (NodeToMatch->getOpcode()) {
default:
break;
case ISD::EntryToken: // These nodes remain the same.
case ISD::BasicBlock:
case ISD::Register:
Add a RegisterMaskSDNode class. This SelectionDAG node will be attached to call nodes by LowerCall(), and eventually becomes a MO_RegisterMask MachineOperand on the MachineInstr representing the call instruction. LowerCall() will attach a register mask that depends on the calling convention. llvm-svn: 148436
2012-01-19 00:52:12 +01:00
case ISD::RegisterMask:
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case ISD::HANDLENODE:
Three changes: 1. Introduce some enums and accessors in the InlineAsm class that eliminate a ton of magic numbers when handling inline asm SDNode. 2. Add a new MDNodeSDNode selection dag node type that holds a MDNode (shocking!) 3. Add a new argument to ISD::INLINEASM nodes that hold !srcloc metadata, propagating it to the instruction emitter, which drops it. No functionality change. llvm-svn: 100605
2010-04-07 07:20:54 +02:00
case ISD::MDNODE_SDNODE:
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case ISD::TargetConstant:
case ISD::TargetConstantFP:
case ISD::TargetConstantPool:
case ISD::TargetFrameIndex:
case ISD::TargetExternalSymbol:
Avoid a Symbol -> Name -> Symbol conversion. Before this we were producing a TargetExternalSymbol from a MCSymbol. That meant extracting the symbol name and fetching the symbol again down the pipeline. This patch adds a DAG.getMCSymbol that lets the MCSymbol pass unchanged on the DAG. Doing so removes the need for MO_NOPREFIX and fixes the root cause of pr23900, allowing r240130 to be committed again. llvm-svn: 240300
2015-06-22 19:46:53 +02:00
case ISD::MCSymbol:
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case ISD::TargetBlockAddress:
case ISD::TargetJumpTable:
case ISD::TargetGlobalTLSAddress:
case ISD::TargetGlobalAddress:
case ISD::TokenFactor:
case ISD::CopyFromReg:
case ISD::CopyToReg:
change the LabelSDNode to be EHLabelSDNode and make it hold an MCSymbol. Make the EH_LABEL MachineInstr hold its label with an MCSymbol instead of ID. Fix a bug in MMI.cpp which would return labels named "Label4" instead of "label4". llvm-svn: 98463
2010-03-14 03:33:54 +01:00
case ISD::EH_LABEL:
Add a new optimization pass: Stack Coloring, that merges disjoint static allocations (allocas). Allocas are known to be disjoint if they are marked by disjoint lifetime markers (@llvm.lifetime.XXX intrinsics). llvm-svn: 163299
2012-09-06 11:17:37 +02:00
case ISD::LIFETIME_START:
case ISD::LIFETIME_END:
Fix some issues in WalkChainUsers dealing with CopyToReg/CopyFromReg/INLINEASM. These are annoying because they have the same opcode before an after isel. Fix this by setting their NodeID to -1 to indicate that they are selected, just like what automatically happens when selecting things that end up being machine nodes. With that done, give IsLegalToFold a new flag that causes it to ignore chains. This lets the HandleMergeInputChains routine be the one place that validates chains after a match is successful, enabling the new hotness in chain processing. This smarter chain processing eliminates the need for "PreprocessRMW" in the X86 and MSP430 backends and enables MSP to start matching it's multiple mem operand instructions more aggressively. I currently #if out the dead code in the X86 backend and MSP backend, I'll remove it for real in a follow-on patch. The testcase changes are: test/CodeGen/X86/sse3.ll: we generate better code test/CodeGen/X86/store_op_load_fold2.ll: PreprocessRMW was miscompiling this before, we now generate correct code Convert it to filecheck while I'm at it. test/CodeGen/MSP430/Inst16mm.ll: Add a testcase for mem/mem folding to make anton happy. :) llvm-svn: 97596
2010-03-02 23:20:06 +01:00
NodeToMatch->setNodeId(-1); // Mark selected.
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
return;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case ISD::AssertSext:
case ISD::AssertZext:
CurDAG->ReplaceAllUsesOfValueWith(SDValue(NodeToMatch, 0),
NodeToMatch->getOperand(0));
SDAG: Don't leave dangling dead nodes after SelectCodeCommon Relying on the caller to clean up after we've replaced all uses of a node won't work when we've migrated to the `void Select(...)` API. llvm-svn: 268774
2016-05-06 20:42:16 +02:00
CurDAG->RemoveDeadNode(NodeToMatch);
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
return;
case ISD::INLINEASM:
Select_INLINEASM(NodeToMatch);
return;
case ISD::READ_REGISTER:
Select_READ_REGISTER(NodeToMatch);
return;
case ISD::WRITE_REGISTER:
Select_WRITE_REGISTER(NodeToMatch);
return;
case ISD::UNDEF:
Select_UNDEF(NodeToMatch);
return;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
assert(!NodeToMatch->isMachineOpcode() && "Node already selected!");
// Set up the node stack with NodeToMatch as the only node on the stack.
SmallVector<SDValue, 8> NodeStack;
SDValue N = SDValue(NodeToMatch, 0);
NodeStack.push_back(N);
// MatchScopes - Scopes used when matching, if a match failure happens, this
// indicates where to continue checking.
SmallVector<MatchScope, 8> MatchScopes;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// RecordedNodes - This is the set of nodes that have been recorded by the
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
// state machine. The second value is the parent of the node, or null if the
// root is recorded.
SmallVector<std::pair<SDValue, SDNode*>, 8> RecordedNodes;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// MatchedMemRefs - This is the set of MemRef's we've seen in the input
// pattern.
SmallVector<MachineMemOperand*, 2> MatchedMemRefs;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// These are the current input chain and glue for use when generating nodes.
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// Various Emit operations change these. For example, emitting a copytoreg
// uses and updates these.
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
SDValue InputChain, InputGlue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// ChainNodesMatched - If a pattern matches nodes that have input/output
// chains, the OPC_EmitMergeInputChains operation is emitted which indicates
// which ones they are. The result is captured into this list so that we can
// update the chain results when the pattern is complete.
SmallVector<SDNode*, 3> ChainNodesMatched;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << "ISEL: Starting pattern match on root node: ";
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
NodeToMatch->dump(CurDAG);
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
dbgs() << '\n');
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Accelerate isel dispatch for tables that start with a top-level OPC_SwitchOpcode to use a table lookup instead of having to go through the interpreter for this. llvm-svn: 97469
2010-03-01 19:47:11 +01:00
// Determine where to start the interpreter. Normally we start at opcode #0,
// but if the state machine starts with an OPC_SwitchOpcode, then we
// accelerate the first lookup (which is guaranteed to be hot) with the
// OpcodeOffset table.
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
unsigned MatcherIndex = 0;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Accelerate isel dispatch for tables that start with a top-level OPC_SwitchOpcode to use a table lookup instead of having to go through the interpreter for this. llvm-svn: 97469
2010-03-01 19:47:11 +01:00
if (!OpcodeOffset.empty()) {
// Already computed the OpcodeOffset table, just index into it.
if (N.getOpcode() < OpcodeOffset.size())
MatcherIndex = OpcodeOffset[N.getOpcode()];
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << " Initial Opcode index to " << MatcherIndex << "\n");
Accelerate isel dispatch for tables that start with a top-level OPC_SwitchOpcode to use a table lookup instead of having to go through the interpreter for this. llvm-svn: 97469
2010-03-01 19:47:11 +01:00
} else if (MatcherTable[0] == OPC_SwitchOpcode) {
// Otherwise, the table isn't computed, but the state machine does start
// with an OPC_SwitchOpcode instruction. Populate the table now, since this
// is the first time we're selecting an instruction.
unsigned Idx = 1;
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
while (true) {
Accelerate isel dispatch for tables that start with a top-level OPC_SwitchOpcode to use a table lookup instead of having to go through the interpreter for this. llvm-svn: 97469
2010-03-01 19:47:11 +01:00
// Get the size of this case.
unsigned CaseSize = MatcherTable[Idx++];
if (CaseSize & 128)
CaseSize = GetVBR(CaseSize, MatcherTable, Idx);
if (CaseSize == 0) break;
// Get the opcode, add the index to the table.
Change tblgen to emit FOOISD opcode names as two bytes instead of one byte. This is important because we're running up to too many opcodes to fit in a byte and it is aggrevated by FIRST_TARGET_MEMORY_OPCODE making the numbering sparse. This just bites the bullet and bloats out the table. In practice, this increases the size of the x86 isel table from 74.5K to 76K. I think we'll cope :) This fixes rdar://7791648 llvm-svn: 99494
2010-03-25 07:33:05 +01:00
uint16_t Opc = MatcherTable[Idx++];
Opc |= (unsigned short)MatcherTable[Idx++] << 8;
Accelerate isel dispatch for tables that start with a top-level OPC_SwitchOpcode to use a table lookup instead of having to go through the interpreter for this. llvm-svn: 97469
2010-03-01 19:47:11 +01:00
if (Opc >= OpcodeOffset.size())
OpcodeOffset.resize((Opc+1)*2);
OpcodeOffset[Opc] = Idx;
Idx += CaseSize;
}
// Okay, do the lookup for the first opcode.
if (N.getOpcode() < OpcodeOffset.size())
MatcherIndex = OpcodeOffset[N.getOpcode()];
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
while (true) {
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
assert(MatcherIndex < TableSize && "Invalid index");
Attempt to make this debug output meaningful, both in the case of multibyte opcodes and in the case of multiple scopes. llvm-svn: 98036
2010-03-09 03:15:05 +01:00
#ifndef NDEBUG
unsigned CurrentOpcodeIndex = MatcherIndex;
#endif
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
BuiltinOpcodes Opcode = (BuiltinOpcodes)MatcherTable[MatcherIndex++];
switch (Opcode) {
case OPC_Scope: {
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
// Okay, the semantics of this operation are that we should push a scope
// then evaluate the first child. However, pushing a scope only to have
// the first check fail (which then pops it) is inefficient. If we can
// determine immediately that the first check (or first several) will
// immediately fail, don't even bother pushing a scope for them.
unsigned FailIndex;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
while (true) {
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
unsigned NumToSkip = MatcherTable[MatcherIndex++];
if (NumToSkip & 128)
NumToSkip = GetVBR(NumToSkip, MatcherTable, MatcherIndex);
// Found the end of the scope with no match.
if (NumToSkip == 0) {
FailIndex = 0;
break;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
FailIndex = MatcherIndex+NumToSkip;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
improve -debug-only=isel comments for cases when we don't enter a scope due to obviously false predicate. llvm-svn: 99723
2010-03-27 19:54:50 +01:00
unsigned MatcherIndexOfPredicate = MatcherIndex;
(void)MatcherIndexOfPredicate; // silence warning.
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
// If we can't evaluate this predicate without pushing a scope (e.g. if
// it is a 'MoveParent') or if the predicate succeeds on this node, we
// push the scope and evaluate the full predicate chain.
bool Result;
MatcherIndex = IsPredicateKnownToFail(MatcherTable, MatcherIndex, N,
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
Result, *this, RecordedNodes);
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
if (!Result)
break;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << " Skipped scope entry (due to false predicate) at "
improve -debug-only=isel comments for cases when we don't enter a scope due to obviously false predicate. llvm-svn: 99723
2010-03-27 19:54:50 +01:00
<< "index " << MatcherIndexOfPredicate
<< ", continuing at " << FailIndex << "\n");
Disable statistics on Release builds and move tests that depend on -stats. Summary: Statistics are still available in Release+Asserts (any +Asserts builds), and stats can also be turned on with LLVM_ENABLE_STATS. Move some of the FastISel stats that were moved under DEBUG() back out of DEBUG(), since stats are disabled across the board now. Many tests depend on grepping "-stats" output. Move those into a orig_dir/Stats/. so that they can be marked as unsupported when building without statistics. Differential Revision: http://llvm-reviews.chandlerc.com/D486 llvm-svn: 176733
2013-03-08 23:56:31 +01:00
++NumDAGIselRetries;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
// Otherwise, we know that this case of the Scope is guaranteed to fail,
// move to the next case.
MatcherIndex = FailIndex;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
// If the whole scope failed to match, bail.
if (FailIndex == 0) break;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// Push a MatchScope which indicates where to go if the first child fails
// to match.
MatchScope NewEntry;
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
NewEntry.FailIndex = FailIndex;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
NewEntry.NodeStack.append(NodeStack.begin(), NodeStack.end());
NewEntry.NumRecordedNodes = RecordedNodes.size();
NewEntry.NumMatchedMemRefs = MatchedMemRefs.size();
NewEntry.InputChain = InputChain;
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
NewEntry.InputGlue = InputGlue;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
NewEntry.HasChainNodesMatched = !ChainNodesMatched.empty();
MatchScopes.push_back(NewEntry);
continue;
}
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
case OPC_RecordNode: {
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// Remember this node, it may end up being an operand in the pattern.
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
SDNode *Parent = nullptr;
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
if (NodeStack.size() > 1)
Parent = NodeStack[NodeStack.size()-2].getNode();
RecordedNodes.push_back(std::make_pair(N, Parent));
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_RecordChild0: case OPC_RecordChild1:
case OPC_RecordChild2: case OPC_RecordChild3:
case OPC_RecordChild4: case OPC_RecordChild5:
case OPC_RecordChild6: case OPC_RecordChild7: {
unsigned ChildNo = Opcode-OPC_RecordChild0;
if (ChildNo >= N.getNumOperands())
break; // Match fails if out of range child #.
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
RecordedNodes.push_back(std::make_pair(N->getOperand(ChildNo),
N.getNode()));
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
}
case OPC_RecordMemRef:
MatchedMemRefs.push_back(cast<MemSDNode>(N)->getMemOperand());
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
flags -> glue for selectiondag llvm-svn: 122509
2010-12-23 18:24:32 +01:00
case OPC_CaptureGlueInput:
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// If the current node has an input glue, capture it in InputGlue.
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
if (N->getNumOperands() != 0 &&
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
N->getOperand(N->getNumOperands()-1).getValueType() == MVT::Glue)
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
InputGlue = N->getOperand(N->getNumOperands()-1);
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_MoveChild: {
unsigned ChildNo = MatcherTable[MatcherIndex++];
if (ChildNo >= N.getNumOperands())
break; // Match fails if out of range child #.
N = N.getOperand(ChildNo);
NodeStack.push_back(N);
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
[CodeGen] Add OPC_MoveChild0-OPC_MoveChild7 opcodes to isel matching tables to optimize table size. Shaves about 12K off the X86 matcher table. llvm-svn: 268209
2016-05-02 03:53:30 +02:00
case OPC_MoveChild0: case OPC_MoveChild1:
case OPC_MoveChild2: case OPC_MoveChild3:
case OPC_MoveChild4: case OPC_MoveChild5:
case OPC_MoveChild6: case OPC_MoveChild7: {
unsigned ChildNo = Opcode-OPC_MoveChild0;
if (ChildNo >= N.getNumOperands())
break; // Match fails if out of range child #.
N = N.getOperand(ChildNo);
NodeStack.push_back(N);
continue;
}
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_MoveParent:
// Pop the current node off the NodeStack.
NodeStack.pop_back();
assert(!NodeStack.empty() && "Node stack imbalance!");
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
N = NodeStack.back();
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
case OPC_CheckSame:
if (!::CheckSame(MatcherTable, MatcherIndex, N, RecordedNodes)) break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
Add OPC_CheckChildSame0-3 to the DAG isel matcher. This replaces sequences of MoveChild, CheckSame, MoveParent. Saves 846 bytes from the X86 DAG isel matcher, ~300 from ARM, ~840 from Hexagon. llvm-svn: 192026
2013-10-05 07:38:16 +02:00
case OPC_CheckChild0Same: case OPC_CheckChild1Same:
case OPC_CheckChild2Same: case OPC_CheckChild3Same:
if (!::CheckChildSame(MatcherTable, MatcherIndex, N, RecordedNodes,
Opcode-OPC_CheckChild0Same))
break;
continue;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_CheckPatternPredicate:
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
if (!::CheckPatternPredicate(MatcherTable, MatcherIndex, *this)) break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
case OPC_CheckPredicate:
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
if (!::CheckNodePredicate(MatcherTable, MatcherIndex, *this,
N.getNode()))
break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
change the new isel matcher to emit ComplexPattern matches as the very last thing before node emission. This should dramatically reduce the number of times we do 'MatchAddress' on X86, speeding up compile time. This also improves comments in the tables and shrinks the table a bit, now down to 80506 bytes for x86. llvm-svn: 97703
2010-03-04 02:23:08 +01:00
case OPC_CheckComplexPat: {
unsigned CPNum = MatcherTable[MatcherIndex++];
unsigned RecNo = MatcherTable[MatcherIndex++];
assert(RecNo < RecordedNodes.size() && "Invalid CheckComplexPat");
[ISel] Keep matching state consistent when folding during X86 address match In the X86 backend, matching an address is initiated by the 'addr' complex pattern and its friends. During this process we may reassociate and-of-shift into shift-of-and (FoldMaskedShiftToScaledMask) to allow folding of the shift into the scale of the address. However as demonstrated by the testcase, this can trigger CSE of not only the shift and the AND which the code is prepared for but also the underlying load node. In the testcase this node is sitting in the RecordedNode and MatchScope data structures of the matcher and becomes a deleted node upon CSE. Returning from the complex pattern function, we try to access it again hitting an assert because the node is no longer a load even though this was checked before. Now obviously changing the DAG this late is bending the rules but I think it makes sense somewhat. Outside of addresses we prefer and-of-shift because it may lead to smaller immediates (FoldMaskAndShiftToScale is an even better example because it create a non-canonical node). We currently don't recognize addresses during DAGCombiner where arguably this canonicalization should be performed. On the other hand, having this in the matcher allows us to cover all the cases where an address can be used in an instruction. I've also talked a little bit to Dan Gohman on llvm-dev who added the RAUW for the new shift node in FoldMaskedShiftToScaledMask. This RAUW is responsible for initiating the recursive CSE on users (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-September/076903.html) but it is not strictly necessary since the shift is hooked into the visited user. Of course it's safer to keep the DAG consistent at all times (e.g. for accurate number of uses, etc.). So rather than changing the fundamentals, I've decided to continue along the previous patches and detect the CSE. This patch installs a very targeted DAGUpdateListener for the duration of a complex-pattern match and updates the matching state accordingly. (Previous patches used HandleSDNode to detect the CSE but that's not practical here). The listener is only installed on X86. I tested that there is no measurable overhead due to this while running through the spec2k BC files with llc. The only thing we pay for is the creation of the listener. The callback never ever triggers in spec2k since this is a corner case. Fixes rdar://problem/18206171 llvm-svn: 219009
2014-10-03 22:00:34 +02:00
// If target can modify DAG during matching, keep the matching state
// consistent.
std::unique_ptr<MatchStateUpdater> MSU;
if (ComplexPatternFuncMutatesDAG())
[SelectionDAG] Fix for PR30775: Assertion `NodeToMatch->getOpcode() != ISD::DELETED_NODE && "NodeToMatch was removed partway through selection"' failed. NodeToMatch can be modified during matching, but code does not handle this situation. Differential Revision: https://reviews.llvm.org/D29292 llvm-svn: 294003
2017-02-03 13:28:40 +01:00
MSU.reset(new MatchStateUpdater(*CurDAG, &NodeToMatch, RecordedNodes,
[ISel] Keep matching state consistent when folding during X86 address match In the X86 backend, matching an address is initiated by the 'addr' complex pattern and its friends. During this process we may reassociate and-of-shift into shift-of-and (FoldMaskedShiftToScaledMask) to allow folding of the shift into the scale of the address. However as demonstrated by the testcase, this can trigger CSE of not only the shift and the AND which the code is prepared for but also the underlying load node. In the testcase this node is sitting in the RecordedNode and MatchScope data structures of the matcher and becomes a deleted node upon CSE. Returning from the complex pattern function, we try to access it again hitting an assert because the node is no longer a load even though this was checked before. Now obviously changing the DAG this late is bending the rules but I think it makes sense somewhat. Outside of addresses we prefer and-of-shift because it may lead to smaller immediates (FoldMaskAndShiftToScale is an even better example because it create a non-canonical node). We currently don't recognize addresses during DAGCombiner where arguably this canonicalization should be performed. On the other hand, having this in the matcher allows us to cover all the cases where an address can be used in an instruction. I've also talked a little bit to Dan Gohman on llvm-dev who added the RAUW for the new shift node in FoldMaskedShiftToScaledMask. This RAUW is responsible for initiating the recursive CSE on users (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-September/076903.html) but it is not strictly necessary since the shift is hooked into the visited user. Of course it's safer to keep the DAG consistent at all times (e.g. for accurate number of uses, etc.). So rather than changing the fundamentals, I've decided to continue along the previous patches and detect the CSE. This patch installs a very targeted DAGUpdateListener for the duration of a complex-pattern match and updates the matching state accordingly. (Previous patches used HandleSDNode to detect the CSE but that's not practical here). The listener is only installed on X86. I tested that there is no measurable overhead due to this while running through the spec2k BC files with llc. The only thing we pay for is the creation of the listener. The callback never ever triggers in spec2k since this is a corner case. Fixes rdar://problem/18206171 llvm-svn: 219009
2014-10-03 22:00:34 +02:00
MatchScopes));
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
if (!CheckComplexPattern(NodeToMatch, RecordedNodes[RecNo].second,
RecordedNodes[RecNo].first, CPNum,
change the new isel matcher to emit ComplexPattern matches as the very last thing before node emission. This should dramatically reduce the number of times we do 'MatchAddress' on X86, speeding up compile time. This also improves comments in the tables and shrinks the table a bit, now down to 80506 bytes for x86. llvm-svn: 97703
2010-03-04 02:23:08 +01:00
RecordedNodes))
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
break;
continue;
change the new isel matcher to emit ComplexPattern matches as the very last thing before node emission. This should dramatically reduce the number of times we do 'MatchAddress' on X86, speeding up compile time. This also improves comments in the tables and shrinks the table a bit, now down to 80506 bytes for x86. llvm-svn: 97703
2010-03-04 02:23:08 +01:00
}
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_CheckOpcode:
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
if (!::CheckOpcode(MatcherTable, MatcherIndex, N.getNode())) break;
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
case OPC_CheckType:
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
if (!::CheckType(MatcherTable, MatcherIndex, N, TLI,
CurDAG->getDataLayout()))
Access the TargetLoweringInfo from the TargetMachine object instead of caching it. The TLI may change between functions. No functionality change. llvm-svn: 184360
2013-06-19 23:36:55 +02:00
break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
case OPC_SwitchOpcode: {
unsigned CurNodeOpcode = N.getOpcode();
some trivial microoptimizations. llvm-svn: 97441
2010-03-01 08:43:08 +01:00
unsigned SwitchStart = MatcherIndex-1; (void)SwitchStart;
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
unsigned CaseSize;
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
while (true) {
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
// Get the size of this case.
CaseSize = MatcherTable[MatcherIndex++];
if (CaseSize & 128)
CaseSize = GetVBR(CaseSize, MatcherTable, MatcherIndex);
if (CaseSize == 0) break;
Change tblgen to emit FOOISD opcode names as two bytes instead of one byte. This is important because we're running up to too many opcodes to fit in a byte and it is aggrevated by FIRST_TARGET_MEMORY_OPCODE making the numbering sparse. This just bites the bullet and bloats out the table. In practice, this increases the size of the x86 isel table from 74.5K to 76K. I think we'll cope :) This fixes rdar://7791648 llvm-svn: 99494
2010-03-25 07:33:05 +01:00
uint16_t Opc = MatcherTable[MatcherIndex++];
Opc |= (unsigned short)MatcherTable[MatcherIndex++] << 8;
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
// If the opcode matches, then we will execute this case.
Change tblgen to emit FOOISD opcode names as two bytes instead of one byte. This is important because we're running up to too many opcodes to fit in a byte and it is aggrevated by FIRST_TARGET_MEMORY_OPCODE making the numbering sparse. This just bites the bullet and bloats out the table. In practice, this increases the size of the x86 isel table from 74.5K to 76K. I think we'll cope :) This fixes rdar://7791648 llvm-svn: 99494
2010-03-25 07:33:05 +01:00
if (CurNodeOpcode == Opc)
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
break;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
// Otherwise, skip over this case.
MatcherIndex += CaseSize;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
// If no cases matched, bail out.
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
if (CaseSize == 0) break;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
// Otherwise, execute the case we found.
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << " OpcodeSwitch from " << SwitchStart
add a new OPC_SwitchOpcode which is semantically equivalent to a scope where every child starts with a CheckOpcode, but executes more efficiently. Enhance DAGISelMatcherOpt to form it. This also fixes a bug in CheckOpcode: apparently the SDNodeInfo objects are not pointer comparable, we have to compare the enum name. llvm-svn: 97438
2010-03-01 07:59:22 +01:00
<< " to " << MatcherIndex << "\n");
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
case OPC_SwitchType: {
Replace getValueType().getSimpleVT() with getSimpleValueType(). llvm-svn: 188442
2013-08-15 04:44:19 +02:00
MVT CurNodeVT = N.getSimpleValueType();
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
unsigned SwitchStart = MatcherIndex-1; (void)SwitchStart;
unsigned CaseSize;
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
while (true) {
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
// Get the size of this case.
CaseSize = MatcherTable[MatcherIndex++];
if (CaseSize & 128)
CaseSize = GetVBR(CaseSize, MatcherTable, MatcherIndex);
if (CaseSize == 0) break;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Simplify uses of MVT and EVT. An MVT can be compared directly with a SimpleValueType, while an EVT supports equality and inequality comparisons with SimpleValueType. llvm-svn: 118169
2010-11-03 13:17:33 +01:00
MVT CaseVT = (MVT::SimpleValueType)MatcherTable[MatcherIndex++];
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
if (CaseVT == MVT::iPTR)
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
CaseVT = TLI->getPointerTy(CurDAG->getDataLayout());
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
// If the VT matches, then we will execute this case.
if (CurNodeVT == CaseVT)
break;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
// Otherwise, skip over this case.
MatcherIndex += CaseSize;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
// If no cases matched, bail out.
if (CaseSize == 0) break;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
// Otherwise, execute the case we found.
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << " TypeSwitch[" << EVT(CurNodeVT).getEVTString()
introduce a new SwitchTypeMatcher node (which is analogous to SwitchOpcodeMatcher) and have DAGISelMatcherOpt form it. This speeds up selection, particularly for X86 which has lots of variants of instructions with only type differences. llvm-svn: 97645
2010-03-03 07:28:15 +01:00
<< "] from " << SwitchStart << " to " << MatcherIndex<<'\n');
continue;
}
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_CheckChild0Type: case OPC_CheckChild1Type:
case OPC_CheckChild2Type: case OPC_CheckChild3Type:
case OPC_CheckChild4Type: case OPC_CheckChild5Type:
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
case OPC_CheckChild6Type: case OPC_CheckChild7Type:
Cache TargetLowering on SelectionDAGISel and update previous calls to getTargetLowering() with the cached variable. llvm-svn: 219284
2014-10-08 09:32:17 +02:00
if (!::CheckChildType(MatcherTable, MatcherIndex, N, TLI,
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
CurDAG->getDataLayout(),
Opcode - OPC_CheckChild0Type))
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
case OPC_CheckCondCode:
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
if (!::CheckCondCode(MatcherTable, MatcherIndex, N)) break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
case OPC_CheckValueType:
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
if (!::CheckValueType(MatcherTable, MatcherIndex, N, TLI,
CurDAG->getDataLayout()))
Access the TargetLoweringInfo from the TargetMachine object instead of caching it. The TLI may change between functions. No functionality change. llvm-svn: 184360
2013-06-19 23:36:55 +02:00
break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
speed up scope node processing: if the first element of a scope entry we're about to process is obviously going to fail, don't bother pushing a scope only to have it immediately be popped. This avoids a lot of scope stack traffic in common cases. Unfortunately, this requires duplicating some of the predicate dispatch. To avoid duplicating the actual logic I pulled each predicate out to its own static function which gets used in both places. llvm-svn: 97651
2010-03-03 08:31:15 +01:00
case OPC_CheckInteger:
if (!::CheckInteger(MatcherTable, MatcherIndex, N)) break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
Add CheckChildInteger to ISelMatcher operations. Removes nearly 2000 bytes from X86 matcher table. llvm-svn: 200821
2014-02-05 06:44:28 +01:00
case OPC_CheckChild0Integer: case OPC_CheckChild1Integer:
case OPC_CheckChild2Integer: case OPC_CheckChild3Integer:
case OPC_CheckChild4Integer:
if (!::CheckChildInteger(MatcherTable, MatcherIndex, N,
Opcode-OPC_CheckChild0Integer)) break;
continue;
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
case OPC_CheckAndImm:
if (!::CheckAndImm(MatcherTable, MatcherIndex, N, *this)) break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
add some of the more obscure predicate types to the Scope accelerator. llvm-svn: 97652
2010-03-03 08:46:25 +01:00
case OPC_CheckOrImm:
if (!::CheckOrImm(MatcherTable, MatcherIndex, N, *this)) break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_CheckFoldableChainNode: {
assert(NodeStack.size() != 1 && "No parent node");
// Verify that all intermediate nodes between the root and this one have
// a single use.
bool HasMultipleUses = false;
for (unsigned i = 1, e = NodeStack.size()-1; i != e; ++i)
if (!NodeStack[i].hasOneUse()) {
HasMultipleUses = true;
break;
}
if (HasMultipleUses) break;
// Check to see that the target thinks this is profitable to fold and that
// we can fold it without inducing cycles in the graph.
if (!IsProfitableToFold(N, NodeStack[NodeStack.size()-2].getNode(),
NodeToMatch) ||
!IsLegalToFold(N, NodeStack[NodeStack.size()-2].getNode(),
Use const qualifiers with TargetLowering. This eliminates several const_casts, and it reinforces the design of the Target classes being immutable. SelectionDAGISel::IsLegalToFold is now a static member function, because PIC16 uses it in an unconventional way. There is more room for API cleanup here. And PIC16's AsmPrinter no longer uses TargetLowering. llvm-svn: 101635
2010-04-17 17:26:15 +02:00
NodeToMatch, OptLevel,
true/*We validate our own chains*/))
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
break;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
}
case OPC_EmitInteger: {
MVT::SimpleValueType VT =
(MVT::SimpleValueType)MatcherTable[MatcherIndex++];
int64_t Val = MatcherTable[MatcherIndex++];
if (Val & 128)
Val = GetVBR(Val, MatcherTable, MatcherIndex);
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
RecordedNodes.push_back(std::pair<SDValue, SDNode*>(
Reapply r235977 "[DebugInfo] Add debug locations to constant SD nodes" [DebugInfo] Add debug locations to constant SD nodes This adds debug location to constant nodes of Selection DAG and updates all places that create constants to pass debug locations (see PR13269). Can't guarantee that all locations are correct, but in a lot of cases choice is obvious, so most of them should be. At least all tests pass. Tests for these changes do not cover everything, instead just check it for SDNodes, ARM and AArch64 where it's easy to get incorrect locations on constants. This is not complete fix as FastISel contains workaround for wrong debug locations, which drops locations from instructions on processing constants, but there isn't currently a way to use debug locations from constants there as llvm::Constant doesn't cache it (yet). Although this is a bit different issue, not directly related to these changes. Differential Revision: http://reviews.llvm.org/D9084 llvm-svn: 235989
2015-04-28 16:05:47 +02:00
CurDAG->getTargetConstant(Val, SDLoc(NodeToMatch),
VT), nullptr));
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
}
case OPC_EmitRegister: {
MVT::SimpleValueType VT =
(MVT::SimpleValueType)MatcherTable[MatcherIndex++];
unsigned RegNo = MatcherTable[MatcherIndex++];
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
RecordedNodes.push_back(std::pair<SDValue, SDNode*>(
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
CurDAG->getRegister(RegNo, VT), nullptr));
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
}
Generalize the register matching code in DAGISel a bit. llvm-svn: 126731
2011-03-01 02:37:19 +01:00
case OPC_EmitRegister2: {
// For targets w/ more than 256 register names, the register enum
// values are stored in two bytes in the matcher table (just like
// opcodes).
MVT::SimpleValueType VT =
(MVT::SimpleValueType)MatcherTable[MatcherIndex++];
unsigned RegNo = MatcherTable[MatcherIndex++];
RegNo |= MatcherTable[MatcherIndex++] << 8;
RecordedNodes.push_back(std::pair<SDValue, SDNode*>(
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
CurDAG->getRegister(RegNo, VT), nullptr));
Generalize the register matching code in DAGISel a bit. llvm-svn: 126731
2011-03-01 02:37:19 +01:00
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_EmitConvertToTarget: {
// Convert from IMM/FPIMM to target version.
unsigned RecNo = MatcherTable[MatcherIndex++];
Fix some assert messages to say the correct opcode name. Looks like one assert got copy and pasted to many places. llvm-svn: 192078
2013-10-07 00:38:19 +02:00
assert(RecNo < RecordedNodes.size() && "Invalid EmitConvertToTarget");
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
SDValue Imm = RecordedNodes[RecNo].first;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
if (Imm->getOpcode() == ISD::Constant) {
No need to go through int64 and APInt when generating a new constant. llvm-svn: 176615
2013-03-07 07:34:49 +01:00
const ConstantInt *Val=cast<ConstantSDNode>(Imm)->getConstantIntValue();
Use SelectionDAG::getTargetConstant* helper functions. NFC. Instead of SelectionDAG::getConstant directly to make it more obvious that we're creating target constants. llvm-svn: 268074
2016-04-29 19:42:45 +02:00
Imm = CurDAG->getTargetConstant(*Val, SDLoc(NodeToMatch),
Imm.getValueType());
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
} else if (Imm->getOpcode() == ISD::ConstantFP) {
const ConstantFP *Val=cast<ConstantFPSDNode>(Imm)->getConstantFPValue();
Use SelectionDAG::getTargetConstant* helper functions. NFC. Instead of SelectionDAG::getConstant directly to make it more obvious that we're creating target constants. llvm-svn: 268074
2016-04-29 19:42:45 +02:00
Imm = CurDAG->getTargetConstantFP(*Val, SDLoc(NodeToMatch),
Imm.getValueType());
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
RecordedNodes.push_back(std::make_pair(Imm, RecordedNodes[RecNo].second));
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add an optimized form of OPC_EmitMergeInputChains for the 1, 0 and 1, 1 cases which are by-far the most frequent. This shrinks the X86 isel table from 77014 -> 74657 bytes. llvm-svn: 99740
2010-03-28 07:50:16 +02:00
case OPC_EmitMergeInputChains1_0: // OPC_EmitMergeInputChains, 1, 0
[CodeGen] Add space-optimized EmitMergeInputChains1_2 to the DAG isel matching tables. Shaves about 5100 bytes from the X86 matcher table. NFC llvm-svn: 262815
2016-03-07 08:29:12 +01:00
case OPC_EmitMergeInputChains1_1: // OPC_EmitMergeInputChains, 1, 1
case OPC_EmitMergeInputChains1_2: { // OPC_EmitMergeInputChains, 1, 2
add an optimized form of OPC_EmitMergeInputChains for the 1, 0 and 1, 1 cases which are by-far the most frequent. This shrinks the X86 isel table from 77014 -> 74657 bytes. llvm-svn: 99740
2010-03-28 07:50:16 +02:00
// These are space-optimized forms of OPC_EmitMergeInputChains.
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
assert(!InputChain.getNode() &&
add an optimized form of OPC_EmitMergeInputChains for the 1, 0 and 1, 1 cases which are by-far the most frequent. This shrinks the X86 isel table from 77014 -> 74657 bytes. llvm-svn: 99740
2010-03-28 07:50:16 +02:00
"EmitMergeInputChains should be the first chain producing node");
assert(ChainNodesMatched.empty() &&
"Should only have one EmitMergeInputChains per match");
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add an optimized form of OPC_EmitMergeInputChains for the 1, 0 and 1, 1 cases which are by-far the most frequent. This shrinks the X86 isel table from 77014 -> 74657 bytes. llvm-svn: 99740
2010-03-28 07:50:16 +02:00
// Read all of the chained nodes.
[CodeGen] Add space-optimized EmitMergeInputChains1_2 to the DAG isel matching tables. Shaves about 5100 bytes from the X86 matcher table. NFC llvm-svn: 262815
2016-03-07 08:29:12 +01:00
unsigned RecNo = Opcode - OPC_EmitMergeInputChains1_0;
Fix some assert messages to say the correct opcode name. Looks like one assert got copy and pasted to many places. llvm-svn: 192078
2013-10-07 00:38:19 +02:00
assert(RecNo < RecordedNodes.size() && "Invalid EmitMergeInputChains");
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
ChainNodesMatched.push_back(RecordedNodes[RecNo].first.getNode());
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add an optimized form of OPC_EmitMergeInputChains for the 1, 0 and 1, 1 cases which are by-far the most frequent. This shrinks the X86 isel table from 77014 -> 74657 bytes. llvm-svn: 99740
2010-03-28 07:50:16 +02:00
// FIXME: What if other value results of the node have uses not matched
// by this pattern?
if (ChainNodesMatched.back() != NodeToMatch &&
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
!RecordedNodes[RecNo].first.hasOneUse()) {
add an optimized form of OPC_EmitMergeInputChains for the 1, 0 and 1, 1 cases which are by-far the most frequent. This shrinks the X86 isel table from 77014 -> 74657 bytes. llvm-svn: 99740
2010-03-28 07:50:16 +02:00
ChainNodesMatched.clear();
break;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
add an optimized form of OPC_EmitMergeInputChains for the 1, 0 and 1, 1 cases which are by-far the most frequent. This shrinks the X86 isel table from 77014 -> 74657 bytes. llvm-svn: 99740
2010-03-28 07:50:16 +02:00
// Merge the input chains if they are not intra-pattern references.
InputChain = HandleMergeInputChains(ChainNodesMatched, CurDAG);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
if (!InputChain.getNode())
add an optimized form of OPC_EmitMergeInputChains for the 1, 0 and 1, 1 cases which are by-far the most frequent. This shrinks the X86 isel table from 77014 -> 74657 bytes. llvm-svn: 99740
2010-03-28 07:50:16 +02:00
break; // Failed to merge.
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_EmitMergeInputChains: {
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
assert(!InputChain.getNode() &&
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
"EmitMergeInputChains should be the first chain producing node");
// This node gets a list of nodes we matched in the input that have
// chains. We want to token factor all of the input chains to these nodes
// together. However, if any of the input chains is actually one of the
// nodes matched in this pattern, then we have an intra-match reference.
// Ignore these because the newly token factored chain should not refer to
// the old nodes.
unsigned NumChains = MatcherTable[MatcherIndex++];
assert(NumChains != 0 && "Can't TF zero chains");
assert(ChainNodesMatched.empty() &&
"Should only have one EmitMergeInputChains per match");
// Read all of the chained nodes.
run HandleMergeInputChains even if we only have one input chain. llvm-svn: 97581
2010-03-02 20:34:59 +01:00
for (unsigned i = 0; i != NumChains; ++i) {
unsigned RecNo = MatcherTable[MatcherIndex++];
Fix some assert messages to say the correct opcode name. Looks like one assert got copy and pasted to many places. llvm-svn: 192078
2013-10-07 00:38:19 +02:00
assert(RecNo < RecordedNodes.size() && "Invalid EmitMergeInputChains");
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
ChainNodesMatched.push_back(RecordedNodes[RecNo].first.getNode());
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// FIXME: What if other value results of the node have uses not matched
// by this pattern?
if (ChainNodesMatched.back() != NodeToMatch &&
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
!RecordedNodes[RecNo].first.hasOneUse()) {
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
ChainNodesMatched.clear();
break;
}
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
// If the inner loop broke out, the match fails.
if (ChainNodesMatched.empty())
break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
// Merge the input chains if they are not intra-pattern references.
InputChain = HandleMergeInputChains(ChainNodesMatched, CurDAG);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
if (!InputChain.getNode())
refactor some code out of OPC_EmitMergeInputChains into a new helper function. llvm-svn: 97525
2010-03-02 01:00:03 +01:00
break; // Failed to merge.
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_EmitCopyToReg: {
unsigned RecNo = MatcherTable[MatcherIndex++];
Fix some assert messages to say the correct opcode name. Looks like one assert got copy and pasted to many places. llvm-svn: 192078
2013-10-07 00:38:19 +02:00
assert(RecNo < RecordedNodes.size() && "Invalid EmitCopyToReg");
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
unsigned DestPhysReg = MatcherTable[MatcherIndex++];
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
if (!InputChain.getNode())
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
InputChain = CurDAG->getEntryNode();
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Track IR ordering of SelectionDAG nodes 2/4. Change SelectionDAG::getXXXNode() interfaces as well as call sites of these functions to pass in SDLoc instead of DebugLoc. llvm-svn: 182703
2013-05-25 04:42:55 +02:00
InputChain = CurDAG->getCopyToReg(InputChain, SDLoc(NodeToMatch),
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
DestPhysReg, RecordedNodes[RecNo].first,
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
InputGlue);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
InputGlue = InputChain.getValue(1);
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_EmitNodeXForm: {
unsigned XFormNo = MatcherTable[MatcherIndex++];
unsigned RecNo = MatcherTable[MatcherIndex++];
Fix some assert messages to say the correct opcode name. Looks like one assert got copy and pasted to many places. llvm-svn: 192078
2013-10-07 00:38:19 +02:00
assert(RecNo < RecordedNodes.size() && "Invalid EmitNodeXForm");
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
SDValue Res = RunSDNodeXForm(RecordedNodes[RecNo].first, XFormNo);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
RecordedNodes.push_back(std::pair<SDValue,SDNode*>(Res, nullptr));
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
continue;
}
[Tablegen] Instrumenting table gen DAGGenISelDAG To help assist in debugging ISEL or to prioritize GlobalISel backend work, this patch adds two more tables to <Target>GenISelDAGISel.inc - one which contains the patterns that are used during selection and the other containing include source location of the patterns Enabled through CMake varialbe LLVM_ENABLE_DAGISEL_COV llvm-svn: 295081
2017-02-14 19:32:41 +01:00
case OPC_Coverage: {
// This is emitted right before MorphNode/EmitNode.
// So it should be safe to assume that this node has been selected
unsigned index = MatcherTable[MatcherIndex++];
index |= (MatcherTable[MatcherIndex++] << 8);
dbgs() << "COVERED: " << getPatternForIndex(index) << "\n";
dbgs() << "INCLUDED: " << getIncludePathForIndex(index) << "\n";
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
[CodeGen] Add some space optimized forms of EmitNode and MorphNodeTo that implicitly indicate the number of result VTs. This shaves about 16K off the X86 matching table taking it down to about 470K. Overall this reduces the llc binary size with all in-tree targets by about 40K. llvm-svn: 268365
2016-05-03 07:54:13 +02:00
case OPC_EmitNode: case OPC_MorphNodeTo:
case OPC_EmitNode0: case OPC_EmitNode1: case OPC_EmitNode2:
case OPC_MorphNodeTo0: case OPC_MorphNodeTo1: case OPC_MorphNodeTo2: {
eliminate GetInt1/2 llvm-svn: 97426
2010-02-28 23:38:43 +01:00
uint16_t TargetOpc = MatcherTable[MatcherIndex++];
TargetOpc |= (unsigned short)MatcherTable[MatcherIndex++] << 8;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
unsigned EmitNodeInfo = MatcherTable[MatcherIndex++];
// Get the result VT list.
[CodeGen] Add some space optimized forms of EmitNode and MorphNodeTo that implicitly indicate the number of result VTs. This shaves about 16K off the X86 matching table taking it down to about 470K. Overall this reduces the llc binary size with all in-tree targets by about 40K. llvm-svn: 268365
2016-05-03 07:54:13 +02:00
unsigned NumVTs;
// If this is one of the compressed forms, get the number of VTs based
// on the Opcode. Otherwise read the next byte from the table.
if (Opcode >= OPC_MorphNodeTo0 && Opcode <= OPC_MorphNodeTo2)
NumVTs = Opcode - OPC_MorphNodeTo0;
else if (Opcode >= OPC_EmitNode0 && Opcode <= OPC_EmitNode2)
NumVTs = Opcode - OPC_EmitNode0;
else
NumVTs = MatcherTable[MatcherIndex++];
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
SmallVector<EVT, 4> VTs;
for (unsigned i = 0; i != NumVTs; ++i) {
MVT::SimpleValueType VT =
(MVT::SimpleValueType)MatcherTable[MatcherIndex++];
Cache TargetLowering on SelectionDAGISel and update previous calls to getTargetLowering() with the cached variable. llvm-svn: 219284
2014-10-08 09:32:17 +02:00
if (VT == MVT::iPTR)
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 04:09:04 +02:00
VT = TLI->getPointerTy(CurDAG->getDataLayout()).SimpleTy;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
VTs.push_back(VT);
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
if (EmitNodeInfo & OPFL_Chain)
VTs.push_back(MVT::Other);
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
if (EmitNodeInfo & OPFL_GlueOutput)
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
VTs.push_back(MVT::Glue);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
some trivial microoptimizations. llvm-svn: 97441
2010-03-01 08:43:08 +01:00
// This is hot code, so optimize the two most common cases of 1 and 2
// results.
SDVTList VTList;
if (VTs.size() == 1)
VTList = CurDAG->getVTList(VTs[0]);
else if (VTs.size() == 2)
VTList = CurDAG->getVTList(VTs[0], VTs[1]);
else
Convert SelectionDAG::getVTList to use ArrayRef llvm-svn: 206357
2014-04-16 08:10:51 +02:00
VTList = CurDAG->getVTList(VTs);
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// Get the operand list.
unsigned NumOps = MatcherTable[MatcherIndex++];
SmallVector<SDValue, 8> Ops;
for (unsigned i = 0; i != NumOps; ++i) {
unsigned RecNo = MatcherTable[MatcherIndex++];
if (RecNo & 128)
RecNo = GetVBR(RecNo, MatcherTable, MatcherIndex);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
assert(RecNo < RecordedNodes.size() && "Invalid EmitNode");
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
Ops.push_back(RecordedNodes[RecNo].first);
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// If there are variadic operands to add, handle them now.
if (EmitNodeInfo & OPFL_VariadicInfo) {
// Determine the start index to copy from.
unsigned FirstOpToCopy = getNumFixedFromVariadicInfo(EmitNodeInfo);
FirstOpToCopy += (EmitNodeInfo & OPFL_Chain) ? 1 : 0;
assert(NodeToMatch->getNumOperands() >= FirstOpToCopy &&
"Invalid variadic node");
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// Copy all of the variadic operands, not including a potential glue
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// input.
for (unsigned i = FirstOpToCopy, e = NodeToMatch->getNumOperands();
i != e; ++i) {
SDValue V = NodeToMatch->getOperand(i);
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
if (V.getValueType() == MVT::Glue) break;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
Ops.push_back(V);
}
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// If this has chain/glue inputs, add them.
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
if (EmitNodeInfo & OPFL_Chain)
Ops.push_back(InputChain);
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
if ((EmitNodeInfo & OPFL_GlueInput) && InputGlue.getNode() != nullptr)
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
Ops.push_back(InputGlue);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// Create the node.
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
SDNode *Res = nullptr;
[CodeGen] Add some space optimized forms of EmitNode and MorphNodeTo that implicitly indicate the number of result VTs. This shaves about 16K off the X86 matching table taking it down to about 470K. Overall this reduces the llc binary size with all in-tree targets by about 40K. llvm-svn: 268365
2016-05-03 07:54:13 +02:00
bool IsMorphNodeTo = Opcode == OPC_MorphNodeTo ||
(Opcode >= OPC_MorphNodeTo0 && Opcode <= OPC_MorphNodeTo2);
if (!IsMorphNodeTo) {
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// If this is a normal EmitNode command, just create the new node and
// add the results to the RecordedNodes list.
Track IR ordering of SelectionDAG nodes 2/4. Change SelectionDAG::getXXXNode() interfaces as well as call sites of these functions to pass in SDLoc instead of DebugLoc. llvm-svn: 182703
2013-05-25 04:42:55 +02:00
Res = CurDAG->getMachineNode(TargetOpc, SDLoc(NodeToMatch),
ArrayRefize getMachineNode(). No functionality change. llvm-svn: 179901
2013-04-20 00:22:57 +02:00
VTList, Ops);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// Add all the non-glue/non-chain results to the RecordedNodes list.
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
for (unsigned i = 0, e = VTs.size(); i != e; ++i) {
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
if (VTs[i] == MVT::Other || VTs[i] == MVT::Glue) break;
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
RecordedNodes.push_back(std::pair<SDValue,SDNode*>(SDValue(Res, i),
[C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 206142
2014-04-14 02:51:57 +02:00
nullptr));
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
Fix PR12599. The X86 target is editing the selection DAG while isel is selecting nodes following a topological ordering. When the DAG hacking triggers CSE, nodes can be deleted and bad things happen. llvm-svn: 155257
2012-04-21 01:36:09 +02:00
} else {
SDAG: Replace some unreachable code with an assert. NFC The current node shouldn't be (and isn't) removed partway through selection. llvm-svn: 271699
2016-06-03 20:09:53 +02:00
assert(NodeToMatch->getOpcode() != ISD::DELETED_NODE &&
"NodeToMatch was removed partway through selection");
Re-apply "SDAG: Update ChainNodesMatched as nodes are deleted" My first attempt at this had an overly aggressive assert - chain nodes will only be removed, but we could hit the assert if a non-chain node was CSE'd (NodeToMatch, for instance). This reapplies r271706 by reverting r271713 and fixing an assert. Original message: Avoid relying on UB by looking into deleted nodes for a marker value. Instead, update the list of chain nodes as we go. llvm-svn: 271733
2016-06-03 22:47:40 +02:00
SelectionDAG::DAGNodeDeletedListener NDL(*CurDAG, [&](SDNode *N,
SDNode *E) {
auto &Chain = ChainNodesMatched;
Use the range variant of find instead of unpacking begin/end If the result of the find is only used to compare against end(), just use is_contained instead. No functionality change is intended. llvm-svn: 278433
2016-08-12 00:21:41 +02:00
assert((!E || !is_contained(Chain, N)) &&
Re-apply "SDAG: Update ChainNodesMatched as nodes are deleted" My first attempt at this had an overly aggressive assert - chain nodes will only be removed, but we could hit the assert if a non-chain node was CSE'd (NodeToMatch, for instance). This reapplies r271706 by reverting r271713 and fixing an assert. Original message: Avoid relying on UB by looking into deleted nodes for a marker value. Instead, update the list of chain nodes as we go. llvm-svn: 271733
2016-06-03 22:47:40 +02:00
"Chain node replaced during MorphNode");
Chain.erase(std::remove(Chain.begin(), Chain.end(), N), Chain.end());
});
SDAG: Replace some unreachable code with an assert. NFC The current node shouldn't be (and isn't) removed partway through selection. llvm-svn: 271699
2016-06-03 20:09:53 +02:00
Res = MorphNode(NodeToMatch, TargetOpc, VTList, Ops, EmitNodeInfo);
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// If the node had chain/glue results, update our notion of the current
// chain and glue.
if (EmitNodeInfo & OPFL_GlueOutput) {
InputGlue = SDValue(Res, VTs.size()-1);
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
if (EmitNodeInfo & OPFL_Chain)
InputChain = SDValue(Res, VTs.size()-2);
} else if (EmitNodeInfo & OPFL_Chain)
InputChain = SDValue(Res, VTs.size()-1);
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
// If the OPFL_MemRefs glue is set on this node, slap all of the
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// accumulated memrefs onto it.
//
// FIXME: This is vastly incorrect for patterns with multiple outputs
// instructions that access memory and for ComplexPatterns that match
// loads.
if (EmitNodeInfo & OPFL_MemRefs) {
Fix PR9955 by only attaching load memory operands to load instructions and similarly for stores. Now "make check" passes with the MachineVerifier forced on with the VerifyCoalescing option! llvm-svn: 131705
2011-05-20 01:44:34 +02:00
// Only attach load or store memory operands if the generated
// instruction may load or store.
Cache SelectionDAGISel TargetInstrInfo lookups on the class and propagate. Also use the TargetSubtargetInfo and the MachineFunction and move TargetRegisterInfo query closer to uses. llvm-svn: 219273
2014-10-08 03:58:03 +02:00
const MCInstrDesc &MCID = TII->get(TargetOpc);
- Rename TargetInstrDesc, TargetOperandInfo to MCInstrDesc and MCOperandInfo and sink them into MC layer. - Added MCInstrInfo, which captures the tablegen generated static data. Chang TargetInstrInfo so it's based off MCInstrInfo. llvm-svn: 134021
2011-06-28 21:10:37 +02:00
bool mayLoad = MCID.mayLoad();
bool mayStore = MCID.mayStore();
Fix PR9955 by only attaching load memory operands to load instructions and similarly for stores. Now "make check" passes with the MachineVerifier forced on with the VerifyCoalescing option! llvm-svn: 131705
2011-05-20 01:44:34 +02:00
unsigned NumMemRefs = 0;
Use SmallVectorImpl::const_iterator instead of SmallVector to avoid specifying the vector size. llvm-svn: 185514
2013-07-03 07:18:47 +02:00
for (SmallVectorImpl<MachineMemOperand *>::const_iterator I =
MatchedMemRefs.begin(), E = MatchedMemRefs.end(); I != E; ++I) {
Fix PR9955 by only attaching load memory operands to load instructions and similarly for stores. Now "make check" passes with the MachineVerifier forced on with the VerifyCoalescing option! llvm-svn: 131705
2011-05-20 01:44:34 +02:00
if ((*I)->isLoad()) {
if (mayLoad)
++NumMemRefs;
} else if ((*I)->isStore()) {
if (mayStore)
++NumMemRefs;
} else {
++NumMemRefs;
}
}
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
MachineSDNode::mmo_iterator MemRefs =
Fix PR9955 by only attaching load memory operands to load instructions and similarly for stores. Now "make check" passes with the MachineVerifier forced on with the VerifyCoalescing option! llvm-svn: 131705
2011-05-20 01:44:34 +02:00
MF->allocateMemRefsArray(NumMemRefs);
MachineSDNode::mmo_iterator MemRefsPos = MemRefs;
Use SmallVectorImpl::const_iterator instead of SmallVector to avoid specifying the vector size. llvm-svn: 185514
2013-07-03 07:18:47 +02:00
for (SmallVectorImpl<MachineMemOperand *>::const_iterator I =
MatchedMemRefs.begin(), E = MatchedMemRefs.end(); I != E; ++I) {
Fix PR9955 by only attaching load memory operands to load instructions and similarly for stores. Now "make check" passes with the MachineVerifier forced on with the VerifyCoalescing option! llvm-svn: 131705
2011-05-20 01:44:34 +02:00
if ((*I)->isLoad()) {
if (mayLoad)
*MemRefsPos++ = *I;
} else if ((*I)->isStore()) {
if (mayStore)
*MemRefsPos++ = *I;
} else {
*MemRefsPos++ = *I;
}
}
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
cast<MachineSDNode>(Res)
Fix PR9955 by only attaching load memory operands to load instructions and similarly for stores. Now "make check" passes with the MachineVerifier forced on with the VerifyCoalescing option! llvm-svn: 131705
2011-05-20 01:44:34 +02:00
->setMemRefs(MemRefs, MemRefs + NumMemRefs);
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << " "
[CodeGen] Add some space optimized forms of EmitNode and MorphNodeTo that implicitly indicate the number of result VTs. This shaves about 16K off the X86 matching table taking it down to about 470K. Overall this reduces the llc binary size with all in-tree targets by about 40K. llvm-svn: 268365
2016-05-03 07:54:13 +02:00
<< (IsMorphNodeTo ? "Morphed" : "Created")
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
<< " node: "; Res->dump(CurDAG); dbgs() << "\n");
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// If this was a MorphNodeTo then we're completely done!
[CodeGen] Add some space optimized forms of EmitNode and MorphNodeTo that implicitly indicate the number of result VTs. This shaves about 16K off the X86 matching table taking it down to about 470K. Overall this reduces the llc binary size with all in-tree targets by about 40K. llvm-svn: 268365
2016-05-03 07:54:13 +02:00
if (IsMorphNodeTo) {
SDAG: Remove OPC_MarkGlueResults and associated logic. NFC This opcode never happens in practice, and yet the logic we have in place to handle it would be undefined behaviour if we ever executed it. Remove it rather than trying to refactor code that's never reached. llvm-svn: 268692
2016-05-06 00:37:45 +02:00
// Update chain uses.
SDAG: Drop a redundant replace and move the dead node removal closer. NFC llvm-svn: 271429
2016-06-01 22:55:26 +02:00
UpdateChains(Res, InputChain, ChainNodesMatched, true);
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
return;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
continue;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
case OPC_CompleteMatch: {
// The match has been completed, and any new nodes (if any) have been
// created. Patch up references to the matched dag to use the newly
// created nodes.
unsigned NumResults = MatcherTable[MatcherIndex++];
for (unsigned i = 0; i != NumResults; ++i) {
unsigned ResSlot = MatcherTable[MatcherIndex++];
if (ResSlot & 128)
ResSlot = GetVBR(ResSlot, MatcherTable, MatcherIndex);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Fix some assert messages to say the correct opcode name. Looks like one assert got copy and pasted to many places. llvm-svn: 192078
2013-10-07 00:38:19 +02:00
assert(ResSlot < RecordedNodes.size() && "Invalid CompleteMatch");
Rework passing parent pointers into complexpatterns, I forgot that complex patterns are matched after the entire pattern has a structural match, therefore the NodeStack isn't in a useful state when the actual call to the matcher happens. llvm-svn: 114489
2010-09-22 00:00:25 +02:00
SDValue Res = RecordedNodes[ResSlot].first;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
now that (parallel) is gone and a variety of bugs in targets are cleaned up, we can remove an old fixme. llvm-svn: 99741
2010-03-28 07:54:03 +02:00
assert(i < NodeToMatch->getNumValues() &&
NodeToMatch->getValueType(i) != MVT::Other &&
rename MVT::Flag to MVT::Glue. "Flag" is a terrible name for something that just glues two nodes together, even if it is sometimes used for flags. llvm-svn: 122310
2010-12-21 03:38:05 +01:00
NodeToMatch->getValueType(i) != MVT::Glue &&
now that (parallel) is gone and a variety of bugs in targets are cleaned up, we can remove an old fixme. llvm-svn: 99741
2010-03-28 07:54:03 +02:00
"Invalid number of results to complete!");
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
assert((NodeToMatch->getValueType(i) == Res.getValueType() ||
NodeToMatch->getValueType(i) == MVT::iPTR ||
Res.getValueType() == MVT::iPTR ||
NodeToMatch->getValueType(i).getSizeInBits() ==
getValueType().getSizeInBits() -> getValueSizeInBits() ; NFCI llvm-svn: 281493
2016-09-14 18:05:51 +02:00
Res.getValueSizeInBits()) &&
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
"invalid replacement");
CurDAG->ReplaceAllUsesOfValueWith(SDValue(NodeToMatch, i), Res);
}
SDAG: Remove OPC_MarkGlueResults and associated logic. NFC This opcode never happens in practice, and yet the logic we have in place to handle it would be undefined behaviour if we ever executed it. Remove it rather than trying to refactor code that's never reached. llvm-svn: 268692
2016-05-06 00:37:45 +02:00
// Update chain uses.
UpdateChains(NodeToMatch, InputChain, ChainNodesMatched, false);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
SDAG: Remove OPC_MarkGlueResults and associated logic. NFC This opcode never happens in practice, and yet the logic we have in place to handle it would be undefined behaviour if we ever executed it. Remove it rather than trying to refactor code that's never reached. llvm-svn: 268692
2016-05-06 00:37:45 +02:00
// If the root node defines glue, we need to update it to the glue result.
// TODO: This never happens in our tests and I think it can be removed /
// replaced with an assert, but if we do it this the way the change is
// NFC.
if (NodeToMatch->getValueType(NodeToMatch->getNumValues() - 1) ==
MVT::Glue &&
InputGlue.getNode())
CurDAG->ReplaceAllUsesOfValueWith(
SDValue(NodeToMatch, NodeToMatch->getNumValues() - 1), InputGlue);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
assert(NodeToMatch->use_empty() &&
"Didn't replace all uses of the node?");
SDAG: Don't leave dangling dead nodes after SelectCodeCommon Relying on the caller to clean up after we've replaced all uses of a node won't work when we've migrated to the `void Select(...)` API. llvm-svn: 268774
2016-05-06 20:42:16 +02:00
CurDAG->RemoveDeadNode(NodeToMatch);
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
return;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// If the code reached this point, then the match failed. See if there is
// another child to try in the current 'Scope', otherwise pop it until we
// find a case to check.
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << " Match failed at index " << CurrentOpcodeIndex << "\n");
Disable statistics on Release builds and move tests that depend on -stats. Summary: Statistics are still available in Release+Asserts (any +Asserts builds), and stats can also be turned on with LLVM_ENABLE_STATS. Move some of the FastISel stats that were moved under DEBUG() back out of DEBUG(), since stats are disabled across the board now. Many tests depend on grepping "-stats" output. Move those into a orig_dir/Stats/. so that they can be marked as unsupported when building without statistics. Differential Revision: http://llvm-reviews.chandlerc.com/D486 llvm-svn: 176733
2013-03-08 23:56:31 +01:00
++NumDAGIselRetries;
[CodeGen] Fix some Clang-tidy modernize and Include What You Use warnings; other minor fixes (NFC). This is preparation to reduce TargetInstrInfo.h dependencies. llvm-svn: 294084
2017-02-04 03:00:53 +01:00
while (true) {
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
if (MatchScopes.empty()) {
CannotYetSelect(NodeToMatch);
SDAG: Make SelectCodeCommon return void This means SelectCode unconditionally returns nullptr now. I'll follow up with a change to make that return void as well, but it seems best to keep that one very mechanical. This is part of the work to have Select return void instead of an SDNode *, which is in turn part of llvm.org/pr26808. llvm-svn: 269136
2016-05-11 00:58:26 +02:00
return;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
}
// Restore the interpreter state back to the point where the scope was
// formed.
MatchScope &LastScope = MatchScopes.back();
RecordedNodes.resize(LastScope.NumRecordedNodes);
NodeStack.clear();
NodeStack.append(LastScope.NodeStack.begin(), LastScope.NodeStack.end());
N = NodeStack.back();
if (LastScope.NumMatchedMemRefs != MatchedMemRefs.size())
MatchedMemRefs.resize(LastScope.NumMatchedMemRefs);
MatcherIndex = LastScope.FailIndex;
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Use dbgs() consistently for -debug printouts llvm-svn: 179894
2013-04-19 23:37:07 +02:00
DEBUG(dbgs() << " Continuing at " << MatcherIndex << "\n");
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
InputChain = LastScope.InputChain;
sdisel flag -> glue. llvm-svn: 122507
2010-12-23 18:13:18 +01:00
InputGlue = LastScope.InputGlue;
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
if (!LastScope.HasChainNodesMatched)
ChainNodesMatched.clear();
// Check to see what the offset is at the new MatcherIndex. If it is zero
// we have reached the end of this scope, otherwise we have another child
// in the current scope to try.
unsigned NumToSkip = MatcherTable[MatcherIndex++];
if (NumToSkip & 128)
NumToSkip = GetVBR(NumToSkip, MatcherTable, MatcherIndex);
// If we have another child in this scope to match, update FailIndex and
// try it.
if (NumToSkip != 0) {
LastScope.FailIndex = MatcherIndex+NumToSkip;
break;
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
hoist the new isel interpreter out of DAGISelHeader.h (which gets #included into the middle of each target's DAGISel class) into a .cpp file where it is only compiled once. llvm-svn: 97425
2010-02-28 23:37:22 +01:00
// End of this scope, pop it and try the next child in the containing
// scope.
MatchScopes.pop_back();
}
}
}
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
Change SelectCode's argument from SDValue to SDNode *, to make it more clear what information these functions are actually using. This is also a micro-optimization, as passing a SDNode * around is simpler than passing a { SDNode *, int } by value or reference. llvm-svn: 92564
2010-01-05 02:24:18 +01:00
void SelectionDAGISel::CannotYetSelect(SDNode *N) {
Revert "Introduce a string_ostream string builder facilty" Temporarily back out commits r211749, r211752 and r211754. llvm-svn: 211814
2014-06-27 00:52:05 +02:00
std::string msg;
raw_string_ostream Msg(msg);
improve "cannot yet select" errors a trivial amount: now they are just as useless, but at least a bit more gramatical llvm-svn: 122305
2010-12-21 03:07:03 +01:00
Msg << "Cannot select: ";
whitespace llvm-svn: 122539
2010-12-24 05:28:06 +01:00
inline CannotYetSelectIntrinsic into CannotYetSelect and simplify. llvm-svn: 97690
2010-03-04 01:21:16 +01:00
if (N->getOpcode() != ISD::INTRINSIC_W_CHAIN &&
N->getOpcode() != ISD::INTRINSIC_WO_CHAIN &&
N->getOpcode() != ISD::INTRINSIC_VOID) {
N->printrFull(Msg, CurDAG);
Add a getName function to MachineFunction. Use it in places that previously did getFunction()->getName(). Remove includes of Function.h that are no longer needed. llvm-svn: 162347
2012-08-22 08:07:19 +02:00
Msg << "\nIn function: " << MF->getName();
inline CannotYetSelectIntrinsic into CannotYetSelect and simplify. llvm-svn: 97690
2010-03-04 01:21:16 +01:00
} else {
bool HasInputChain = N->getOperand(0).getValueType() == MVT::Other;
unsigned iid =
cast<ConstantSDNode>(N->getOperand(HasInputChain))->getZExtValue();
if (iid < Intrinsic::num_intrinsics)
Fix some more asserts after r279466. That commit added a new version of Intrinsic::getName which should only be called when the intrinsic has no overloaded types. There are several debugging paths, such as SDNode::dump which are printing the name of the intrinsic but don't have the overloaded types. These paths should be ok to just print the name instead of crashing. The fix here is ultimately to just add a 'None' second argument as that calls the overload capable getName, which is less efficient, but this is a debugging path anyway, and not perf critical. Thanks to Björn Pettersson for pointing out that there were more crashes. llvm-svn: 279528
2016-08-23 18:23:45 +02:00
Msg << "intrinsic %" << Intrinsic::getName((Intrinsic::ID)iid, None);
inline CannotYetSelectIntrinsic into CannotYetSelect and simplify. llvm-svn: 97690
2010-03-04 01:21:16 +01:00
else if (const TargetIntrinsicInfo *TII = TM.getIntrinsicInfo())
Msg << "target intrinsic %" << TII->getName(iid);
else
Msg << "unknown intrinsic #" << iid;
}
rename llvm::llvm_report_error -> llvm::report_fatal_error llvm-svn: 100709
2010-04-08 00:58:41 +02:00
report_fatal_error(Msg.str());
Move some code from being emitted as boilerplate duplicated in every *ISelDAGToDAG.cpp to being regular code in SelectionDAGISel.cpp. llvm-svn: 85530
2009-10-29 23:30:23 +01:00
}
Drop 'const' llvm-svn: 36662
2007-05-03 03:11:54 +02:00
char SelectionDAGISel::ID = 0;
Reference in New Issue Copy Permalink