mirror of
https://github.com/RPCS3/llvm-mirror.git
synced 2024-11-24 03:33:20 +01:00
f8e085075d
6775 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Florian Hahn
|
72dee1bd0b |
Revert r334374 [TableGen] Move some shared_ptrs to avoid unnecessary copies (NFC).
This breaks some builders. llvm-svn: 334376 |
||
|
Florian Hahn
|
fd5d41cdb9 |
[TableGen] Move some shared_ptrs to avoid unnecessary copies (NFC).
Those changes were suggested post-commit for D47463. llvm-svn: 334374 |
||
Daniel Sanders
|
1799831af2 |
[tablegen] Improve performance on *GenRegisterInfo.inc by replacing SparseVector with BitVector. NFC
Summary: Generating X86GenRegisterInfo.inc and AArch64GenRegisterInfo.inc is 8-9% faster on my build. Reviewers: bogner, javed.absar Reviewed By: bogner Subscribers: llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D47907 llvm-svn: 334337 |
||
|
Florian Hahn
|
cc1e276c98 |
[TableGen] Make DAGInstruction own Pattern to avoid leaking it.
Reviewers: dsanders, craig.topper, stoklund, nhaehnle Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D47525 llvm-svn: 334275 |
||
Craig Topper
|
e36da260b7 |
[X86] Rename vy512mem->vy512xmem and vz256xmem->vz256mem.
The index size is represented by the letter after the 'v'. The number represents the memory size. If an 'x' appears after the number its means the index register can be from VR128X/VR256X instead of VR128/VR256. As vy512mem uses a VR256X index it should have an x. And vz256mem uses a VR512 index so it shouldn't have an x. I admit these names kind of suck and are confusing. llvm-svn: 334120 |
||
Hans Wennborg
|
f9ea32fcfd |
Relax shtest-run-at-line.py
The test was failing on Windows machines which had bash.exe on PATH (but not in the so called lit tools dir, containing cmp.exe, grep.exe etc.). The problem was that the outer lit invocation would load LLVMConfig from utils/lit/lit/llvm/config.py, which looks up the tools path with getToolsPath(). That has a surprising side effect of also setting bashPath, in our case setting it to empty. The outer lit invocation would thus configure the pdbg0 and pdbg1 substitutions based on not running with bash. But the inner lit invocation would not load LLVMConfig, so bash would be found on PATH, that would be used as external shell, and so the output wouldn't match pdbg0 and pdbg1. It seems weird to me that getBashPath() will return different results depending on whether getToolsPath() has been called before, but I also don't know how to fix it properly. This commit just relaxes the test case, because there doesn't seem to be much point in testing for the exact syntax of the run file as long as it works. (See https://crbug.com/850023) llvm-svn: 334100 |
||
Kristof Beyls
|
211bc0b9fa |
Avoid UnicodeEncodeError on non-ascii reviewer names
... by using unicode instead of byte strings where non-ascii strings can be formatted in. llvm-svn: 334098 |
||
Vlad Tsyrklevich
|
48f4d3ae20 |
[Analyzer] Fix the Z3 lit test config
Summary: The '%analyze' extra_args config argument seems to have been erroneously deleted in r315627 disabling Z3 tests for the clang analyzer. Add the flag back. Reviewers: george.karpenkov, NoQ, ddcc Reviewed By: george.karpenkov Subscribers: xazax.hun, szepet, delcypher, a.sidorin, llvm-commits Differential Revision: https://reviews.llvm.org/D47722 llvm-svn: 334066 |
||
Greg Bedwell
|
d0b4720ec4 |
[UpdateTestChecks] Error if --llvm-mca-binary gets an empty string
If the command line was mistyped like:
./update_mca_test_checks.py --llvm-mca-binary= /path/to/llvm-mca *.s
^-- extra whitespace
then /path/to/llvm-mca would get treated by argparse as a test-path
pattern and could actually be opened in write mode and overwritten.
llvm-svn: 334029
|
||
Nicolai Haehnle
|
7eea83d527 |
TableGen/DAGPatterns: Allow bit constants in addition to int constants
Summary: Implicit casting is a simple quality of life improvement. Change-Id: I3d2b31b8b8f12cbb1e84f691e359fa713a9c4b42 Reviewers: tra, simon_tatham, craig.topper, MartinO, arsenm Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D47432 llvm-svn: 333904 |
||
|
Greg Bedwell
|
75a3de70a1 |
[llvm-mca][UpdateTestChecks] Prevent an IndexError being raised when given empty input
llvm-svn: 333894 |
||
Roman Lebedev
|
98a53476a7 |
[llvm-mca] Make sure not to end the test files with an empty line.
Summary: It's super irritating. [properly configured] git client then complains about that double-newline, and you have to use `--force` to ignore the warning, since even if you fix it manually, it will be reintroduced the very next runtime :/ Reviewers: RKSimon, andreadb, courbet, craig.topper, javed.absar, gbedwell Reviewed By: gbedwell Subscribers: javed.absar, tschuett, gbedwell, llvm-commits Differential Revision: https://reviews.llvm.org/D47697 llvm-svn: 333887 |
||
Simon Pilgrim
|
2196dd6cfa |
[Utils][X86] Help update_llc_test_checks.py to recognise retl/retq to reduce CHECK duplication (PR35003)
This patch replaces the --x86_extra_scrub command line argument to automatically support a second level of regex-scrubbing if it improves the matching of nearly-identical code patterns. The argument '--extra_scrub' is there now to force extra matching if required.
This is mostly useful to help us share 32-bit/64-bit x86 vector tests which only differs by retl/retq instructions, but any scrubber can now technically support this, meaning test checks don't have to be needlessly obfuscated.
I've updated some of the existing checks that had been manually run with --x86_extra_scrub, to demonstrate the extra "ret{{[l|q]}}" scrub now only happens when useful, and re-run the sse42-intrinsics file to show extra matches - most sse/avx intrinsics files should be able to now share 32/64 checks.
Tested with the opt/analysis scripts as well which share common code - AFAICT the other update scripts use their own versions.
Differential Revision: https://reviews.llvm.org/D47485
llvm-svn: 333749
|
||
Andrea Di Biagio
|
09b5879706 |
[MCSchedule] Add the ability to compute the latency and throughput information for MCInst.
This patch extends the MCSchedModel API with new methods that can be used to obtain the latency and reciprocal througput information for an MCInst. Scheduling models have recently gained the ability to resolve variant scheduling classes associated with MCInst objects. Before, models were only able to resolve a variant scheduling class from a MachineInstr object. This patch is mainly required by D47374 to avoid regressing a pair of x86 specific -print-schedule tests for btver2. Patch D47374 introduces a new variant class to teach the btver scheduling model (x86 target) how to correctly compute the latency profile for some zero-idioms using the new scheduling predicates. The new methods added by this patch would be mainly used by llc when flag -print-schedule is specified. In particular, tests that contain inline assembly require that code is parsed at code emission stage into a sequence of MCInst. That forces the print-schedule functionality to query the latency/rthroughput information for MCInst instructions too. If we don't expose this new API, then we lose "-print-schedule" test coverage as soon as variant scheduling classes are added to the x86 models. The tablegen SubtargetEmitter changes teaches how to query latency profile information using a object that derives from TargetSubtargetInfo. Note that this should really have been part of r333286. To avoid code duplication, the logic that "resolves" variant scheduling classes for MCInst, has been moved to a common place in MC. That logic is used by the "resolveVariantSchedClass" methods redefined in override by the tablegen'd GenSubtargetInfo classes. Differential Revision: https://reviews.llvm.org/D47536 llvm-svn: 333650 |
||
Joel E. Denny
|
a04841e104 |
[lit] Fix windows cmd.exe test config for r333620
llvm-svn: 333630 |
||
|
Joel E. Denny
|
bd389c40fd |
[lit] Terminate ": RUN at line N" with ";" not "&&"
This fixes projects/compiler-rt/test/fuzzer/sigusr.test, which was broken by r333614. The trouble was that "&&" changes the command for which "$!" gives the pid. llvm-svn: 333620 |
||
|
Joel E. Denny
|
3204e36001 |
[lit] Report line number for failed RUN command
(Relands r333584, reverted in 333592.) When debugging test failures with -vv (or -v in the case of the internal shell), this makes it easier to locate the RUN line that failed. For example, clang's test/Driver/linux-ld.c has 892 total RUN lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines after concatenation for line continuations. When reading the generated shell script, this also makes it easier to locate the RUN line that produced each command. To support reporting RUN line numbers in the case of the internal shell, this patch extends the internal shell to support the null command, ":", except pipelines are not supported. To support reporting RUN line numbers in the case of windows cmd.exe as the external shell, this patch extends -vv to set "echo on" instead of "echo off" in bat files. (Support for windows cmd.exe as a lit external shell will likely be dropped later, but I found out too late.) Reviewed By: delcypher, asmith, stella.stamenova, jmorse, lebedev.ri, rnk Differential Revision: https://reviews.llvm.org/D44598 llvm-svn: 333614 |
||
|
Joel E. Denny
|
664a77c58b |
Revert r333584: [lit] Report line number for failed RUN command
It breaks test-suite. llvm-svn: 333592 |
||
|
Florian Hahn
|
207dd7516e |
[TableGen] Avoid leaking TreePatternNodes by using shared_ptr.
By using std::shared_ptr for TreePatternNode, we can avoid leaking them. Reviewers: craig.topper, dsanders, stoklund, tstellar, zturner Reviewed By: dsanders Differential Revision: https://reviews.llvm.org/D47463 llvm-svn: 333591 |
||
|
Joel E. Denny
|
f5a15f4aa3 |
[lit] Report line number for failed RUN command
(Relands r330755 (reverted in r330848) with fix for PR37239.) When debugging test failures with -vv (or -v in the case of the internal shell), this makes it easier to locate the RUN line that failed. For example, clang's test/Driver/linux-ld.c has 892 total RUN lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines after concatenation for line continuations. When reading the generated shell script, this also makes it easier to locate the RUN line that produced each command. To support reporting RUN line numbers in the case of the internal shell, this patch extends the internal shell to support the null command, ":", except pipelines are not supported. To support reporting RUN line numbers in the case of windows cmd.exe as the external shell, this patch extends -vv to set "echo on" instead of "echo off" in bat files. (Support for windows cmd.exe as a lit external shell will likely be dropped later, but I found out too late.) Reviewed By: delcypher, asmith, stella.stamenova, jmorse, lebedev.ri, rnk Differential Revision: https://reviews.llvm.org/D44598 llvm-svn: 333584 |
||
|
Florian Hahn
|
7618648e0d |
[TableGen] Use explicit constructor for InstMemo
This should fix a few buildbot failures with old GCC versions. llvm-svn: 333448 |
||
|
Florian Hahn
|
c24f7d921e |
[TableGen] Fix leaking of PhysRegInputs.
Instead of dynamically allocating the vector for PhysRegs, we can allocate it on the stack and move it into InstructionMemo. Reviewers: mcrosier, craig.topper, RKSimon, dsanders Reviewed By: dsanders Differential Revision: https://reviews.llvm.org/D47461 llvm-svn: 333438 |
||
|
Florian Hahn
|
b1baaee29e |
[TableGen] Fix leaking synthesized registers.
By keeping track of unique_ptrs to the synthesized definitions in CodeGenRegBank we avoid leaking them. Reviewers: dsanders, kparzysz, stoklund Reviewed By: dsanders Differential Revision: https://reviews.llvm.org/D47462 llvm-svn: 333434 |
||
|
Andrea Di Biagio
|
8e881f054d |
[Tablegen] Avoid generating empty switch statements. NFC
This fixes an MSVC warning (warning C4065: switch statement contains 'default' but no 'case' labels) introduced with revision 333293. llvm-svn: 333363 |
||
|
Andrea Di Biagio
|
f8f1287b06 |
[Tablegen][SubtargetEmitter] Add a default case to the auto-generated switch in MCSubtargetInfo::resolveVariantSchedClass(). NFC
llvm-svn: 333293 |
||
|
Andrea Di Biagio
|
d69daf9e61 |
[RFC][Patch 2/3] Add a MCSubtargetInfo hook to resolve variant scheduling classes.
This patch is the second of a sequence of three patches related to LLVM-dev RFC "MC support for varinat scheduling classes". https://lists.llvm.org/pipermail/llvm-dev/2018-May/123181.html The goal of this patch is to enable the resolution of variant classes in MC with the help of a new method named `MCSubtargetInfo::resolveVariantSchedClass()`. This patch also teaches the SubtargetEmitter how to automatically generate the definition of method resolveVariantSchedClass(). That definition is emitted within a sub-class of MCSubtargetInfo named XXXGenMCSubtargetInfo (where XXX is the name of the Target). Differential Revision: https://reviews.llvm.org/D47077 llvm-svn: 333286 |
||
|
Andrea Di Biagio
|
c26fd3adc6 |
[RFC][Patch 1/3] Add a new class of predicates for variant scheduling classes.
This patch is the first of a sequence of three patches described by the LLVM-dev RFC "MC support for variant scheduling classes". http://lists.llvm.org/pipermail/llvm-dev/2018-May/123181.html The goal of this patch is to introduce a new class of scheduling predicates for SchedReadVariant and SchedWriteVariant. An MCSchedPredicate can be used instead of a normal SchedPredicate to model checks on the instruction (either a MachineInstr or a MCInst). Internally, an MCSchedPredicate encapsulates an MCInstPredicate definition. MCInstPredicate allows the definition of expressions with a well-known semantic, that can be used to generate code for both MachineInstr and MCInst. This is the first step toward teaching to tools like lllvm-mca how to resolve variant scheduling classes. Differential Revision: https://reviews.llvm.org/D46695 llvm-svn: 333282 |
||
Chris Matthews
|
0b61b91c62 |
Use quoteattr to ensure we make well formed attributes
We were making malformed XML on tests with ' in the name. Switch to using saxutils to set all of our attributes, so it can handle quotes etc correctly. llvm-svn: 333249 |
||
|
Greg Bedwell
|
fbe9943602 |
[UpdateTestChecks] Improved update_mca_test_checks block analysis
Previously update_mca_test_checks worked entirely at "block" level where a block is some sequence of lines delimited by at least one empty line. This generally worked well, but could sometimes lead to excessive repetition of check lines for various prefixes if some block was almost identical between prefixes, but not quite (for example, due to a different dispatch width in the otherwise identical summary views). This new analyis attempts to split blocks further in the case where the following conditions are met: a) There is some prefix common to every RUN line (typically 'ALL'). b) The first line of the block is common to the output with every prefix. c) The block has the same number of lines for the output with every prefix. Also, regenerated all llvm-mca test files with the following command: update_mca_test_checks.py "../test/tools/llvm-mca/*/*.s" "../test/tools/llvm-mca/*/*/*.s" The new analysis showed a "multiple lines not disambiguated by prefixes" warning for test "AArch64/Exynos/scheduler-queue-usage.s" so I've also added some explicit prefixes to each of the RUN lines in that test. Differential Revision: https://reviews.llvm.org/D47321 llvm-svn: 333204 |
||
Roman Tereshin
|
3066add2d0 |
[GlobalISel][InstructionSelect] Switching over root LLTs, perf patch 10
This patch continues a series of patches started by r332907 (reapplied as r332917). In this commit we introduce new matching opcode for the MatchTable: GIM_SwitchType, similar to GIM_SwitchOpcode, and use it to switch over LLTs of def operands of root instructions on the 2nd level of the MatchTable within GIM_SwitchOpcode's cases. This is expected to decrease time GlobalISel spends in its InstructionSelect pass by about 6.5% for an -O0 build as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64 (cross-compile on x86). Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 333146 |
||
|
Roman Tereshin
|
14fdf4cba1 |
[GlobalISel][InstructionSelect] Moving Reg Bank Checks forward, perf patch 9
This patch continues a series of patches started by r332907 (reapplied as r332917). In this commit we move register bank checks back from epilogue of every rule matcher to a position locally close to the rest of the checks for a particular (nested) instruction. This increases the number of common conditions within 2nd level groups. This is expected to decrease time GlobalISel spends in its InstructionSelect pass by about 2% for an -O0 build as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64 (cross-compile on x86). Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 333144 |
||
|
Roman Tereshin
|
8c9a32911c |
[GlobalISel][InstructionSelect] Maximizing # of Group's common conditions, perf patch 8
This patch continues a series of patches started by r332907 (reapplied as r332917). In this commit we greedily stuff 2nd level GroupMatcher's common conditions with as many predicates as possible. This is purely post-processing and it doesn't change which rules are put into the groups in the first place: that decision is made by looking at the first common predicate only. The compile time improvements are minor and well within error margin, however, it's highly improbable that this transformation could pessimize performance, thus I'm still committing it for potential gains for targets not implementing GlobalISel yet and out of tree targets. Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 333139 |
||
|
Roman Tereshin
|
0d915067bd |
[GlobalISel][Tablegen] Assign small opcodes to pseudos
Sort pseudo instructions first while emitting enum's for target instructions info. That puts them close to each other and to generic G_* opcodes for GlobalISel. This makes it easier to build small jump tables over opcodes that could be directly embedded into MatchTable's Tablegen'erated for GlobalISel's InstructionSelect. Reviewed By: bogner Differential Revision: https://reviews.llvm.org/D47240 llvm-svn: 333135 |
||
|
Roman Tereshin
|
069717c1b2 |
[GlobalISel][InstructionSelect] Sorting MatchTable's 2nd level by root LLT, perf patch 7
This patch continues a series of patches started by r332907 (reapplied as r332917). In this commit we sort rules within their 2nd level by the type check on def operand of the root instruction, which allows for better nesting grouping on the level. This is expected to decrease time GlobalISel spends in its InstructionSelect pass by roughly 22% for an -O0 build as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64 (cross-compile on x86). Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 333131 |
||
|
Roman Tereshin
|
70950ef979 |
[Tablegen] Tidying up InstRegexOp a little, NFC
Differential Review: https://reviews.llvm.org/D47240 llvm-svn: 333121 |
||
|
Roman Tereshin
|
00a81de35d |
[GlobalISel][InstructionSelect] Moving type checks forward, perf patch 6
This patch continues a series of patches started by r332907 (reapplied as r332917) In this commit we sort type checks towards the beginning of every rule within the MatchTable as they fail often and it's best to fail early. This is expected to decrease time GlobalISel spends in its InstructionSelect pass by roughly 7% for an -O0 build as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64. The amalgamation is a large single-file C-source that makes compiler backend performance improvements to stand out from frontend. It's also a part of CTMark. Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 333114 |
||
Petar Jovanovic
|
8c311a61bc |
[X86][MIPS][ARM] New machine instruction property 'isMoveReg'
This property is needed in order to follow values movement between registers. This property is used in TII to implement method that returns true if simple copy like instruction is recognized, along with source and destination machine operands. Patch by Nikola Prica. Differential Revision: https://reviews.llvm.org/D45204 llvm-svn: 333093 |
||
Nicola Zaghen
|
80a94d5002 |
Remove DEBUG macro.
Now that the LLVM_DEBUG() macro landed on the various sub-projects the DEBUG macro can be removed. Also change the new uses of DEBUG to LLVM_DEBUG. Differential Revision: https://reviews.llvm.org/D46952 llvm-svn: 333091 |
||
|
Roman Tereshin
|
3e1ac7e466 |
[GlobalISel][InstructionSelect] MatchTable second level grouping, perf patch 5
This patch continues a series of patches started by r332907 (reapplied as r332917) In this commit we start grouping rules with common first condition on the second level of the table. This is expected to decrease time GlobalISel spends in its InstructionSelect pass by roughly 13% for an -O0 build as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64. Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 333053 |
||
|
Roman Tereshin
|
16be0df7da |
[GlobalISel][InstructionSelect] Switching MatchTable over opcodes, perf patch 4
This patch continues a series of patches started by r332907 (reapplied as r332917) In this commit we introduce a new matching opcode GIM_SwitchOpcode that implements a jump table over opcodes and start emitting them for root instructions. This is expected to decrease time GlobalISel spends in its InstructionSelect pass by roughly 20% for an -O0 build as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64. To some degree, we assume here that the opcodes form a dense set, which is true at the moment for all upstream targets given the limitations of our rule importing mechanism. It might not be true for out of tree targets, specifically due to pseudo's. If so, we might noticeably increase the size of the MatchTable with this patch due to padding zeros. This will be addressed later. Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 333017 |
||
|
Roman Tereshin
|
98360ae694 |
[GlobalISel][InstructionSelect] Sorting MatchTable's first level by opcodes and num operands, perf patch 3
This patch continues a series of patches started by r332907 (reapplied as r332917) In this commit we start sorting the rules by the opcode first, and if the same, by the number of operands of the root instructions. This allows better grouping and safe as patterns with different opcodes are mutually exclusive. This is expected to decrease time GlobalISel spends in its InstructionSelect pass by roughly 18% for an -O0 build as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64. I'm also removing RuleMatcher::getFirstConditionAsRootType() function here and moving it to a later patch within the series as it's not used yet and was causing a warning on sanitizer-ppc64le-linux bot. Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 333001 |
||
|
Roman Tereshin
|
9f288d1be9 |
Reverting 332999 to get it a proper commit message
llvm-svn: 333000 |
||
|
Roman Tereshin
|
0b04e29b2c |
diff --git a/utils/TableGen/GlobalISelEmitter.cpp b/utils/TableGen/GlobalISelEmitter.cpp
index cdc9df7bf6b..be08165a200 100644
--- a/utils/TableGen/GlobalISelEmitter.cpp
+++ b/utils/TableGen/GlobalISelEmitter.cpp
@@ -1,4531 +1,4539 @@
//===- GlobalISelEmitter.cpp - Generate an instruction selector -----------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
/// \file
/// This tablegen backend emits code for use by the GlobalISel instruction
/// selector. See include/llvm/CodeGen/TargetGlobalISel.td.
///
/// This file analyzes the patterns recognized by the SelectionDAGISel tablegen
/// backend, filters out the ones that are unsupported, maps
/// SelectionDAG-specific constructs to their GlobalISel counterpart
/// (when applicable: MVT to LLT; SDNode to generic Instruction).
///
/// Not all patterns are supported: pass the tablegen invocation
/// "-warn-on-skipped-patterns" to emit a warning when a pattern is skipped,
/// as well as why.
///
/// The generated file defines a single method:
/// bool <Target>InstructionSelector::selectImpl(MachineInstr &I) const;
/// intended to be used in InstructionSelector::select as the first-step
/// selector for the patterns that don't require complex C++.
///
/// FIXME: We'll probably want to eventually define a base
/// "TargetGenInstructionSelector" class.
///
//===----------------------------------------------------------------------===//
#include "CodeGenDAGPatterns.h"
#include "SubtargetFeatureInfo.h"
#include "llvm/ADT/Optional.h"
#include "llvm/ADT/SmallSet.h"
#include "llvm/ADT/Statistic.h"
#include "llvm/Support/CodeGenCoverage.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/Error.h"
#include "llvm/Support/LowLevelTypeImpl.h"
#include "llvm/Support/MachineValueType.h"
#include "llvm/Support/ScopedPrinter.h"
#include "llvm/TableGen/Error.h"
#include "llvm/TableGen/Record.h"
#include "llvm/TableGen/TableGenBackend.h"
#include <numeric>
#include <string>
using namespace llvm;
#define DEBUG_TYPE "gisel-emitter"
STATISTIC(NumPatternTotal, "Total number of patterns");
STATISTIC(NumPatternImported, "Number of patterns imported from SelectionDAG");
STATISTIC(NumPatternImportsSkipped, "Number of SelectionDAG imports skipped");
STATISTIC(NumPatternsTested, "Number of patterns executed according to coverage information");
STATISTIC(NumPatternEmitted, "Number of patterns emitted");
cl::OptionCategory GlobalISelEmitterCat("Options for -gen-global-isel");
static cl::opt<bool> WarnOnSkippedPatterns(
"warn-on-skipped-patterns",
cl::desc("Explain why a pattern was skipped for inclusion "
"in the GlobalISel selector"),
cl::init(false), cl::cat(GlobalISelEmitterCat));
static cl::opt<bool> GenerateCoverage(
"instrument-gisel-coverage",
cl::desc("Generate coverage instrumentation for GlobalISel"),
cl::init(false), cl::cat(GlobalISelEmitterCat));
static cl::opt<std::string> UseCoverageFile(
"gisel-coverage-file", cl::init(""),
cl::desc("Specify file to retrieve coverage information from"),
cl::cat(GlobalISelEmitterCat));
static cl::opt<bool> OptimizeMatchTable(
"optimize-match-table",
cl::desc("Generate an optimized version of the match table"),
cl::init(true), cl::cat(GlobalISelEmitterCat));
namespace {
//===- Helper functions ---------------------------------------------------===//
/// Get the name of the enum value used to number the predicate function.
std::string getEnumNameForPredicate(const TreePredicateFn &Predicate) {
return "GIPFP_" + Predicate.getImmTypeIdentifier().str() + "_" +
Predicate.getFnName();
}
/// Get the opcode used to check this predicate.
std::string getMatchOpcodeForPredicate(const TreePredicateFn &Predicate) {
return "GIM_Check" + Predicate.getImmTypeIdentifier().str() + "ImmPredicate";
}
/// This class stands in for LLT wherever we want to tablegen-erate an
/// equivalent at compiler run-time.
class LLTCodeGen {
private:
LLT Ty;
public:
LLTCodeGen() = default;
LLTCodeGen(const LLT &Ty) : Ty(Ty) {}
std::string getCxxEnumValue() const {
std::string Str;
raw_string_ostream OS(Str);
emitCxxEnumValue(OS);
return OS.str();
}
void emitCxxEnumValue(raw_ostream &OS) const {
if (Ty.isScalar()) {
OS << "GILLT_s" << Ty.getSizeInBits();
return;
}
if (Ty.isVector()) {
OS << "GILLT_v" << Ty.getNumElements() << "s" << Ty.getScalarSizeInBits();
return;
}
if (Ty.isPointer()) {
OS << "GILLT_p" << Ty.getAddressSpace();
if (Ty.getSizeInBits() > 0)
OS << "s" << Ty.getSizeInBits();
return;
}
llvm_unreachable("Unhandled LLT");
}
void emitCxxConstructorCall(raw_ostream &OS) const {
if (Ty.isScalar()) {
OS << "LLT::scalar(" << Ty.getSizeInBits() << ")";
return;
}
if (Ty.isVector()) {
OS << "LLT::vector(" << Ty.getNumElements() << ", "
<< Ty.getScalarSizeInBits() << ")";
return;
}
if (Ty.isPointer() && Ty.getSizeInBits() > 0) {
OS << "LLT::pointer(" << Ty.getAddressSpace() << ", "
<< Ty.getSizeInBits() << ")";
return;
}
llvm_unreachable("Unhandled LLT");
}
const LLT &get() const { return Ty; }
/// This ordering is used for std::unique() and llvm::sort(). There's no
/// particular logic behind the order but either A < B or B < A must be
/// true if A != B.
bool operator<(const LLTCodeGen &Other) const {
if (Ty.isValid() != Other.Ty.isValid())
return Ty.isValid() < Other.Ty.isValid();
if (!Ty.isValid())
return false;
if (Ty.isVector() != Other.Ty.isVector())
return Ty.isVector() < Other.Ty.isVector();
if (Ty.isScalar() != Other.Ty.isScalar())
return Ty.isScalar() < Other.Ty.isScalar();
if (Ty.isPointer() != Other.Ty.isPointer())
return Ty.isPointer() < Other.Ty.isPointer();
if (Ty.isPointer() && Ty.getAddressSpace() != Other.Ty.getAddressSpace())
return Ty.getAddressSpace() < Other.Ty.getAddressSpace();
if (Ty.isVector() && Ty.getNumElements() != Other.Ty.getNumElements())
return Ty.getNumElements() < Other.Ty.getNumElements();
return Ty.getSizeInBits() < Other.Ty.getSizeInBits();
}
bool operator==(const LLTCodeGen &B) const { return Ty == B.Ty; }
};
// Track all types that are used so we can emit the corresponding enum.
std::set<LLTCodeGen> KnownTypes;
class InstructionMatcher;
/// Convert an MVT to an equivalent LLT if possible, or the invalid LLT() for
/// MVTs that don't map cleanly to an LLT (e.g., iPTR, *any, ...).
static Optional<LLTCodeGen> MVTToLLT(MVT::SimpleValueType SVT) {
MVT VT(SVT);
if (VT.isVector() && VT.getVectorNumElements() != 1)
return LLTCodeGen(
LLT::vector(VT.getVectorNumElements(), VT.getScalarSizeInBits()));
if (VT.isInteger() || VT.isFloatingPoint())
return LLTCodeGen(LLT::scalar(VT.getSizeInBits()));
return None;
}
static std::string explainPredicates(const TreePatternNode *N) {
std::string Explanation = "";
StringRef Separator = "";
for (const auto &P : N->getPredicateFns()) {
Explanation +=
(Separator + P.getOrigPatFragRecord()->getRecord()->getName()).str();
Separator = ", ";
if (P.isAlwaysTrue())
Explanation += " always-true";
if (P.isImmediatePattern())
Explanation += " immediate";
if (P.isUnindexed())
Explanation += " unindexed";
if (P.isNonExtLoad())
Explanation += " non-extload";
if (P.isAnyExtLoad())
Explanation += " extload";
if (P.isSignExtLoad())
Explanation += " sextload";
if (P.isZeroExtLoad())
Explanation += " zextload";
if (P.isNonTruncStore())
Explanation += " non-truncstore";
if (P.isTruncStore())
Explanation += " truncstore";
if (Record *VT = P.getMemoryVT())
Explanation += (" MemVT=" + VT->getName()).str();
if (Record *VT = P.getScalarMemoryVT())
Explanation += (" ScalarVT(MemVT)=" + VT->getName()).str();
if (P.isAtomicOrderingMonotonic())
Explanation += " monotonic";
if (P.isAtomicOrderingAcquire())
Explanation += " acquire";
if (P.isAtomicOrderingRelease())
Explanation += " release";
if (P.isAtomicOrderingAcquireRelease())
Explanation += " acq_rel";
if (P.isAtomicOrderingSequentiallyConsistent())
Explanation += " seq_cst";
if (P.isAtomicOrderingAcquireOrStronger())
Explanation += " >=acquire";
if (P.isAtomicOrderingWeakerThanAcquire())
Explanation += " <acquire";
if (P.isAtomicOrderingReleaseOrStronger())
Explanation += " >=release";
if (P.isAtomicOrderingWeakerThanRelease())
Explanation += " <release";
}
return Explanation;
}
std::string explainOperator(Record *Operator) {
if (Operator->isSubClassOf("SDNode"))
return (" (" + Operator->getValueAsString("Opcode") + ")").str();
if (Operator->isSubClassOf("Intrinsic"))
return (" (Operator is an Intrinsic, " + Operator->getName() + ")").str();
if (Operator->isSubClassOf("ComplexPattern"))
return (" (Operator is an unmapped ComplexPattern, " + Operator->getName() +
")")
.str();
if (Operator->isSubClassOf("SDNodeXForm"))
return (" (Operator is an unmapped SDNodeXForm, " + Operator->getName() +
")")
.str();
return (" (Operator " + Operator->getName() + " not understood)").str();
}
/// Helper function to let the emitter report skip reason error messages.
static Error failedImport(const Twine &Reason) {
return make_error<StringError>(Reason, inconvertibleErrorCode());
}
static Error isTrivialOperatorNode(const TreePatternNode *N) {
std::string Explanation = "";
std::string Separator = "";
bool HasUnsupportedPredicate = false;
for (const auto &Predicate : N->getPredicateFns()) {
if (Predicate.isAlwaysTrue())
continue;
if (Predicate.isImmediatePattern())
continue;
if (Predicate.isNonExtLoad() || Predicate.isAnyExtLoad() ||
Predicate.isSignExtLoad() || Predicate.isZeroExtLoad())
continue;
if (Predicate.isNonTruncStore())
continue;
if (Predicate.isLoad() && Predicate.getMemoryVT())
continue;
if (Predicate.isLoad() || Predicate.isStore()) {
if (Predicate.isUnindexed())
continue;
}
if (Predicate.isAtomic() && Predicate.getMemoryVT())
continue;
if (Predicate.isAtomic() &&
(Predicate.isAtomicOrderingMonotonic() ||
Predicate.isAtomicOrderingAcquire() ||
Predicate.isAtomicOrderingRelease() ||
Predicate.isAtomicOrderingAcquireRelease() ||
Predicate.isAtomicOrderingSequentiallyConsistent() ||
Predicate.isAtomicOrderingAcquireOrStronger() ||
Predicate.isAtomicOrderingWeakerThanAcquire() ||
Predicate.isAtomicOrderingReleaseOrStronger() ||
Predicate.isAtomicOrderingWeakerThanRelease()))
continue;
HasUnsupportedPredicate = true;
Explanation = Separator + "Has a predicate (" + explainPredicates(N) + ")";
Separator = ", ";
Explanation += (Separator + "first-failing:" +
Predicate.getOrigPatFragRecord()->getRecord()->getName())
.str();
break;
}
if (!HasUnsupportedPredicate)
return Error::success();
return failedImport(Explanation);
}
static Record *getInitValueAsRegClass(Init *V) {
if (DefInit *VDefInit = dyn_cast<DefInit>(V)) {
if (VDefInit->getDef()->isSubClassOf("RegisterOperand"))
return VDefInit->getDef()->getValueAsDef("RegClass");
if (VDefInit->getDef()->isSubClassOf("RegisterClass"))
return VDefInit->getDef();
}
return nullptr;
}
std::string
getNameForFeatureBitset(const std::vector<Record *> &FeatureBitset) {
std::string Name = "GIFBS";
for (const auto &Feature : FeatureBitset)
Name += ("_" + Feature->getName()).str();
return Name;
}
//===- MatchTable Helpers -------------------------------------------------===//
class MatchTable;
/// A record to be stored in a MatchTable.
///
/// This class represents any and all output that may be required to emit the
/// MatchTable. Instances are most often configured to represent an opcode or
/// value that will be emitted to the table with some formatting but it can also
/// represent commas, comments, and other formatting instructions.
struct MatchTableRecord {
enum RecordFlagsBits {
MTRF_None = 0x0,
/// Causes EmitStr to be formatted as comment when emitted.
MTRF_Comment = 0x1,
/// Causes the record value to be followed by a comma when emitted.
MTRF_CommaFollows = 0x2,
/// Causes the record value to be followed by a line break when emitted.
MTRF_LineBreakFollows = 0x4,
/// Indicates that the record defines a label and causes an additional
/// comment to be emitted containing the index of the label.
MTRF_Label = 0x8,
/// Causes the record to be emitted as the index of the label specified by
/// LabelID along with a comment indicating where that label is.
MTRF_JumpTarget = 0x10,
/// Causes the formatter to add a level of indentation before emitting the
/// record.
MTRF_Indent = 0x20,
/// Causes the formatter to remove a level of indentation after emitting the
/// record.
MTRF_Outdent = 0x40,
};
/// When MTRF_Label or MTRF_JumpTarget is used, indicates a label id to
/// reference or define.
unsigned LabelID;
/// The string to emit. Depending on the MTRF_* flags it may be a comment, a
/// value, a label name.
std::string EmitStr;
private:
/// The number of MatchTable elements described by this record. Comments are 0
/// while values are typically 1. Values >1 may occur when we need to emit
/// values that exceed the size of a MatchTable element.
unsigned NumElements;
public:
/// A bitfield of RecordFlagsBits flags.
unsigned Flags;
/// The actual run-time value, if known
int64_t RawValue;
MatchTableRecord(Optional<unsigned> LabelID_, StringRef EmitStr,
unsigned NumElements, unsigned Flags,
int64_t RawValue = std::numeric_limits<int64_t>::min())
: LabelID(LabelID_.hasValue() ? LabelID_.getValue() : ~0u),
EmitStr(EmitStr), NumElements(NumElements), Flags(Flags),
RawValue(RawValue) {
assert((!LabelID_.hasValue() || LabelID != ~0u) &&
"This value is reserved for non-labels");
}
MatchTableRecord(const MatchTableRecord &Other) = default;
MatchTableRecord(MatchTableRecord &&Other) = default;
/// Useful if a Match Table Record gets optimized out
void turnIntoComment() {
Flags |= MTRF_Comment;
Flags &= ~MTRF_CommaFollows;
NumElements = 0;
}
/// For Jump Table generation purposes
bool operator<(const MatchTableRecord &Other) const {
return RawValue < Other.RawValue;
}
int64_t getRawValue() const { return RawValue; }
void emit(raw_ostream &OS, bool LineBreakNextAfterThis,
const MatchTable &Table) const;
unsigned size() const { return NumElements; }
};
class Matcher;
/// Holds the contents of a generated MatchTable to enable formatting and the
/// necessary index tracking needed to support GIM_Try.
class MatchTable {
/// An unique identifier for the table. The generated table will be named
/// MatchTable${ID}.
unsigned ID;
/// The records that make up the table. Also includes comments describing the
/// values being emitted and line breaks to format it.
std::vector<MatchTableRecord> Contents;
/// The currently defined labels.
DenseMap<unsigned, unsigned> LabelMap;
/// Tracks the sum of MatchTableRecord::NumElements as the table is built.
unsigned CurrentSize = 0;
/// A unique identifier for a MatchTable label.
unsigned CurrentLabelID = 0;
/// Determines if the table should be instrumented for rule coverage tracking.
bool IsWithCoverage;
public:
static MatchTableRecord LineBreak;
static MatchTableRecord Comment(StringRef Comment) {
return MatchTableRecord(None, Comment, 0, MatchTableRecord::MTRF_Comment);
}
static MatchTableRecord Opcode(StringRef Opcode, int IndentAdjust = 0) {
unsigned ExtraFlags = 0;
if (IndentAdjust > 0)
ExtraFlags |= MatchTableRecord::MTRF_Indent;
if (IndentAdjust < 0)
ExtraFlags |= MatchTableRecord::MTRF_Outdent;
return MatchTableRecord(None, Opcode, 1,
MatchTableRecord::MTRF_CommaFollows | ExtraFlags);
}
static MatchTableRecord NamedValue(StringRef NamedValue) {
return MatchTableRecord(None, NamedValue, 1,
MatchTableRecord::MTRF_CommaFollows);
}
static MatchTableRecord NamedValue(StringRef NamedValue, int64_t RawValue) {
return MatchTableRecord(None, NamedValue, 1,
MatchTableRecord::MTRF_CommaFollows, RawValue);
}
static MatchTableRecord NamedValue(StringRef Namespace,
StringRef NamedValue) {
return MatchTableRecord(None, (Namespace + "::" + NamedValue).str(), 1,
MatchTableRecord::MTRF_CommaFollows);
}
static MatchTableRecord NamedValue(StringRef Namespace, StringRef NamedValue,
int64_t RawValue) {
return MatchTableRecord(None, (Namespace + "::" + NamedValue).str(), 1,
MatchTableRecord::MTRF_CommaFollows, RawValue);
}
static MatchTableRecord IntValue(int64_t IntValue) {
return MatchTableRecord(None, llvm::to_string(IntValue), 1,
MatchTableRecord::MTRF_CommaFollows);
}
static MatchTableRecord Label(unsigned LabelID) {
return MatchTableRecord(LabelID, "Label " + llvm::to_string(LabelID), 0,
MatchTableRecord::MTRF_Label |
MatchTableRecord::MTRF_Comment |
MatchTableRecord::MTRF_LineBreakFollows);
}
static MatchTableRecord JumpTarget(unsigned LabelID) {
return MatchTableRecord(LabelID, "Label " + llvm::to_string(LabelID), 1,
MatchTableRecord::MTRF_JumpTarget |
MatchTableRecord::MTRF_Comment |
MatchTableRecord::MTRF_CommaFollows);
}
static MatchTable buildTable(ArrayRef<Matcher *> Rules, bool WithCoverage);
MatchTable(bool WithCoverage, unsigned ID = 0)
: ID(ID), IsWithCoverage(WithCoverage) {}
bool isWithCoverage() const { return IsWithCoverage; }
void push_back(const MatchTableRecord &Value) {
if (Value.Flags & MatchTableRecord::MTRF_Label)
defineLabel(Value.LabelID);
Contents.push_back(Value);
CurrentSize += Value.size();
}
unsigned allocateLabelID() { return CurrentLabelID++; }
void defineLabel(unsigned LabelID) {
LabelMap.insert(std::make_pair(LabelID, CurrentSize));
}
unsigned getLabelIndex(unsigned LabelID) const {
const auto I = LabelMap.find(LabelID);
assert(I != LabelMap.end() && "Use of undeclared label");
return I->second;
}
void emitUse(raw_ostream &OS) const { OS << "MatchTable" << ID; }
void emitDeclaration(raw_ostream &OS) const {
unsigned Indentation = 4;
OS << " constexpr static int64_t MatchTable" << ID << "[] = {";
LineBreak.emit(OS, true, *this);
OS << std::string(Indentation, ' ');
for (auto I = Contents.begin(), E = Contents.end(); I != E;
++I) {
bool LineBreakIsNext = false;
const auto &NextI = std::next(I);
if (NextI != E) {
if (NextI->EmitStr == "" &&
NextI->Flags == MatchTableRecord::MTRF_LineBreakFollows)
LineBreakIsNext = true;
}
if (I->Flags & MatchTableRecord::MTRF_Indent)
Indentation += 2;
I->emit(OS, LineBreakIsNext, *this);
if (I->Flags & MatchTableRecord::MTRF_LineBreakFollows)
OS << std::string(Indentation, ' ');
if (I->Flags & MatchTableRecord::MTRF_Outdent)
Indentation -= 2;
}
OS << "};\n";
}
};
MatchTableRecord MatchTable::LineBreak = {
None, "" /* Emit String */, 0 /* Elements */,
MatchTableRecord::MTRF_LineBreakFollows};
void MatchTableRecord::emit(raw_ostream &OS, bool LineBreakIsNextAfterThis,
const MatchTable &Table) const {
bool UseLineComment =
LineBreakIsNextAfterThis | (Flags & MTRF_LineBreakFollows);
if (Flags & (MTRF_JumpTarget | MTRF_CommaFollows))
UseLineComment = false;
if (Flags & MTRF_Comment)
OS << (UseLineComment ? "// " : "/*");
OS << EmitStr;
if (Flags & MTRF_Label)
OS << ": @" << Table.getLabelIndex(LabelID);
if (Flags & MTRF_Comment && !UseLineComment)
OS << "*/";
if (Flags & MTRF_JumpTarget) {
if (Flags & MTRF_Comment)
OS << " ";
OS << Table.getLabelIndex(LabelID);
}
if (Flags & MTRF_CommaFollows) {
OS << ",";
if (!LineBreakIsNextAfterThis && !(Flags & MTRF_LineBreakFollows))
OS << " ";
}
if (Flags & MTRF_LineBreakFollows)
OS << "\n";
}
MatchTable &operator<<(MatchTable &Table, const MatchTableRecord &Value) {
Table.push_back(Value);
return Table;
}
//===- Matchers -----------------------------------------------------------===//
class OperandMatcher;
class MatchAction;
class PredicateMatcher;
class RuleMatcher;
class Matcher {
public:
virtual ~Matcher() = default;
virtual void optimize() {}
virtual void emit(MatchTable &Table) = 0;
virtual bool hasFirstCondition() const = 0;
virtual const PredicateMatcher &getFirstCondition() const = 0;
virtual std::unique_ptr<PredicateMatcher> popFirstCondition() = 0;
};
MatchTable MatchTable::buildTable(ArrayRef<Matcher *> Rules,
bool WithCoverage) {
MatchTable Table(WithCoverage);
for (Matcher *Rule : Rules)
Rule->emit(Table);
return Table << MatchTable::Opcode("GIM_Reject") << MatchTable::LineBreak;
}
class GroupMatcher final : public Matcher {
/// Conditions that form a common prefix of all the matchers contained.
SmallVector<std::unique_ptr<PredicateMatcher>, 1> Conditions;
/// All the nested matchers, sharing a common prefix.
std::vector<Matcher *> Matchers;
/// An owning collection for any auxiliary matchers created while optimizing
/// nested matchers contained.
std::vector<std::unique_ptr<Matcher>> MatcherStorage;
public:
/// Add a matcher to the collection of nested matchers if it meets the
/// requirements, and return true. If it doesn't, do nothing and return false.
///
/// Expected to preserve its argument, so it could be moved out later on.
bool addMatcher(Matcher &Candidate);
/// Mark the matcher as fully-built and ensure any invariants expected by both
/// optimize() and emit(...) methods. Generally, both sequences of calls
/// are expected to lead to a sensible result:
///
/// addMatcher(...)*; finalize(); optimize(); emit(...); and
/// addMatcher(...)*; finalize(); emit(...);
///
/// or generally
///
/// addMatcher(...)*; finalize(); { optimize()*; emit(...); }*
///
/// Multiple calls to optimize() are expected to be handled gracefully, though
/// optimize() is not expected to be idempotent. Multiple calls to finalize()
/// aren't generally supported. emit(...) is expected to be non-mutating and
/// producing the exact same results upon repeated calls.
///
/// addMatcher() calls after the finalize() call are not supported.
///
/// finalize() and optimize() are both allowed to mutate the contained
/// matchers, so moving them out after finalize() is not supported.
void finalize();
void optimize() override {}
void emit(MatchTable &Table) override;
/// Could be used to move out the matchers added previously, unless finalize()
/// has been already called. If any of the matchers are moved out, the group
/// becomes safe to destroy, but not safe to re-use for anything else.
iterator_range<std::vector<Matcher *>::iterator> matchers() {
return make_range(Matchers.begin(), Matchers.end());
}
size_t size() const { return Matchers.size(); }
bool empty() const { return Matchers.empty(); }
std::unique_ptr<PredicateMatcher> popFirstCondition() override {
assert(!Conditions.empty() &&
"Trying to pop a condition from a condition-less group");
std::unique_ptr<PredicateMatcher> P = std::move(Conditions.front());
Conditions.erase(Conditions.begin());
return P;
}
const PredicateMatcher &getFirstCondition() const override {
assert(!Conditions.empty() &&
"Trying to get a condition from a condition-less group");
return *Conditions.front();
}
bool hasFirstCondition() const override { return !Conditions.empty(); }
private:
/// See if a candidate matcher could be added to this group solely by
/// analyzing its first condition.
bool candidateConditionMatches(const PredicateMatcher &Predicate) const;
};
/// Generates code to check that a match rule matches.
class RuleMatcher : public Matcher {
public:
using ActionList = std::list<std::unique_ptr<MatchAction>>;
using action_iterator = ActionList::iterator;
protected:
/// A list of matchers that all need to succeed for the current rule to match.
/// FIXME: This currently supports a single match position but could be
/// extended to support multiple positions to support div/rem fusion or
/// load-multiple instructions.
using MatchersTy = std::vector<std::unique_ptr<InstructionMatcher>> ;
MatchersTy Matchers;
/// A list of actions that need to be taken when all predicates in this rule
/// have succeeded.
ActionList Actions;
using DefinedInsnVariablesMap = std::map<InstructionMatcher *, unsigned>;
/// A map of instruction matchers to the local variables
DefinedInsnVariablesMap InsnVariableIDs;
using MutatableInsnSet = SmallPtrSet<InstructionMatcher *, 4>;
// The set of instruction matchers that have not yet been claimed for mutation
// by a BuildMI.
MutatableInsnSet MutatableInsns;
/// A map of named operands defined by the matchers that may be referenced by
/// the renderers.
StringMap<OperandMatcher *> DefinedOperands;
/// ID for the next instruction variable defined with implicitlyDefineInsnVar()
unsigned NextInsnVarID;
/// ID for the next output instruction allocated with allocateOutputInsnID()
unsigned NextOutputInsnID;
/// ID for the next temporary register ID allocated with allocateTempRegID()
unsigned NextTempRegID;
std::vector<Record *> RequiredFeatures;
std::vector<std::unique_ptr<PredicateMatcher>> EpilogueMatchers;
ArrayRef<SMLoc> SrcLoc;
typedef std::tuple<Record *, unsigned, unsigned>
DefinedComplexPatternSubOperand;
typedef StringMap<DefinedComplexPatternSubOperand>
DefinedComplexPatternSubOperandMap;
/// A map of Symbolic Names to ComplexPattern sub-operands.
DefinedComplexPatternSubOperandMap ComplexSubOperands;
uint64_t RuleID;
static uint64_t NextRuleID;
public:
RuleMatcher(ArrayRef<SMLoc> SrcLoc)
: Matchers(), Actions(), InsnVariableIDs(), MutatableInsns(),
DefinedOperands(), NextInsnVarID(0), NextOutputInsnID(0),
NextTempRegID(0), SrcLoc(SrcLoc), ComplexSubOperands(),
RuleID(NextRuleID++) {}
RuleMatcher(RuleMatcher &&Other) = default;
RuleMatcher &operator=(RuleMatcher &&Other) = default;
uint64_t getRuleID() const { return RuleID; }
InstructionMatcher &addInstructionMatcher(StringRef SymbolicName);
void addRequiredFeature(Record *Feature);
const std::vector<Record *> &getRequiredFeatures() const;
template <class Kind, class... Args> Kind &addAction(Args &&... args);
template <class Kind, class... Args>
action_iterator insertAction(action_iterator InsertPt, Args &&... args);
/// Define an instruction without emitting any code to do so.
unsigned implicitlyDefineInsnVar(InstructionMatcher &Matcher);
unsigned getInsnVarID(InstructionMatcher &InsnMatcher) const;
DefinedInsnVariablesMap::const_iterator defined_insn_vars_begin() const {
return InsnVariableIDs.begin();
}
DefinedInsnVariablesMap::const_iterator defined_insn_vars_end() const {
return InsnVariableIDs.end();
}
iterator_range<typename DefinedInsnVariablesMap::const_iterator>
defined_insn_vars() const {
return make_range(defined_insn_vars_begin(), defined_insn_vars_end());
}
MutatableInsnSet::const_iterator mutatable_insns_begin() const {
return MutatableInsns.begin();
}
MutatableInsnSet::const_iterator mutatable_insns_end() const {
return MutatableInsns.end();
}
iterator_range<typename MutatableInsnSet::const_iterator>
mutatable_insns() const {
return make_range(mutatable_insns_begin(), mutatable_insns_end());
}
void reserveInsnMatcherForMutation(InstructionMatcher *InsnMatcher) {
bool R = MutatableInsns.erase(InsnMatcher);
assert(R && "Reserving a mutatable insn that isn't available");
(void)R;
}
action_iterator actions_begin() { return Actions.begin(); }
action_iterator actions_end() { return Actions.end(); }
iterator_range<action_iterator> actions() {
return make_range(actions_begin(), actions_end());
}
void defineOperand(StringRef SymbolicName, OperandMatcher &OM);
void defineComplexSubOperand(StringRef SymbolicName, Record *ComplexPattern,
unsigned RendererID, unsigned SubOperandID) {
assert(ComplexSubOperands.count(SymbolicName) == 0 && "Already defined");
ComplexSubOperands[SymbolicName] =
std::make_tuple(ComplexPattern, RendererID, SubOperandID);
}
Optional<DefinedComplexPatternSubOperand>
getComplexSubOperand(StringRef SymbolicName) const {
const auto &I = ComplexSubOperands.find(SymbolicName);
if (I == ComplexSubOperands.end())
return None;
return I->second;
}
InstructionMatcher &getInstructionMatcher(StringRef SymbolicName) const;
const OperandMatcher &getOperandMatcher(StringRef Name) const;
void optimize() override;
void emit(MatchTable &Table) override;
/// Compare the priority of this object and B.
///
/// Returns true if this object is more important than B.
bool isHigherPriorityThan(const RuleMatcher &B) const;
/// Report the maximum number of temporary operands needed by the rule
/// matcher.
unsigned countRendererFns() const;
std::unique_ptr<PredicateMatcher> popFirstCondition() override;
const PredicateMatcher &getFirstCondition() const override;
- LLTCodeGen getFirstConditionAsRootType();
bool hasFirstCondition() const override;
unsigned getNumOperands() const;
StringRef getOpcode() const;
// FIXME: Remove this as soon as possible
InstructionMatcher &insnmatchers_front() const { return *Matchers.front(); }
unsigned allocateOutputInsnID() { return NextOutputInsnID++; }
unsigned allocateTempRegID() { return NextTempRegID++; }
iterator_range<MatchersTy::iterator> insnmatchers() {
return make_range(Matchers.begin(), Matchers.end());
}
bool insnmatchers_empty() const { return Matchers.empty(); }
void insnmatchers_pop_front() { Matchers.erase(Matchers.begin()); }
};
uint64_t RuleMatcher::NextRuleID = 0;
using action_iterator = RuleMatcher::action_iterator;
template <class PredicateTy> class PredicateListMatcher {
private:
/// Template instantiations should specialize this to return a string to use
/// for the comment emitted when there are no predicates.
std::string getNoPredicateComment() const;
protected:
using PredicatesTy = std::deque<std::unique_ptr<PredicateTy>>;
PredicatesTy Predicates;
/// Track if the list of predicates was manipulated by one of the optimization
/// methods.
bool Optimized = false;
public:
/// Construct a new predicate and add it to the matcher.
template <class Kind, class... Args>
Optional<Kind *> addPredicate(Args &&... args);
typename PredicatesTy::iterator predicates_begin() {
return Predicates.begin();
}
typename PredicatesTy::iterator predicates_end() {
return Predicates.end();
}
iterator_range<typename PredicatesTy::iterator> predicates() {
return make_range(predicates_begin(), predicates_end());
}
typename PredicatesTy::size_type predicates_size() const {
return Predicates.size();
}
bool predicates_empty() const { return Predicates.empty(); }
std::unique_ptr<PredicateTy> predicates_pop_front() {
std::unique_ptr<PredicateTy> Front = std::move(Predicates.front());
Predicates.pop_front();
Optimized = true;
return Front;
}
void prependPredicate(std::unique_ptr<PredicateTy> &&Predicate) {
Predicates.push_front(std::move(Predicate));
}
void eraseNullPredicates() {
const auto NewEnd =
std::stable_partition(Predicates.begin(), Predicates.end(),
std::logical_not<std::unique_ptr<PredicateTy>>());
if (NewEnd != Predicates.begin()) {
Predicates.erase(Predicates.begin(), NewEnd);
Optimized = true;
}
}
/// Emit MatchTable opcodes that tests whether all the predicates are met.
template <class... Args>
void emitPredicateListOpcodes(MatchTable &Table, Args &&... args) {
if (Predicates.empty() && !Optimized) {
Table << MatchTable::Comment(getNoPredicateComment())
<< MatchTable::LineBreak;
return;
}
for (const auto &Predicate : predicates())
Predicate->emitPredicateOpcodes(Table, std::forward<Args>(args)...);
}
};
class PredicateMatcher {
public:
/// This enum is used for RTTI and also defines the priority that is given to
/// the predicate when generating the matcher code. Kinds with higher priority
/// must be tested first.
///
/// The relative priority of OPM_LLT, OPM_RegBank, and OPM_MBB do not matter
/// but OPM_Int must have priority over OPM_RegBank since constant integers
/// are represented by a virtual register defined by a G_CONSTANT instruction.
///
/// Note: The relative priority between IPM_ and OPM_ does not matter, they
/// are currently not compared between each other.
enum PredicateKind {
IPM_Opcode,
IPM_NumOperands,
IPM_ImmPredicate,
IPM_AtomicOrderingMMO,
IPM_MemoryLLTSize,
IPM_MemoryVsLLTSize,
OPM_SameOperand,
OPM_ComplexPattern,
OPM_IntrinsicID,
OPM_Instruction,
OPM_Int,
OPM_LiteralInt,
OPM_LLT,
OPM_PointerToAny,
OPM_RegBank,
OPM_MBB,
};
protected:
PredicateKind Kind;
unsigned InsnVarID;
unsigned OpIdx;
public:
PredicateMatcher(PredicateKind Kind, unsigned InsnVarID, unsigned OpIdx = ~0)
: Kind(Kind), InsnVarID(InsnVarID), OpIdx(OpIdx) {}
unsigned getInsnVarID() const { return InsnVarID; }
unsigned getOpIdx() const { return OpIdx; }
virtual ~PredicateMatcher() = default;
/// Emit MatchTable opcodes that check the predicate for the given operand.
virtual void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const = 0;
PredicateKind getKind() const { return Kind; }
virtual bool isIdentical(const PredicateMatcher &B) const {
return B.getKind() == getKind() && InsnVarID == B.InsnVarID &&
OpIdx == B.OpIdx;
}
virtual bool isIdenticalDownToValue(const PredicateMatcher &B) const {
return hasValue() && PredicateMatcher::isIdentical(B);
}
virtual MatchTableRecord getValue() const {
assert(hasValue() && "Can not get a value of a value-less predicate!");
llvm_unreachable("Not implemented yet");
}
virtual bool hasValue() const { return false; }
/// Report the maximum number of temporary operands needed by the predicate
/// matcher.
virtual unsigned countRendererFns() const { return 0; }
};
/// Generates code to check a predicate of an operand.
///
/// Typical predicates include:
/// * Operand is a particular register.
/// * Operand is assigned a particular register bank.
/// * Operand is an MBB.
class OperandPredicateMatcher : public PredicateMatcher {
public:
OperandPredicateMatcher(PredicateKind Kind, unsigned InsnVarID,
unsigned OpIdx)
: PredicateMatcher(Kind, InsnVarID, OpIdx) {}
virtual ~OperandPredicateMatcher() {}
/// Compare the priority of this object and B.
///
/// Returns true if this object is more important than B.
virtual bool isHigherPriorityThan(const OperandPredicateMatcher &B) const;
};
template <>
std::string
PredicateListMatcher<OperandPredicateMatcher>::getNoPredicateComment() const {
return "No operand predicates";
}
/// Generates code to check that a register operand is defined by the same exact
/// one as another.
class SameOperandMatcher : public OperandPredicateMatcher {
std::string MatchingName;
public:
SameOperandMatcher(unsigned InsnVarID, unsigned OpIdx, StringRef MatchingName)
: OperandPredicateMatcher(OPM_SameOperand, InsnVarID, OpIdx),
MatchingName(MatchingName) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_SameOperand;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override;
bool isIdentical(const PredicateMatcher &B) const override {
return OperandPredicateMatcher::isIdentical(B) &&
MatchingName == cast<SameOperandMatcher>(&B)->MatchingName;
}
};
/// Generates code to check that an operand is a particular LLT.
class LLTOperandMatcher : public OperandPredicateMatcher {
protected:
LLTCodeGen Ty;
public:
static std::map<LLTCodeGen, unsigned> TypeIDValues;
static void initTypeIDValuesMap() {
TypeIDValues.clear();
unsigned ID = 0;
for (const LLTCodeGen LLTy : KnownTypes)
TypeIDValues[LLTy] = ID++;
}
LLTOperandMatcher(unsigned InsnVarID, unsigned OpIdx, const LLTCodeGen &Ty)
: OperandPredicateMatcher(OPM_LLT, InsnVarID, OpIdx), Ty(Ty) {
KnownTypes.insert(Ty);
}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_LLT;
}
bool isIdentical(const PredicateMatcher &B) const override {
return OperandPredicateMatcher::isIdentical(B) &&
Ty == cast<LLTOperandMatcher>(&B)->Ty;
}
MatchTableRecord getValue() const override {
const auto VI = TypeIDValues.find(Ty);
if (VI == TypeIDValues.end())
return MatchTable::NamedValue(getTy().getCxxEnumValue());
return MatchTable::NamedValue(getTy().getCxxEnumValue(), VI->second);
}
bool hasValue() const override {
if (TypeIDValues.size() != KnownTypes.size())
initTypeIDValuesMap();
return TypeIDValues.count(Ty);
}
LLTCodeGen getTy() const { return Ty; }
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckType") << MatchTable::Comment("MI")
<< MatchTable::IntValue(InsnVarID) << MatchTable::Comment("Op")
<< MatchTable::IntValue(OpIdx) << MatchTable::Comment("Type")
<< getValue() << MatchTable::LineBreak;
}
};
std::map<LLTCodeGen, unsigned> LLTOperandMatcher::TypeIDValues;
/// Generates code to check that an operand is a pointer to any address space.
///
/// In SelectionDAG, the types did not describe pointers or address spaces. As a
/// result, iN is used to describe a pointer of N bits to any address space and
/// PatFrag predicates are typically used to constrain the address space. There's
/// no reliable means to derive the missing type information from the pattern so
/// imported rules must test the components of a pointer separately.
///
/// If SizeInBits is zero, then the pointer size will be obtained from the
/// subtarget.
class PointerToAnyOperandMatcher : public OperandPredicateMatcher {
protected:
unsigned SizeInBits;
public:
PointerToAnyOperandMatcher(unsigned InsnVarID, unsigned OpIdx,
unsigned SizeInBits)
: OperandPredicateMatcher(OPM_PointerToAny, InsnVarID, OpIdx),
SizeInBits(SizeInBits) {}
static bool classof(const OperandPredicateMatcher *P) {
return P->getKind() == OPM_PointerToAny;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckPointerToAny")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("Op") << MatchTable::IntValue(OpIdx)
<< MatchTable::Comment("SizeInBits")
<< MatchTable::IntValue(SizeInBits) << MatchTable::LineBreak;
}
};
/// Generates code to check that an operand is a particular target constant.
class ComplexPatternOperandMatcher : public OperandPredicateMatcher {
protected:
const OperandMatcher &Operand;
const Record &TheDef;
unsigned getAllocatedTemporariesBaseID() const;
public:
bool isIdentical(const PredicateMatcher &B) const override { return false; }
ComplexPatternOperandMatcher(unsigned InsnVarID, unsigned OpIdx,
const OperandMatcher &Operand,
const Record &TheDef)
: OperandPredicateMatcher(OPM_ComplexPattern, InsnVarID, OpIdx),
Operand(Operand), TheDef(TheDef) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_ComplexPattern;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
unsigned ID = getAllocatedTemporariesBaseID();
Table << MatchTable::Opcode("GIM_CheckComplexPattern")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("Op") << MatchTable::IntValue(OpIdx)
<< MatchTable::Comment("Renderer") << MatchTable::IntValue(ID)
<< MatchTable::NamedValue(("GICP_" + TheDef.getName()).str())
<< MatchTable::LineBreak;
}
unsigned countRendererFns() const override {
return 1;
}
};
/// Generates code to check that an operand is in a particular register bank.
class RegisterBankOperandMatcher : public OperandPredicateMatcher {
protected:
const CodeGenRegisterClass &RC;
public:
RegisterBankOperandMatcher(unsigned InsnVarID, unsigned OpIdx,
const CodeGenRegisterClass &RC)
: OperandPredicateMatcher(OPM_RegBank, InsnVarID, OpIdx), RC(RC) {}
bool isIdentical(const PredicateMatcher &B) const override {
return OperandPredicateMatcher::isIdentical(B) &&
RC.getDef() == cast<RegisterBankOperandMatcher>(&B)->RC.getDef();
}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_RegBank;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckRegBankForClass")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("Op") << MatchTable::IntValue(OpIdx)
<< MatchTable::Comment("RC")
<< MatchTable::NamedValue(RC.getQualifiedName() + "RegClassID")
<< MatchTable::LineBreak;
}
};
/// Generates code to check that an operand is a basic block.
class MBBOperandMatcher : public OperandPredicateMatcher {
public:
MBBOperandMatcher(unsigned InsnVarID, unsigned OpIdx)
: OperandPredicateMatcher(OPM_MBB, InsnVarID, OpIdx) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_MBB;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckIsMBB") << MatchTable::Comment("MI")
<< MatchTable::IntValue(InsnVarID) << MatchTable::Comment("Op")
<< MatchTable::IntValue(OpIdx) << MatchTable::LineBreak;
}
};
/// Generates code to check that an operand is a G_CONSTANT with a particular
/// int.
class ConstantIntOperandMatcher : public OperandPredicateMatcher {
protected:
int64_t Value;
public:
ConstantIntOperandMatcher(unsigned InsnVarID, unsigned OpIdx, int64_t Value)
: OperandPredicateMatcher(OPM_Int, InsnVarID, OpIdx), Value(Value) {}
bool isIdentical(const PredicateMatcher &B) const override {
return OperandPredicateMatcher::isIdentical(B) &&
Value == cast<ConstantIntOperandMatcher>(&B)->Value;
}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_Int;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckConstantInt")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("Op") << MatchTable::IntValue(OpIdx)
<< MatchTable::IntValue(Value) << MatchTable::LineBreak;
}
};
/// Generates code to check that an operand is a raw int (where MO.isImm() or
/// MO.isCImm() is true).
class LiteralIntOperandMatcher : public OperandPredicateMatcher {
protected:
int64_t Value;
public:
LiteralIntOperandMatcher(unsigned InsnVarID, unsigned OpIdx, int64_t Value)
: OperandPredicateMatcher(OPM_LiteralInt, InsnVarID, OpIdx),
Value(Value) {}
bool isIdentical(const PredicateMatcher &B) const override {
return OperandPredicateMatcher::isIdentical(B) &&
Value == cast<LiteralIntOperandMatcher>(&B)->Value;
}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_LiteralInt;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckLiteralInt")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("Op") << MatchTable::IntValue(OpIdx)
<< MatchTable::IntValue(Value) << MatchTable::LineBreak;
}
};
/// Generates code to check that an operand is an intrinsic ID.
class IntrinsicIDOperandMatcher : public OperandPredicateMatcher {
protected:
const CodeGenIntrinsic *II;
public:
IntrinsicIDOperandMatcher(unsigned InsnVarID, unsigned OpIdx,
const CodeGenIntrinsic *II)
: OperandPredicateMatcher(OPM_IntrinsicID, InsnVarID, OpIdx), II(II) {}
bool isIdentical(const PredicateMatcher &B) const override {
return OperandPredicateMatcher::isIdentical(B) &&
II == cast<IntrinsicIDOperandMatcher>(&B)->II;
}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_IntrinsicID;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckIntrinsicID")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("Op") << MatchTable::IntValue(OpIdx)
<< MatchTable::NamedValue("Intrinsic::" + II->EnumName)
<< MatchTable::LineBreak;
}
};
/// Generates code to check that a set of predicates match for a particular
/// operand.
class OperandMatcher : public PredicateListMatcher<OperandPredicateMatcher> {
protected:
InstructionMatcher &Insn;
unsigned OpIdx;
std::string SymbolicName;
/// The index of the first temporary variable allocated to this operand. The
/// number of allocated temporaries can be found with
/// countRendererFns().
unsigned AllocatedTemporariesBaseID;
public:
OperandMatcher(InstructionMatcher &Insn, unsigned OpIdx,
const std::string &SymbolicName,
unsigned AllocatedTemporariesBaseID)
: Insn(Insn), OpIdx(OpIdx), SymbolicName(SymbolicName),
AllocatedTemporariesBaseID(AllocatedTemporariesBaseID) {}
bool hasSymbolicName() const { return !SymbolicName.empty(); }
const StringRef getSymbolicName() const { return SymbolicName; }
void setSymbolicName(StringRef Name) {
assert(SymbolicName.empty() && "Operand already has a symbolic name");
SymbolicName = Name;
}
/// Construct a new operand predicate and add it to the matcher.
template <class Kind, class... Args>
Optional<Kind *> addPredicate(Args &&... args) {
if (isSameAsAnotherOperand())
return None;
Predicates.emplace_back(llvm::make_unique<Kind>(
getInsnVarID(), getOpIdx(), std::forward<Args>(args)...));
return static_cast<Kind *>(Predicates.back().get());
}
unsigned getOpIdx() const { return OpIdx; }
unsigned getInsnVarID() const;
std::string getOperandExpr(unsigned InsnVarID) const {
return "State.MIs[" + llvm::to_string(InsnVarID) + "]->getOperand(" +
llvm::to_string(OpIdx) + ")";
}
InstructionMatcher &getInstructionMatcher() const { return Insn; }
Error addTypeCheckPredicate(const TypeSetByHwMode &VTy,
bool OperandIsAPointer);
/// Emit MatchTable opcodes that test whether the instruction named in
/// InsnVarID matches all the predicates and all the operands.
void emitPredicateOpcodes(MatchTable &Table, RuleMatcher &Rule) {
if (!Optimized) {
std::string Comment;
raw_string_ostream CommentOS(Comment);
CommentOS << "MIs[" << getInsnVarID() << "] ";
if (SymbolicName.empty())
CommentOS << "Operand " << OpIdx;
else
CommentOS << SymbolicName;
Table << MatchTable::Comment(CommentOS.str()) << MatchTable::LineBreak;
}
emitPredicateListOpcodes(Table, Rule);
}
/// Compare the priority of this object and B.
///
/// Returns true if this object is more important than B.
bool isHigherPriorityThan(OperandMatcher &B) {
// Operand matchers involving more predicates have higher priority.
if (predicates_size() > B.predicates_size())
return true;
if (predicates_size() < B.predicates_size())
return false;
// This assumes that predicates are added in a consistent order.
for (auto &&Predicate : zip(predicates(), B.predicates())) {
if (std::get<0>(Predicate)->isHigherPriorityThan(*std::get<1>(Predicate)))
return true;
if (std::get<1>(Predicate)->isHigherPriorityThan(*std::get<0>(Predicate)))
return false;
}
return false;
};
/// Report the maximum number of temporary operands needed by the operand
/// matcher.
unsigned countRendererFns() {
return std::accumulate(
predicates().begin(), predicates().end(), 0,
[](unsigned A,
const std::unique_ptr<OperandPredicateMatcher> &Predicate) {
return A + Predicate->countRendererFns();
});
}
unsigned getAllocatedTemporariesBaseID() const {
return AllocatedTemporariesBaseID;
}
bool isSameAsAnotherOperand() {
for (const auto &Predicate : predicates())
if (isa<SameOperandMatcher>(Predicate))
return true;
return false;
}
};
Error OperandMatcher::addTypeCheckPredicate(const TypeSetByHwMode &VTy,
bool OperandIsAPointer) {
if (!VTy.isMachineValueType())
return failedImport("unsupported typeset");
if (VTy.getMachineValueType() == MVT::iPTR && OperandIsAPointer) {
addPredicate<PointerToAnyOperandMatcher>(0);
return Error::success();
}
auto OpTyOrNone = MVTToLLT(VTy.getMachineValueType().SimpleTy);
if (!OpTyOrNone)
return failedImport("unsupported type");
if (OperandIsAPointer)
addPredicate<PointerToAnyOperandMatcher>(OpTyOrNone->get().getSizeInBits());
else
addPredicate<LLTOperandMatcher>(*OpTyOrNone);
return Error::success();
}
unsigned ComplexPatternOperandMatcher::getAllocatedTemporariesBaseID() const {
return Operand.getAllocatedTemporariesBaseID();
}
/// Generates code to check a predicate on an instruction.
///
/// Typical predicates include:
/// * The opcode of the instruction is a particular value.
/// * The nsw/nuw flag is/isn't set.
class InstructionPredicateMatcher : public PredicateMatcher {
public:
InstructionPredicateMatcher(PredicateKind Kind, unsigned InsnVarID)
: PredicateMatcher(Kind, InsnVarID) {}
virtual ~InstructionPredicateMatcher() {}
/// Compare the priority of this object and B.
///
/// Returns true if this object is more important than B.
virtual bool
isHigherPriorityThan(const InstructionPredicateMatcher &B) const {
return Kind < B.Kind;
};
};
template <>
std::string
PredicateListMatcher<PredicateMatcher>::getNoPredicateComment() const {
return "No instruction predicates";
}
/// Generates code to check the opcode of an instruction.
class InstructionOpcodeMatcher : public InstructionPredicateMatcher {
protected:
const CodeGenInstruction *I;
static DenseMap<const CodeGenInstruction *, unsigned> OpcodeValues;
public:
static void initOpcodeValuesMap(const CodeGenTarget &Target) {
OpcodeValues.clear();
unsigned OpcodeValue = 0;
for (const CodeGenInstruction *I : Target.getInstructionsByEnumValue())
OpcodeValues[I] = OpcodeValue++;
}
InstructionOpcodeMatcher(unsigned InsnVarID, const CodeGenInstruction *I)
: InstructionPredicateMatcher(IPM_Opcode, InsnVarID), I(I) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == IPM_Opcode;
}
bool isIdentical(const PredicateMatcher &B) const override {
return InstructionPredicateMatcher::isIdentical(B) &&
I == cast<InstructionOpcodeMatcher>(&B)->I;
}
MatchTableRecord getValue() const override {
const auto VI = OpcodeValues.find(I);
if (VI != OpcodeValues.end())
return MatchTable::NamedValue(I->Namespace, I->TheDef->getName(),
VI->second);
return MatchTable::NamedValue(I->Namespace, I->TheDef->getName());
}
bool hasValue() const override { return OpcodeValues.count(I); }
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckOpcode") << MatchTable::Comment("MI")
<< MatchTable::IntValue(InsnVarID) << getValue()
<< MatchTable::LineBreak;
}
/// Compare the priority of this object and B.
///
/// Returns true if this object is more important than B.
bool
isHigherPriorityThan(const InstructionPredicateMatcher &B) const override {
if (InstructionPredicateMatcher::isHigherPriorityThan(B))
return true;
if (B.InstructionPredicateMatcher::isHigherPriorityThan(*this))
return false;
// Prioritize opcodes for cosmetic reasons in the generated source. Although
// this is cosmetic at the moment, we may want to drive a similar ordering
// using instruction frequency information to improve compile time.
if (const InstructionOpcodeMatcher *BO =
dyn_cast<InstructionOpcodeMatcher>(&B))
return I->TheDef->getName() < BO->I->TheDef->getName();
return false;
};
bool isConstantInstruction() const {
return I->TheDef->getName() == "G_CONSTANT";
}
StringRef getOpcode() const { return I->TheDef->getName(); }
unsigned getNumOperands() const { return I->Operands.size(); }
StringRef getOperandType(unsigned OpIdx) const {
return I->Operands[OpIdx].OperandType;
}
};
DenseMap<const CodeGenInstruction *, unsigned>
InstructionOpcodeMatcher::OpcodeValues;
class InstructionNumOperandsMatcher final : public InstructionPredicateMatcher {
unsigned NumOperands = 0;
public:
InstructionNumOperandsMatcher(unsigned InsnVarID, unsigned NumOperands)
: InstructionPredicateMatcher(IPM_NumOperands, InsnVarID),
NumOperands(NumOperands) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == IPM_NumOperands;
}
bool isIdentical(const PredicateMatcher &B) const override {
return InstructionPredicateMatcher::isIdentical(B) &&
NumOperands == cast<InstructionNumOperandsMatcher>(&B)->NumOperands;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckNumOperands")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("Expected")
<< MatchTable::IntValue(NumOperands) << MatchTable::LineBreak;
}
};
/// Generates code to check that this instruction is a constant whose value
/// meets an immediate predicate.
///
/// Immediates are slightly odd since they are typically used like an operand
/// but are represented as an operator internally. We typically write simm8:$src
/// in a tablegen pattern, but this is just syntactic sugar for
/// (imm:i32)<<P:Predicate_simm8>>:$imm which more directly describes the nodes
/// that will be matched and the predicate (which is attached to the imm
/// operator) that will be tested. In SelectionDAG this describes a
/// ConstantSDNode whose internal value will be tested using the simm8 predicate.
///
/// The corresponding GlobalISel representation is %1 = G_CONSTANT iN Value. In
/// this representation, the immediate could be tested with an
/// InstructionMatcher, InstructionOpcodeMatcher, OperandMatcher, and a
/// OperandPredicateMatcher-subclass to check the Value meets the predicate but
/// there are two implementation issues with producing that matcher
/// configuration from the SelectionDAG pattern:
/// * ImmLeaf is a PatFrag whose root is an InstructionMatcher. This means that
/// were we to sink the immediate predicate to the operand we would have to
/// have two partial implementations of PatFrag support, one for immediates
/// and one for non-immediates.
/// * At the point we handle the predicate, the OperandMatcher hasn't been
/// created yet. If we were to sink the predicate to the OperandMatcher we
/// would also have to complicate (or duplicate) the code that descends and
/// creates matchers for the subtree.
/// Overall, it's simpler to handle it in the place it was found.
class InstructionImmPredicateMatcher : public InstructionPredicateMatcher {
protected:
TreePredicateFn Predicate;
public:
InstructionImmPredicateMatcher(unsigned InsnVarID,
const TreePredicateFn &Predicate)
: InstructionPredicateMatcher(IPM_ImmPredicate, InsnVarID),
Predicate(Predicate) {}
bool isIdentical(const PredicateMatcher &B) const override {
return InstructionPredicateMatcher::isIdentical(B) &&
Predicate.getOrigPatFragRecord() ==
cast<InstructionImmPredicateMatcher>(&B)
->Predicate.getOrigPatFragRecord();
}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == IPM_ImmPredicate;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode(getMatchOpcodeForPredicate(Predicate))
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("Predicate")
<< MatchTable::NamedValue(getEnumNameForPredicate(Predicate))
<< MatchTable::LineBreak;
}
};
/// Generates code to check that a memory instruction has a atomic ordering
/// MachineMemoryOperand.
class AtomicOrderingMMOPredicateMatcher : public InstructionPredicateMatcher {
public:
enum AOComparator {
AO_Exactly,
AO_OrStronger,
AO_WeakerThan,
};
protected:
StringRef Order;
AOComparator Comparator;
public:
AtomicOrderingMMOPredicateMatcher(unsigned InsnVarID, StringRef Order,
AOComparator Comparator = AO_Exactly)
: InstructionPredicateMatcher(IPM_AtomicOrderingMMO, InsnVarID),
Order(Order), Comparator(Comparator) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == IPM_AtomicOrderingMMO;
}
bool isIdentical(const PredicateMatcher &B) const override {
if (!InstructionPredicateMatcher::isIdentical(B))
return false;
const auto &R = *cast<AtomicOrderingMMOPredicateMatcher>(&B);
return Order == R.Order && Comparator == R.Comparator;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
StringRef Opcode = "GIM_CheckAtomicOrdering";
if (Comparator == AO_OrStronger)
Opcode = "GIM_CheckAtomicOrderingOrStrongerThan";
if (Comparator == AO_WeakerThan)
Opcode = "GIM_CheckAtomicOrderingWeakerThan";
Table << MatchTable::Opcode(Opcode) << MatchTable::Comment("MI")
<< MatchTable::IntValue(InsnVarID) << MatchTable::Comment("Order")
<< MatchTable::NamedValue(("(int64_t)AtomicOrdering::" + Order).str())
<< MatchTable::LineBreak;
}
};
/// Generates code to check that the size of an MMO is exactly N bytes.
class MemorySizePredicateMatcher : public InstructionPredicateMatcher {
protected:
unsigned MMOIdx;
uint64_t Size;
public:
MemorySizePredicateMatcher(unsigned InsnVarID, unsigned MMOIdx, unsigned Size)
: InstructionPredicateMatcher(IPM_MemoryLLTSize, InsnVarID),
MMOIdx(MMOIdx), Size(Size) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == IPM_MemoryLLTSize;
}
bool isIdentical(const PredicateMatcher &B) const override {
return InstructionPredicateMatcher::isIdentical(B) &&
MMOIdx == cast<MemorySizePredicateMatcher>(&B)->MMOIdx &&
Size == cast<MemorySizePredicateMatcher>(&B)->Size;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIM_CheckMemorySizeEqualTo")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("MMO") << MatchTable::IntValue(MMOIdx)
<< MatchTable::Comment("Size") << MatchTable::IntValue(Size)
<< MatchTable::LineBreak;
}
};
/// Generates code to check that the size of an MMO is less-than, equal-to, or
/// greater than a given LLT.
class MemoryVsLLTSizePredicateMatcher : public InstructionPredicateMatcher {
public:
enum RelationKind {
GreaterThan,
EqualTo,
LessThan,
};
protected:
unsigned MMOIdx;
RelationKind Relation;
unsigned OpIdx;
public:
MemoryVsLLTSizePredicateMatcher(unsigned InsnVarID, unsigned MMOIdx,
enum RelationKind Relation,
unsigned OpIdx)
: InstructionPredicateMatcher(IPM_MemoryVsLLTSize, InsnVarID),
MMOIdx(MMOIdx), Relation(Relation), OpIdx(OpIdx) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == IPM_MemoryVsLLTSize;
}
bool isIdentical(const PredicateMatcher &B) const override {
return InstructionPredicateMatcher::isIdentical(B) &&
MMOIdx == cast<MemoryVsLLTSizePredicateMatcher>(&B)->MMOIdx &&
Relation == cast<MemoryVsLLTSizePredicateMatcher>(&B)->Relation &&
OpIdx == cast<MemoryVsLLTSizePredicateMatcher>(&B)->OpIdx;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
Table << MatchTable::Opcode(Relation == EqualTo
? "GIM_CheckMemorySizeEqualToLLT"
: Relation == GreaterThan
? "GIM_CheckMemorySizeGreaterThanLLT"
: "GIM_CheckMemorySizeLessThanLLT")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("MMO") << MatchTable::IntValue(MMOIdx)
<< MatchTable::Comment("OpIdx") << MatchTable::IntValue(OpIdx)
<< MatchTable::LineBreak;
}
};
/// Generates code to check that a set of predicates and operands match for a
/// particular instruction.
///
/// Typical predicates include:
/// * Has a specific opcode.
/// * Has an nsw/nuw flag or doesn't.
class InstructionMatcher final : public PredicateListMatcher<PredicateMatcher> {
protected:
typedef std::vector<std::unique_ptr<OperandMatcher>> OperandVec;
RuleMatcher &Rule;
/// The operands to match. All rendered operands must be present even if the
/// condition is always true.
OperandVec Operands;
bool NumOperandsCheck = true;
std::string SymbolicName;
unsigned InsnVarID;
public:
InstructionMatcher(RuleMatcher &Rule, StringRef SymbolicName)
: Rule(Rule), SymbolicName(SymbolicName) {
// We create a new instruction matcher.
// Get a new ID for that instruction.
InsnVarID = Rule.implicitlyDefineInsnVar(*this);
}
/// Construct a new instruction predicate and add it to the matcher.
template <class Kind, class... Args>
Optional<Kind *> addPredicate(Args &&... args) {
Predicates.emplace_back(
llvm::make_unique<Kind>(getInsnVarID(), std::forward<Args>(args)...));
return static_cast<Kind *>(Predicates.back().get());
}
RuleMatcher &getRuleMatcher() const { return Rule; }
unsigned getInsnVarID() const { return InsnVarID; }
/// Add an operand to the matcher.
OperandMatcher &addOperand(unsigned OpIdx, const std::string &SymbolicName,
unsigned AllocatedTemporariesBaseID) {
Operands.emplace_back(new OperandMatcher(*this, OpIdx, SymbolicName,
AllocatedTemporariesBaseID));
if (!SymbolicName.empty())
Rule.defineOperand(SymbolicName, *Operands.back());
return *Operands.back();
}
OperandMatcher &getOperand(unsigned OpIdx) {
auto I = std::find_if(Operands.begin(), Operands.end(),
[&OpIdx](const std::unique_ptr<OperandMatcher> &X) {
return X->getOpIdx() == OpIdx;
});
if (I != Operands.end())
return **I;
llvm_unreachable("Failed to lookup operand");
}
StringRef getSymbolicName() const { return SymbolicName; }
unsigned getNumOperands() const { return Operands.size(); }
OperandVec::iterator operands_begin() { return Operands.begin(); }
OperandVec::iterator operands_end() { return Operands.end(); }
iterator_range<OperandVec::iterator> operands() {
return make_range(operands_begin(), operands_end());
}
OperandVec::const_iterator operands_begin() const { return Operands.begin(); }
OperandVec::const_iterator operands_end() const { return Operands.end(); }
iterator_range<OperandVec::const_iterator> operands() const {
return make_range(operands_begin(), operands_end());
}
bool operands_empty() const { return Operands.empty(); }
void pop_front() { Operands.erase(Operands.begin()); }
void optimize();
/// Emit MatchTable opcodes that test whether the instruction named in
/// InsnVarName matches all the predicates and all the operands.
void emitPredicateOpcodes(MatchTable &Table, RuleMatcher &Rule) {
if (NumOperandsCheck)
InstructionNumOperandsMatcher(InsnVarID, getNumOperands())
.emitPredicateOpcodes(Table, Rule);
emitPredicateListOpcodes(Table, Rule);
for (const auto &Operand : Operands)
Operand->emitPredicateOpcodes(Table, Rule);
}
/// Compare the priority of this object and B.
///
/// Returns true if this object is more important than B.
bool isHigherPriorityThan(InstructionMatcher &B) {
// Instruction matchers involving more operands have higher priority.
if (Operands.size() > B.Operands.size())
return true;
if (Operands.size() < B.Operands.size())
return false;
for (auto &&P : zip(predicates(), B.predicates())) {
auto L = static_cast<InstructionPredicateMatcher *>(std::get<0>(P).get());
auto R = static_cast<InstructionPredicateMatcher *>(std::get<1>(P).get());
if (L->isHigherPriorityThan(*R))
return true;
if (R->isHigherPriorityThan(*L))
return false;
}
for (const auto &Operand : zip(Operands, B.Operands)) {
if (std::get<0>(Operand)->isHigherPriorityThan(*std::get<1>(Operand)))
return true;
if (std::get<1>(Operand)->isHigherPriorityThan(*std::get<0>(Operand)))
return false;
}
return false;
};
/// Report the maximum number of temporary operands needed by the instruction
/// matcher.
unsigned countRendererFns() {
return std::accumulate(
predicates().begin(), predicates().end(), 0,
[](unsigned A,
const std::unique_ptr<PredicateMatcher> &Predicate) {
return A + Predicate->countRendererFns();
}) +
std::accumulate(
Operands.begin(), Operands.end(), 0,
[](unsigned A, const std::unique_ptr<OperandMatcher> &Operand) {
return A + Operand->countRendererFns();
});
}
InstructionOpcodeMatcher &getOpcodeMatcher() {
for (auto &P : predicates())
if (auto *OpMatcher = dyn_cast<InstructionOpcodeMatcher>(P.get()))
return *OpMatcher;
llvm_unreachable("Didn't find an opcode matcher");
}
bool isConstantInstruction() {
return getOpcodeMatcher().isConstantInstruction();
}
StringRef getOpcode() { return getOpcodeMatcher().getOpcode(); }
};
StringRef RuleMatcher::getOpcode() const {
return Matchers.front()->getOpcode();
}
unsigned RuleMatcher::getNumOperands() const {
return Matchers.front()->getNumOperands();
}
-LLTCodeGen RuleMatcher::getFirstConditionAsRootType() {
- InstructionMatcher &InsnMatcher = *Matchers.front();
- if (!InsnMatcher.predicates_empty())
- if (const auto *TM =
- dyn_cast<LLTOperandMatcher>(&**InsnMatcher.predicates_begin()))
- if (TM->getInsnVarID() == 0 && TM->getOpIdx() == 0)
- return TM->getTy();
- return {};
-}
-
/// Generates code to check that the operand is a register defined by an
/// instruction that matches the given instruction matcher.
///
/// For example, the pattern:
/// (set $dst, (G_MUL (G_ADD $src1, $src2), $src3))
/// would use an InstructionOperandMatcher for operand 1 of the G_MUL to match
/// the:
/// (G_ADD $src1, $src2)
/// subpattern.
class InstructionOperandMatcher : public OperandPredicateMatcher {
protected:
std::unique_ptr<InstructionMatcher> InsnMatcher;
public:
InstructionOperandMatcher(unsigned InsnVarID, unsigned OpIdx,
RuleMatcher &Rule, StringRef SymbolicName)
: OperandPredicateMatcher(OPM_Instruction, InsnVarID, OpIdx),
InsnMatcher(new InstructionMatcher(Rule, SymbolicName)) {}
static bool classof(const PredicateMatcher *P) {
return P->getKind() == OPM_Instruction;
}
InstructionMatcher &getInsnMatcher() const { return *InsnMatcher; }
void emitCaptureOpcodes(MatchTable &Table, RuleMatcher &Rule) const {
const unsigned NewInsnVarID = InsnMatcher->getInsnVarID();
Table << MatchTable::Opcode("GIM_RecordInsn")
<< MatchTable::Comment("DefineMI")
<< MatchTable::IntValue(NewInsnVarID) << MatchTable::Comment("MI")
<< MatchTable::IntValue(getInsnVarID())
<< MatchTable::Comment("OpIdx") << MatchTable::IntValue(getOpIdx())
<< MatchTable::Comment("MIs[" + llvm::to_string(NewInsnVarID) + "]")
<< MatchTable::LineBreak;
}
void emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const override {
emitCaptureOpcodes(Table, Rule);
InsnMatcher->emitPredicateOpcodes(Table, Rule);
}
bool isHigherPriorityThan(const OperandPredicateMatcher &B) const override {
if (OperandPredicateMatcher::isHigherPriorityThan(B))
return true;
if (B.OperandPredicateMatcher::isHigherPriorityThan(*this))
return false;
if (const InstructionOperandMatcher *BP =
dyn_cast<InstructionOperandMatcher>(&B))
if (InsnMatcher->isHigherPriorityThan(*BP->InsnMatcher))
return true;
return false;
}
};
void InstructionMatcher::optimize() {
SmallVector<std::unique_ptr<PredicateMatcher>, 8> Stash;
const auto &OpcMatcher = getOpcodeMatcher();
Stash.push_back(predicates_pop_front());
if (Stash.back().get() == &OpcMatcher) {
if (NumOperandsCheck && OpcMatcher.getNumOperands() < getNumOperands())
Stash.emplace_back(
new InstructionNumOperandsMatcher(InsnVarID, getNumOperands()));
NumOperandsCheck = false;
}
if (InsnVarID > 0) {
assert(!Operands.empty() && "Nested instruction is expected to def a vreg");
for (auto &OP : Operands[0]->predicates())
OP.reset();
Operands[0]->eraseNullPredicates();
}
while (!Stash.empty())
prependPredicate(Stash.pop_back_val());
}
//===- Actions ------------------------------------------------------------===//
class OperandRenderer {
public:
enum RendererKind {
OR_Copy,
OR_CopyOrAddZeroReg,
OR_CopySubReg,
OR_CopyConstantAsImm,
OR_CopyFConstantAsFPImm,
OR_Imm,
OR_Register,
OR_TempRegister,
OR_ComplexPattern,
OR_Custom
};
protected:
RendererKind Kind;
public:
OperandRenderer(RendererKind Kind) : Kind(Kind) {}
virtual ~OperandRenderer() {}
RendererKind getKind() const { return Kind; }
virtual void emitRenderOpcodes(MatchTable &Table,
RuleMatcher &Rule) const = 0;
};
/// A CopyRenderer emits code to copy a single operand from an existing
/// instruction to the one being built.
class CopyRenderer : public OperandRenderer {
protected:
unsigned NewInsnID;
/// The name of the operand.
const StringRef SymbolicName;
public:
CopyRenderer(unsigned NewInsnID, StringRef SymbolicName)
: OperandRenderer(OR_Copy), NewInsnID(NewInsnID),
SymbolicName(SymbolicName) {
assert(!SymbolicName.empty() && "Cannot copy from an unspecified source");
}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_Copy;
}
const StringRef getSymbolicName() const { return SymbolicName; }
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
const OperandMatcher &Operand = Rule.getOperandMatcher(SymbolicName);
unsigned OldInsnVarID = Rule.getInsnVarID(Operand.getInstructionMatcher());
Table << MatchTable::Opcode("GIR_Copy") << MatchTable::Comment("NewInsnID")
<< MatchTable::IntValue(NewInsnID) << MatchTable::Comment("OldInsnID")
<< MatchTable::IntValue(OldInsnVarID) << MatchTable::Comment("OpIdx")
<< MatchTable::IntValue(Operand.getOpIdx())
<< MatchTable::Comment(SymbolicName) << MatchTable::LineBreak;
}
};
/// A CopyOrAddZeroRegRenderer emits code to copy a single operand from an
/// existing instruction to the one being built. If the operand turns out to be
/// a 'G_CONSTANT 0' then it replaces the operand with a zero register.
class CopyOrAddZeroRegRenderer : public OperandRenderer {
protected:
unsigned NewInsnID;
/// The name of the operand.
const StringRef SymbolicName;
const Record *ZeroRegisterDef;
public:
CopyOrAddZeroRegRenderer(unsigned NewInsnID,
StringRef SymbolicName, Record *ZeroRegisterDef)
: OperandRenderer(OR_CopyOrAddZeroReg), NewInsnID(NewInsnID),
SymbolicName(SymbolicName), ZeroRegisterDef(ZeroRegisterDef) {
assert(!SymbolicName.empty() && "Cannot copy from an unspecified source");
}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_CopyOrAddZeroReg;
}
const StringRef getSymbolicName() const { return SymbolicName; }
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
const OperandMatcher &Operand = Rule.getOperandMatcher(SymbolicName);
unsigned OldInsnVarID = Rule.getInsnVarID(Operand.getInstructionMatcher());
Table << MatchTable::Opcode("GIR_CopyOrAddZeroReg")
<< MatchTable::Comment("NewInsnID") << MatchTable::IntValue(NewInsnID)
<< MatchTable::Comment("OldInsnID")
<< MatchTable::IntValue(OldInsnVarID) << MatchTable::Comment("OpIdx")
<< MatchTable::IntValue(Operand.getOpIdx())
<< MatchTable::NamedValue(
(ZeroRegisterDef->getValue("Namespace")
? ZeroRegisterDef->getValueAsString("Namespace")
: ""),
ZeroRegisterDef->getName())
<< MatchTable::Comment(SymbolicName) << MatchTable::LineBreak;
}
};
/// A CopyConstantAsImmRenderer emits code to render a G_CONSTANT instruction to
/// an extended immediate operand.
class CopyConstantAsImmRenderer : public OperandRenderer {
protected:
unsigned NewInsnID;
/// The name of the operand.
const std::string SymbolicName;
bool Signed;
public:
CopyConstantAsImmRenderer(unsigned NewInsnID, StringRef SymbolicName)
: OperandRenderer(OR_CopyConstantAsImm), NewInsnID(NewInsnID),
SymbolicName(SymbolicName), Signed(true) {}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_CopyConstantAsImm;
}
const StringRef getSymbolicName() const { return SymbolicName; }
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
InstructionMatcher &InsnMatcher = Rule.getInstructionMatcher(SymbolicName);
unsigned OldInsnVarID = Rule.getInsnVarID(InsnMatcher);
Table << MatchTable::Opcode(Signed ? "GIR_CopyConstantAsSImm"
: "GIR_CopyConstantAsUImm")
<< MatchTable::Comment("NewInsnID") << MatchTable::IntValue(NewInsnID)
<< MatchTable::Comment("OldInsnID")
<< MatchTable::IntValue(OldInsnVarID)
<< MatchTable::Comment(SymbolicName) << MatchTable::LineBreak;
}
};
/// A CopyFConstantAsFPImmRenderer emits code to render a G_FCONSTANT
/// instruction to an extended immediate operand.
class CopyFConstantAsFPImmRenderer : public OperandRenderer {
protected:
unsigned NewInsnID;
/// The name of the operand.
const std::string SymbolicName;
public:
CopyFConstantAsFPImmRenderer(unsigned NewInsnID, StringRef SymbolicName)
: OperandRenderer(OR_CopyFConstantAsFPImm), NewInsnID(NewInsnID),
SymbolicName(SymbolicName) {}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_CopyFConstantAsFPImm;
}
const StringRef getSymbolicName() const { return SymbolicName; }
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
InstructionMatcher &InsnMatcher = Rule.getInstructionMatcher(SymbolicName);
unsigned OldInsnVarID = Rule.getInsnVarID(InsnMatcher);
Table << MatchTable::Opcode("GIR_CopyFConstantAsFPImm")
<< MatchTable::Comment("NewInsnID") << MatchTable::IntValue(NewInsnID)
<< MatchTable::Comment("OldInsnID")
<< MatchTable::IntValue(OldInsnVarID)
<< MatchTable::Comment(SymbolicName) << MatchTable::LineBreak;
}
};
/// A CopySubRegRenderer emits code to copy a single register operand from an
/// existing instruction to the one being built and indicate that only a
/// subregister should be copied.
class CopySubRegRenderer : public OperandRenderer {
protected:
unsigned NewInsnID;
/// The name of the operand.
const StringRef SymbolicName;
/// The subregister to extract.
const CodeGenSubRegIndex *SubReg;
public:
CopySubRegRenderer(unsigned NewInsnID, StringRef SymbolicName,
const CodeGenSubRegIndex *SubReg)
: OperandRenderer(OR_CopySubReg), NewInsnID(NewInsnID),
SymbolicName(SymbolicName), SubReg(SubReg) {}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_CopySubReg;
}
const StringRef getSymbolicName() const { return SymbolicName; }
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
const OperandMatcher &Operand = Rule.getOperandMatcher(SymbolicName);
unsigned OldInsnVarID = Rule.getInsnVarID(Operand.getInstructionMatcher());
Table << MatchTable::Opcode("GIR_CopySubReg")
<< MatchTable::Comment("NewInsnID") << MatchTable::IntValue(NewInsnID)
<< MatchTable::Comment("OldInsnID")
<< MatchTable::IntValue(OldInsnVarID) << MatchTable::Comment("OpIdx")
<< MatchTable::IntValue(Operand.getOpIdx())
<< MatchTable::Comment("SubRegIdx")
<< MatchTable::IntValue(SubReg->EnumValue)
<< MatchTable::Comment(SymbolicName) << MatchTable::LineBreak;
}
};
/// Adds a specific physical register to the instruction being built.
/// This is typically useful for WZR/XZR on AArch64.
class AddRegisterRenderer : public OperandRenderer {
protected:
unsigned InsnID;
const Record *RegisterDef;
public:
AddRegisterRenderer(unsigned InsnID, const Record *RegisterDef)
: OperandRenderer(OR_Register), InsnID(InsnID), RegisterDef(RegisterDef) {
}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_Register;
}
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIR_AddRegister")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::NamedValue(
(RegisterDef->getValue("Namespace")
? RegisterDef->getValueAsString("Namespace")
: ""),
RegisterDef->getName())
<< MatchTable::LineBreak;
}
};
/// Adds a specific temporary virtual register to the instruction being built.
/// This is used to chain instructions together when emitting multiple
/// instructions.
class TempRegRenderer : public OperandRenderer {
protected:
unsigned InsnID;
unsigned TempRegID;
bool IsDef;
public:
TempRegRenderer(unsigned InsnID, unsigned TempRegID, bool IsDef = false)
: OperandRenderer(OR_Register), InsnID(InsnID), TempRegID(TempRegID),
IsDef(IsDef) {}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_TempRegister;
}
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIR_AddTempRegister")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::Comment("TempRegID") << MatchTable::IntValue(TempRegID)
<< MatchTable::Comment("TempRegFlags");
if (IsDef)
Table << MatchTable::NamedValue("RegState::Define");
else
Table << MatchTable::IntValue(0);
Table << MatchTable::LineBreak;
}
};
/// Adds a specific immediate to the instruction being built.
class ImmRenderer : public OperandRenderer {
protected:
unsigned InsnID;
int64_t Imm;
public:
ImmRenderer(unsigned InsnID, int64_t Imm)
: OperandRenderer(OR_Imm), InsnID(InsnID), Imm(Imm) {}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_Imm;
}
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIR_AddImm") << MatchTable::Comment("InsnID")
<< MatchTable::IntValue(InsnID) << MatchTable::Comment("Imm")
<< MatchTable::IntValue(Imm) << MatchTable::LineBreak;
}
};
/// Adds operands by calling a renderer function supplied by the ComplexPattern
/// matcher function.
class RenderComplexPatternOperand : public OperandRenderer {
private:
unsigned InsnID;
const Record &TheDef;
/// The name of the operand.
const StringRef SymbolicName;
/// The renderer number. This must be unique within a rule since it's used to
/// identify a temporary variable to hold the renderer function.
unsigned RendererID;
/// When provided, this is the suboperand of the ComplexPattern operand to
/// render. Otherwise all the suboperands will be rendered.
Optional<unsigned> SubOperand;
unsigned getNumOperands() const {
return TheDef.getValueAsDag("Operands")->getNumArgs();
}
public:
RenderComplexPatternOperand(unsigned InsnID, const Record &TheDef,
StringRef SymbolicName, unsigned RendererID,
Optional<unsigned> SubOperand = None)
: OperandRenderer(OR_ComplexPattern), InsnID(InsnID), TheDef(TheDef),
SymbolicName(SymbolicName), RendererID(RendererID),
SubOperand(SubOperand) {}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_ComplexPattern;
}
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
Table << MatchTable::Opcode(SubOperand.hasValue() ? "GIR_ComplexSubOperandRenderer"
: "GIR_ComplexRenderer")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::Comment("RendererID")
<< MatchTable::IntValue(RendererID);
if (SubOperand.hasValue())
Table << MatchTable::Comment("SubOperand")
<< MatchTable::IntValue(SubOperand.getValue());
Table << MatchTable::Comment(SymbolicName) << MatchTable::LineBreak;
}
};
class CustomRenderer : public OperandRenderer {
protected:
unsigned InsnID;
const Record &Renderer;
/// The name of the operand.
const std::string SymbolicName;
public:
CustomRenderer(unsigned InsnID, const Record &Renderer,
StringRef SymbolicName)
: OperandRenderer(OR_Custom), InsnID(InsnID), Renderer(Renderer),
SymbolicName(SymbolicName) {}
static bool classof(const OperandRenderer *R) {
return R->getKind() == OR_Custom;
}
void emitRenderOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
InstructionMatcher &InsnMatcher = Rule.getInstructionMatcher(SymbolicName);
unsigned OldInsnVarID = Rule.getInsnVarID(InsnMatcher);
Table << MatchTable::Opcode("GIR_CustomRenderer")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::Comment("OldInsnID")
<< MatchTable::IntValue(OldInsnVarID)
<< MatchTable::Comment("Renderer")
<< MatchTable::NamedValue(
"GICR_" + Renderer.getValueAsString("RendererFn").str())
<< MatchTable::Comment(SymbolicName) << MatchTable::LineBreak;
}
};
/// An action taken when all Matcher predicates succeeded for a parent rule.
///
/// Typical actions include:
/// * Changing the opcode of an instruction.
/// * Adding an operand to an instruction.
class MatchAction {
public:
virtual ~MatchAction() {}
/// Emit the MatchTable opcodes to implement the action.
virtual void emitActionOpcodes(MatchTable &Table,
RuleMatcher &Rule) const = 0;
};
/// Generates a comment describing the matched rule being acted upon.
class DebugCommentAction : public MatchAction {
private:
std::string S;
public:
DebugCommentAction(StringRef S) : S(S) {}
void emitActionOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
Table << MatchTable::Comment(S) << MatchTable::LineBreak;
}
};
/// Generates code to build an instruction or mutate an existing instruction
/// into the desired instruction when this is possible.
class BuildMIAction : public MatchAction {
private:
unsigned InsnID;
const CodeGenInstruction *I;
InstructionMatcher *Matched;
std::vector<std::unique_ptr<OperandRenderer>> OperandRenderers;
/// True if the instruction can be built solely by mutating the opcode.
bool canMutate(RuleMatcher &Rule, const InstructionMatcher *Insn) const {
if (!Insn)
return false;
if (OperandRenderers.size() != Insn->getNumOperands())
return false;
for (const auto &Renderer : enumerate(OperandRenderers)) {
if (const auto *Copy = dyn_cast<CopyRenderer>(&*Renderer.value())) {
const OperandMatcher &OM = Rule.getOperandMatcher(Copy->getSymbolicName());
if (Insn != &OM.getInstructionMatcher() ||
OM.getOpIdx() != Renderer.index())
return false;
} else
return false;
}
return true;
}
public:
BuildMIAction(unsigned InsnID, const CodeGenInstruction *I)
: InsnID(InsnID), I(I), Matched(nullptr) {}
unsigned getInsnID() const { return InsnID; }
const CodeGenInstruction *getCGI() const { return I; }
void chooseInsnToMutate(RuleMatcher &Rule) {
for (auto *MutateCandidate : Rule.mutatable_insns()) {
if (canMutate(Rule, MutateCandidate)) {
// Take the first one we're offered that we're able to mutate.
Rule.reserveInsnMatcherForMutation(MutateCandidate);
Matched = MutateCandidate;
return;
}
}
}
template <class Kind, class... Args>
Kind &addRenderer(Args&&... args) {
OperandRenderers.emplace_back(
llvm::make_unique<Kind>(InsnID, std::forward<Args>(args)...));
return *static_cast<Kind *>(OperandRenderers.back().get());
}
void emitActionOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
if (Matched) {
assert(canMutate(Rule, Matched) &&
"Arranged to mutate an insn that isn't mutatable");
unsigned RecycleInsnID = Rule.getInsnVarID(*Matched);
Table << MatchTable::Opcode("GIR_MutateOpcode")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::Comment("RecycleInsnID")
<< MatchTable::IntValue(RecycleInsnID)
<< MatchTable::Comment("Opcode")
<< MatchTable::NamedValue(I->Namespace, I->TheDef->getName())
<< MatchTable::LineBreak;
if (!I->ImplicitDefs.empty() || !I->ImplicitUses.empty()) {
for (auto Def : I->ImplicitDefs) {
auto Namespace = Def->getValue("Namespace")
? Def->getValueAsString("Namespace")
: "";
Table << MatchTable::Opcode("GIR_AddImplicitDef")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::NamedValue(Namespace, Def->getName())
<< MatchTable::LineBreak;
}
for (auto Use : I->ImplicitUses) {
auto Namespace = Use->getValue("Namespace")
? Use->getValueAsString("Namespace")
: "";
Table << MatchTable::Opcode("GIR_AddImplicitUse")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::NamedValue(Namespace, Use->getName())
<< MatchTable::LineBreak;
}
}
return;
}
// TODO: Simple permutation looks like it could be almost as common as
// mutation due to commutative operations.
Table << MatchTable::Opcode("GIR_BuildMI") << MatchTable::Comment("InsnID")
<< MatchTable::IntValue(InsnID) << MatchTable::Comment("Opcode")
<< MatchTable::NamedValue(I->Namespace, I->TheDef->getName())
<< MatchTable::LineBreak;
for (const auto &Renderer : OperandRenderers)
Renderer->emitRenderOpcodes(Table, Rule);
if (I->mayLoad || I->mayStore) {
Table << MatchTable::Opcode("GIR_MergeMemOperands")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::Comment("MergeInsnID's");
// Emit the ID's for all the instructions that are matched by this rule.
// TODO: Limit this to matched instructions that mayLoad/mayStore or have
// some other means of having a memoperand. Also limit this to
// emitted instructions that expect to have a memoperand too. For
// example, (G_SEXT (G_LOAD x)) that results in separate load and
// sign-extend instructions shouldn't put the memoperand on the
// sign-extend since it has no effect there.
std::vector<unsigned> MergeInsnIDs;
for (const auto &IDMatcherPair : Rule.defined_insn_vars())
MergeInsnIDs.push_back(IDMatcherPair.second);
llvm::sort(MergeInsnIDs.begin(), MergeInsnIDs.end());
for (const auto &MergeInsnID : MergeInsnIDs)
Table << MatchTable::IntValue(MergeInsnID);
Table << MatchTable::NamedValue("GIU_MergeMemOperands_EndOfList")
<< MatchTable::LineBreak;
}
// FIXME: This is a hack but it's sufficient for ISel. We'll need to do
// better for combines. Particularly when there are multiple match
// roots.
if (InsnID == 0)
Table << MatchTable::Opcode("GIR_EraseFromParent")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::LineBreak;
}
};
/// Generates code to constrain the operands of an output instruction to the
/// register classes specified by the definition of that instruction.
class ConstrainOperandsToDefinitionAction : public MatchAction {
unsigned InsnID;
public:
ConstrainOperandsToDefinitionAction(unsigned InsnID) : InsnID(InsnID) {}
void emitActionOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIR_ConstrainSelectedInstOperands")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::LineBreak;
}
};
/// Generates code to constrain the specified operand of an output instruction
/// to the specified register class.
class ConstrainOperandToRegClassAction : public MatchAction {
unsigned InsnID;
unsigned OpIdx;
const CodeGenRegisterClass &RC;
public:
ConstrainOperandToRegClassAction(unsigned InsnID, unsigned OpIdx,
const CodeGenRegisterClass &RC)
: InsnID(InsnID), OpIdx(OpIdx), RC(RC) {}
void emitActionOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIR_ConstrainOperandRC")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::Comment("Op") << MatchTable::IntValue(OpIdx)
<< MatchTable::Comment("RC " + RC.getName())
<< MatchTable::IntValue(RC.EnumValue) << MatchTable::LineBreak;
}
};
/// Generates code to create a temporary register which can be used to chain
/// instructions together.
class MakeTempRegisterAction : public MatchAction {
private:
LLTCodeGen Ty;
unsigned TempRegID;
public:
MakeTempRegisterAction(const LLTCodeGen &Ty, unsigned TempRegID)
: Ty(Ty), TempRegID(TempRegID) {}
void emitActionOpcodes(MatchTable &Table, RuleMatcher &Rule) const override {
Table << MatchTable::Opcode("GIR_MakeTempReg")
<< MatchTable::Comment("TempRegID") << MatchTable::IntValue(TempRegID)
<< MatchTable::Comment("TypeID")
<< MatchTable::NamedValue(Ty.getCxxEnumValue())
<< MatchTable::LineBreak;
}
};
InstructionMatcher &RuleMatcher::addInstructionMatcher(StringRef SymbolicName) {
Matchers.emplace_back(new InstructionMatcher(*this, SymbolicName));
MutatableInsns.insert(Matchers.back().get());
return *Matchers.back();
}
void RuleMatcher::addRequiredFeature(Record *Feature) {
RequiredFeatures.push_back(Feature);
}
const std::vector<Record *> &RuleMatcher::getRequiredFeatures() const {
return RequiredFeatures;
}
// Emplaces an action of the specified Kind at the end of the action list.
//
// Returns a reference to the newly created action.
//
// Like std::vector::emplace_back(), may invalidate all iterators if the new
// size exceeds the capacity. Otherwise, only invalidates the past-the-end
// iterator.
template <class Kind, class... Args>
Kind &RuleMatcher::addAction(Args &&... args) {
Actions.emplace_back(llvm::make_unique<Kind>(std::forward<Args>(args)...));
return *static_cast<Kind *>(Actions.back().get());
}
// Emplaces an action of the specified Kind before the given insertion point.
//
// Returns an iterator pointing at the newly created instruction.
//
// Like std::vector::insert(), may invalidate all iterators if the new size
// exceeds the capacity. Otherwise, only invalidates the iterators from the
// insertion point onwards.
template <class Kind, class... Args>
action_iterator RuleMatcher::insertAction(action_iterator InsertPt,
Args &&... args) {
return Actions.emplace(InsertPt,
llvm::make_unique<Kind>(std::forward<Args>(args)...));
}
unsigned RuleMatcher::implicitlyDefineInsnVar(InstructionMatcher &Matcher) {
unsigned NewInsnVarID = NextInsnVarID++;
InsnVariableIDs[&Matcher] = NewInsnVarID;
return NewInsnVarID;
}
unsigned RuleMatcher::getInsnVarID(InstructionMatcher &InsnMatcher) const {
const auto &I = InsnVariableIDs.find(&InsnMatcher);
if (I != InsnVariableIDs.end())
return I->second;
llvm_unreachable("Matched Insn was not captured in a local variable");
}
void RuleMatcher::defineOperand(StringRef SymbolicName, OperandMatcher &OM) {
if (DefinedOperands.find(SymbolicName) == DefinedOperands.end()) {
DefinedOperands[SymbolicName] = &OM;
return;
}
// If the operand is already defined, then we must ensure both references in
// the matcher have the exact same node.
OM.addPredicate<SameOperandMatcher>(OM.getSymbolicName());
}
InstructionMatcher &
RuleMatcher::getInstructionMatcher(StringRef SymbolicName) const {
for (const auto &I : InsnVariableIDs)
if (I.first->getSymbolicName() == SymbolicName)
return *I.first;
llvm_unreachable(
("Failed to lookup instruction " + SymbolicName).str().c_str());
}
const OperandMatcher &
RuleMatcher::getOperandMatcher(StringRef Name) const {
const auto &I = DefinedOperands.find(Name);
if (I == DefinedOperands.end())
PrintFatalError(SrcLoc, "Operand " + Name + " was not declared in matcher");
return *I->second;
}
void RuleMatcher::emit(MatchTable &Table) {
if (Matchers.empty())
llvm_unreachable("Unexpected empty matcher!");
// The representation supports rules that require multiple roots such as:
// %ptr(p0) = ...
// %elt0(s32) = G_LOAD %ptr
// %1(p0) = G_ADD %ptr, 4
// %elt1(s32) = G_LOAD p0 %1
// which could be usefully folded into:
// %ptr(p0) = ...
// %elt0(s32), %elt1(s32) = TGT_LOAD_PAIR %ptr
// on some targets but we don't need to make use of that yet.
assert(Matchers.size() == 1 && "Cannot handle multi-root matchers yet");
unsigned LabelID = Table.allocateLabelID();
Table << MatchTable::Opcode("GIM_Try", +1)
<< MatchTable::Comment("On fail goto")
<< MatchTable::JumpTarget(LabelID)
<< MatchTable::Comment(("Rule ID " + Twine(RuleID) + " //").str())
<< MatchTable::LineBreak;
if (!RequiredFeatures.empty()) {
Table << MatchTable::Opcode("GIM_CheckFeatures")
<< MatchTable::NamedValue(getNameForFeatureBitset(RequiredFeatures))
<< MatchTable::LineBreak;
}
Matchers.front()->emitPredicateOpcodes(Table, *this);
// We must also check if it's safe to fold the matched instructions.
if (InsnVariableIDs.size() >= 2) {
// Invert the map to create stable ordering (by var names)
SmallVector<unsigned, 2> InsnIDs;
for (const auto &Pair : InsnVariableIDs) {
// Skip the root node since it isn't moving anywhere. Everything else is
// sinking to meet it.
if (Pair.first == Matchers.front().get())
continue;
InsnIDs.push_back(Pair.second);
}
llvm::sort(InsnIDs.begin(), InsnIDs.end());
for (const auto &InsnID : InsnIDs) {
// Reject the difficult cases until we have a more accurate check.
Table << MatchTable::Opcode("GIM_CheckIsSafeToFold")
<< MatchTable::Comment("InsnID") << MatchTable::IntValue(InsnID)
<< MatchTable::LineBreak;
// FIXME: Emit checks to determine it's _actually_ safe to fold and/or
// account for unsafe cases.
//
// Example:
// MI1--> %0 = ...
// %1 = ... %0
// MI0--> %2 = ... %0
// It's not safe to erase MI1. We currently handle this by not
// erasing %0 (even when it's dead).
//
// Example:
// MI1--> %0 = load volatile @a
// %1 = load volatile @a
// MI0--> %2 = ... %0
// It's not safe to sink %0's def past %1. We currently handle
// this by rejecting all loads.
//
// Example:
// MI1--> %0 = load @a
// %1 = store @a
// MI0--> %2 = ... %0
// It's not safe to sink %0's def past %1. We currently handle
// this by rejecting all loads.
//
// Example:
// G_CONDBR %cond, @BB1
// BB0:
// MI1--> %0 = load @a
// G_BR @BB1
// BB1:
// MI0--> %2 = ... %0
// It's not always safe to sink %0 across control flow. In this
// case it may introduce a memory fault. We currentl handle this
// by rejecting all loads.
}
}
for (const auto &PM : EpilogueMatchers)
PM->emitPredicateOpcodes(Table, *this);
for (const auto &MA : Actions)
MA->emitActionOpcodes(Table, *this);
if (Table.isWithCoverage())
Table << MatchTable::Opcode("GIR_Coverage") << MatchTable::IntValue(RuleID)
<< MatchTable::LineBreak;
else
Table << MatchTable::Comment(("GIR_Coverage, " + Twine(RuleID) + ",").str())
<< MatchTable::LineBreak;
Table << MatchTable::Opcode("GIR_Done", -1) << MatchTable::LineBreak
<< MatchTable::Label(LabelID);
++NumPatternEmitted;
}
bool RuleMatcher::isHigherPriorityThan(const RuleMatcher &B) const {
// Rules involving more match roots have higher priority.
if (Matchers.size() > B.Matchers.size())
return true;
if (Matchers.size() < B.Matchers.size())
return false;
for (const auto &Matcher : zip(Matchers, B.Matchers)) {
if (std::get<0>(Matcher)->isHigherPriorityThan(*std::get<1>(Matcher)))
return true;
if (std::get<1>(Matcher)->isHigherPriorityThan(*std::get<0>(Matcher)))
return false;
}
return false;
}
unsigned RuleMatcher::countRendererFns() const {
return std::accumulate(
Matchers.begin(), Matchers.end(), 0,
[](unsigned A, const std::unique_ptr<InstructionMatcher> &Matcher) {
return A + Matcher->countRendererFns();
});
}
bool OperandPredicateMatcher::isHigherPriorityThan(
const OperandPredicateMatcher &B) const {
// Generally speaking, an instruction is more important than an Int or a
// LiteralInt because it can cover more nodes but theres an exception to
// this. G_CONSTANT's are less important than either of those two because they
// are more permissive.
const InstructionOperandMatcher *AOM =
dyn_cast<InstructionOperandMatcher>(this);
const InstructionOperandMatcher *BOM =
dyn_cast<InstructionOperandMatcher>(&B);
bool AIsConstantInsn = AOM && AOM->getInsnMatcher().isConstantInstruction();
bool BIsConstantInsn = BOM && BOM->getInsnMatcher().isConstantInstruction();
if (AOM && BOM) {
// The relative priorities between a G_CONSTANT and any other instruction
// don't actually matter but this code is needed to ensure a strict weak
// ordering. This is particularly important on Windows where the rules will
// be incorrectly sorted without it.
if (AIsConstantInsn != BIsConstantInsn)
return AIsConstantInsn < BIsConstantInsn;
return false;
}
if (AOM && AIsConstantInsn && (B.Kind == OPM_Int || B.Kind == OPM_LiteralInt))
return false;
if (BOM && BIsConstantInsn && (Kind == OPM_Int || Kind == OPM_LiteralInt))
return true;
return Kind < B.Kind;
}
void SameOperandMatcher::emitPredicateOpcodes(MatchTable &Table,
RuleMatcher &Rule) const {
const OperandMatcher &OtherOM = Rule.getOperandMatcher(MatchingName);
unsigned OtherInsnVarID = Rule.getInsnVarID(OtherOM.getInstructionMatcher());
assert(OtherInsnVarID == OtherOM.getInstructionMatcher().getInsnVarID());
Table << MatchTable::Opcode("GIM_CheckIsSameOperand")
<< MatchTable::Comment("MI") << MatchTable::IntValue(InsnVarID)
<< MatchTable::Comment("OpIdx") << MatchTable::IntValue(OpIdx)
<< MatchTable::Comment("OtherMI")
<< MatchTable::IntValue(OtherInsnVarID)
<< MatchTable::Comment("OtherOpIdx")
<< MatchTable::IntValue(OtherOM.getOpIdx())
<< MatchTable::LineBreak;
}
//===- GlobalISelEmitter class --------------------------------------------===//
class GlobalISelEmitter {
public:
explicit GlobalISelEmitter(RecordKeeper &RK);
void run(raw_ostream &OS);
private:
const RecordKeeper &RK;
const CodeGenDAGPatterns CGP;
const CodeGenTarget &Target;
CodeGenRegBank CGRegs;
/// Keep track of the equivalence between SDNodes and Instruction by mapping
/// SDNodes to the GINodeEquiv mapping. We need to map to the GINodeEquiv to
/// check for attributes on the relation such as CheckMMOIsNonAtomic.
/// This is defined using 'GINodeEquiv' in the target description.
DenseMap<Record *, Record *> NodeEquivs;
/// Keep track of the equivalence between ComplexPattern's and
/// GIComplexOperandMatcher. Map entries are specified by subclassing
/// GIComplexPatternEquiv.
DenseMap<const Record *, const Record *> ComplexPatternEquivs;
/// Keep track of the equivalence between SDNodeXForm's and
/// GICustomOperandRenderer. Map entries are specified by subclassing
/// GISDNodeXFormEquiv.
DenseMap<const Record *, const Record *> SDNodeXFormEquivs;
/// Keep track of Scores of PatternsToMatch similar to how the DAG does.
/// This adds compatibility for RuleMatchers to use this for ordering rules.
DenseMap<uint64_t, int> RuleMatcherScores;
// Map of predicates to their subtarget features.
SubtargetFeatureInfoMap SubtargetFeatures;
// Rule coverage information.
Optional<CodeGenCoverage> RuleCoverage;
void gatherOpcodeValues();
void gatherTypeIDValues();
void gatherNodeEquivs();
Record *findNodeEquiv(Record *N) const;
const CodeGenInstruction *getEquivNode(Record &Equiv,
const TreePatternNode *N) const;
Error importRulePredicates(RuleMatcher &M, ArrayRef<Predicate> Predicates);
Expected<InstructionMatcher &> createAndImportSelDAGMatcher(
RuleMatcher &Rule, InstructionMatcher &InsnMatcher,
const TreePatternNode *Src, unsigned &TempOpIdx) const;
Error importComplexPatternOperandMatcher(OperandMatcher &OM, Record *R,
unsigned &TempOpIdx) const;
Error importChildMatcher(RuleMatcher &Rule, InstructionMatcher &InsnMatcher,
const TreePatternNode *SrcChild,
bool OperandIsAPointer, unsigned OpIdx,
unsigned &TempOpIdx) const;
Expected<BuildMIAction &>
createAndImportInstructionRenderer(RuleMatcher &M,
const TreePatternNode *Dst);
Expected<action_iterator> createAndImportSubInstructionRenderer(
action_iterator InsertPt, RuleMatcher &M, const TreePatternNode *Dst,
unsigned TempReg);
Expected<action_iterator>
createInstructionRenderer(action_iterator InsertPt, RuleMatcher &M,
const TreePatternNode *Dst);
void importExplicitDefRenderers(BuildMIAction &DstMIBuilder);
Expected<action_iterator>
importExplicitUseRenderers(action_iterator InsertPt, RuleMatcher &M,
BuildMIAction &DstMIBuilder,
const llvm::TreePatternNode *Dst);
Expected<action_iterator>
importExplicitUseRenderer(action_iterator InsertPt, RuleMatcher &Rule,
BuildMIAction &DstMIBuilder,
TreePatternNode *DstChild);
Error importDefaultOperandRenderers(BuildMIAction &DstMIBuilder,
DagInit *DefaultOps) const;
Error
importImplicitDefRenderers(BuildMIAction &DstMIBuilder,
const std::vector<Record *> &ImplicitDefs) const;
void emitImmPredicates(raw_ostream &OS, StringRef TypeIdentifier,
StringRef Type,
std::function<bool(const Record *R)> Filter);
/// Analyze pattern \p P, returning a matcher for it if possible.
/// Otherwise, return an Error explaining why we don't support it.
Expected<RuleMatcher> runOnPattern(const PatternToMatch &P);
void declareSubtargetFeature(Record *Predicate);
MatchTable buildMatchTable(MutableArrayRef<RuleMatcher> Rules, bool Optimize,
bool WithCoverage);
public:
/// Takes a sequence of \p Rules and group them based on the predicates
/// they share. \p MatcherStorage is used as a memory container
/// for the group that are created as part of this process.
///
/// What this optimization does looks like if GroupT = GroupMatcher:
/// Output without optimization:
/// \verbatim
/// # R1
/// # predicate A
/// # predicate B
/// ...
/// # R2
/// # predicate A // <-- effectively this is going to be checked twice.
/// // Once in R1 and once in R2.
/// # predicate C
/// \endverbatim
/// Output with optimization:
/// \verbatim
/// # Group1_2
/// # predicate A // <-- Check is now shared.
/// # R1
/// # predicate B
/// # R2
/// # predicate C
/// \endverbatim
template <class GroupT>
static std::vector<Matcher *> optimizeRules(
ArrayRef<Matcher *> Rules,
std::vector<std::unique_ptr<Matcher>> &MatcherStorage);
};
void GlobalISelEmitter::gatherOpcodeValues() {
InstructionOpcodeMatcher::initOpcodeValuesMap(Target);
}
void GlobalISelEmitter::gatherTypeIDValues() {
LLTOperandMatcher::initTypeIDValuesMap();
}
void GlobalISelEmitter::gatherNodeEquivs() {
assert(NodeEquivs.empty());
for (Record *Equiv : RK.getAllDerivedDefinitions("GINodeEquiv"))
NodeEquivs[Equiv->getValueAsDef("Node")] = Equiv;
assert(ComplexPatternEquivs.empty());
for (Record *Equiv : RK.getAllDerivedDefinitions("GIComplexPatternEquiv")) {
Record *SelDAGEquiv = Equiv->getValueAsDef("SelDAGEquivalent");
if (!SelDAGEquiv)
continue;
ComplexPatternEquivs[SelDAGEquiv] = Equiv;
}
assert(SDNodeXFormEquivs.empty());
for (Record *Equiv : RK.getAllDerivedDefinitions("GISDNodeXFormEquiv")) {
Record *SelDAGEquiv = Equiv->getValueAsDef("SelDAGEquivalent");
if (!SelDAGEquiv)
continue;
SDNodeXFormEquivs[SelDAGEquiv] = Equiv;
}
}
Record *GlobalISelEmitter::findNodeEquiv(Record *N) const {
return NodeEquivs.lookup(N);
}
const CodeGenInstruction *
GlobalISelEmitter::getEquivNode(Record &Equiv, const TreePatternNode *N) const {
for (const auto &Predicate : N->getPredicateFns()) {
if (!Equiv.isValueUnset("IfSignExtend") && Predicate.isLoad() &&
Predicate.isSignExtLoad())
return &Target.getInstruction(Equiv.getValueAsDef("IfSignExtend"));
if (!Equiv.isValueUnset("IfZeroExtend") && Predicate.isLoad() &&
Predicate.isZeroExtLoad())
return &Target.getInstruction(Equiv.getValueAsDef("IfZeroExtend"));
}
return &Target.getInstruction(Equiv.getValueAsDef("I"));
}
GlobalISelEmitter::GlobalISelEmitter(RecordKeeper &RK)
: RK(RK), CGP(RK), Target(CGP.getTargetInfo()),
CGRegs(RK, Target.getHwModes()) {}
//===- Emitter ------------------------------------------------------------===//
Error
GlobalISelEmitter::importRulePredicates(RuleMatcher &M,
ArrayRef<Predicate> Predicates) {
for (const Predicate &P : Predicates) {
if (!P.Def)
continue;
declareSubtargetFeature(P.Def);
M.addRequiredFeature(P.Def);
}
return Error::success();
}
Expected<InstructionMatcher &> GlobalISelEmitter::createAndImportSelDAGMatcher(
RuleMatcher &Rule, InstructionMatcher &InsnMatcher,
const TreePatternNode *Src, unsigned &TempOpIdx) const {
Record *SrcGIEquivOrNull = nullptr;
const CodeGenInstruction *SrcGIOrNull = nullptr;
// Start with the defined operands (i.e., the results of the root operator).
if (Src->getExtTypes().size() > 1)
return failedImport("Src pattern has multiple results");
if (Src->isLeaf()) {
Init *SrcInit = Src->getLeafValue();
if (isa<IntInit>(SrcInit)) {
InsnMatcher.addPredicate<InstructionOpcodeMatcher>(
&Target.getInstruction(RK.getDef("G_CONSTANT")));
} else
return failedImport(
"Unable to deduce gMIR opcode to handle Src (which is a leaf)");
} else {
SrcGIEquivOrNull = findNodeEquiv(Src->getOperator());
if (!SrcGIEquivOrNull)
return failedImport("Pattern operator lacks an equivalent Instruction" +
explainOperator(Src->getOperator()));
SrcGIOrNull = getEquivNode(*SrcGIEquivOrNull, Src);
// The operators look good: match the opcode
InsnMatcher.addPredicate<InstructionOpcodeMatcher>(SrcGIOrNull);
}
unsigned OpIdx = 0;
for (const TypeSetByHwMode &VTy : Src->getExtTypes()) {
// Results don't have a name unless they are the root node. The caller will
// set the name if appropriate.
OperandMatcher &OM = InsnMatcher.addOperand(OpIdx++, "", TempOpIdx);
if (auto Error = OM.addTypeCheckPredicate(VTy, false /* OperandIsAPointer */))
return failedImport(toString(std::move(Error)) +
" for result of Src pattern operator");
}
for (const auto &Predicate : Src->getPredicateFns()) {
if (Predicate.isAlwaysTrue())
continue;
if (Predicate.isImmediatePattern()) {
InsnMatcher.addPredicate<InstructionImmPredicateMatcher>(Predicate);
continue;
}
// G_LOAD is used for both non-extending and any-extending loads.
if (Predicate.isLoad() && Predicate.isNonExtLoad()) {
InsnMatcher.addPredicate<MemoryVsLLTSizePredicateMatcher>(
0, MemoryVsLLTSizePredicateMatcher::EqualTo, 0);
continue;
}
if (Predicate.isLoad() && Predicate.isAnyExtLoad()) {
InsnMatcher.addPredicate<MemoryVsLLTSizePredicateMatcher>(
0, MemoryVsLLTSizePredicateMatcher::LessThan, 0);
continue;
}
// No check required. We already did it by swapping the opcode.
if (!SrcGIEquivOrNull->isValueUnset("IfSignExtend") &&
Predicate.isSignExtLoad())
continue;
// No check required. We already did it by swapping the opcode.
if (!SrcGIEquivOrNull->isValueUnset("IfZeroExtend") &&
Predicate.isZeroExtLoad())
continue;
// No check required. G_STORE by itself is a non-extending store.
if (Predicate.isNonTruncStore())
continue;
if (Predicate.isLoad() || Predicate.isStore() || Predicate.isAtomic()) {
if (Predicate.getMemoryVT() != nullptr) {
Optional<LLTCodeGen> MemTyOrNone =
MVTToLLT(getValueType(Predicate.getMemoryVT()));
if (!MemTyOrNone)
return failedImport("MemVT could not be converted to LLT");
// MMO's work in bytes so we must take care of unusual types like i1
// don't round down.
unsigned MemSizeInBits =
llvm::alignTo(MemTyOrNone->get().getSizeInBits(), 8);
InsnMatcher.addPredicate<MemorySizePredicateMatcher>(
0, MemSizeInBits / 8);
continue;
}
}
if (Predicate.isLoad() || Predicate.isStore()) {
// No check required. A G_LOAD/G_STORE is an unindexed load.
if (Predicate.isUnindexed())
continue;
}
if (Predicate.isAtomic()) {
if (Predicate.isAtomicOrderingMonotonic()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>(
"Monotonic");
continue;
}
if (Predicate.isAtomicOrderingAcquire()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>("Acquire");
continue;
}
if (Predicate.isAtomicOrderingRelease()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>("Release");
continue;
}
if (Predicate.isAtomicOrderingAcquireRelease()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>(
"AcquireRelease");
continue;
}
if (Predicate.isAtomicOrderingSequentiallyConsistent()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>(
"SequentiallyConsistent");
continue;
}
if (Predicate.isAtomicOrderingAcquireOrStronger()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>(
"Acquire", AtomicOrderingMMOPredicateMatcher::AO_OrStronger);
continue;
}
if (Predicate.isAtomicOrderingWeakerThanAcquire()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>(
"Acquire", AtomicOrderingMMOPredicateMatcher::AO_WeakerThan);
continue;
}
if (Predicate.isAtomicOrderingReleaseOrStronger()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>(
"Release", AtomicOrderingMMOPredicateMatcher::AO_OrStronger);
continue;
}
if (Predicate.isAtomicOrderingWeakerThanRelease()) {
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>(
"Release", AtomicOrderingMMOPredicateMatcher::AO_WeakerThan);
continue;
}
}
return failedImport("Src pattern child has predicate (" +
explainPredicates(Src) + ")");
}
if (SrcGIEquivOrNull && SrcGIEquivOrNull->getValueAsBit("CheckMMOIsNonAtomic"))
InsnMatcher.addPredicate<AtomicOrderingMMOPredicateMatcher>("NotAtomic");
if (Src->isLeaf()) {
Init *SrcInit = Src->getLeafValue();
if (IntInit *SrcIntInit = dyn_cast<IntInit>(SrcInit)) {
OperandMatcher &OM =
InsnMatcher.addOperand(OpIdx++, Src->getName(), TempOpIdx);
OM.addPredicate<LiteralIntOperandMatcher>(SrcIntInit->getValue());
} else
return failedImport(
"Unable to deduce gMIR opcode to handle Src (which is a leaf)");
} else {
assert(SrcGIOrNull &&
"Expected to have already found an equivalent Instruction");
if (SrcGIOrNull->TheDef->getName() == "G_CONSTANT" ||
SrcGIOrNull->TheDef->getName() == "G_FCONSTANT") {
// imm/fpimm still have operands but we don't need to do anything with it
// here since we don't support ImmLeaf predicates yet. However, we still
// need to note the hidden operand to get GIM_CheckNumOperands correct.
InsnMatcher.addOperand(OpIdx++, "", TempOpIdx);
return InsnMatcher;
}
// Match the used operands (i.e. the children of the operator).
for (unsigned i = 0, e = Src->getNumChildren(); i != e; ++i) {
TreePatternNode *SrcChild = Src->getChild(i);
// SelectionDAG allows pointers to be represented with iN since it doesn't
// distinguish between pointers and integers but they are different types in GlobalISel.
// Coerce integers to pointers to address space 0 if the context indicates a pointer.
bool OperandIsAPointer = SrcGIOrNull->isOperandAPointer(i);
// For G_INTRINSIC/G_INTRINSIC_W_SIDE_EFFECTS, the operand immediately
// following the defs is an intrinsic ID.
if ((SrcGIOrNull->TheDef->getName() == "G_INTRINSIC" ||
SrcGIOrNull->TheDef->getName() == "G_INTRINSIC_W_SIDE_EFFECTS") &&
i == 0) {
if (const CodeGenIntrinsic *II = Src->getIntrinsicInfo(CGP)) {
OperandMatcher &OM =
InsnMatcher.addOperand(OpIdx++, SrcChild->getName(), TempOpIdx);
OM.addPredicate<IntrinsicIDOperandMatcher>(II);
continue;
}
return failedImport("Expected IntInit containing instrinsic ID)");
}
if (auto Error =
importChildMatcher(Rule, InsnMatcher, SrcChild, OperandIsAPointer,
OpIdx++, TempOpIdx))
return std::move(Error);
}
}
return InsnMatcher;
}
Error GlobalISelEmitter::importComplexPatternOperandMatcher(
OperandMatcher &OM, Record *R, unsigned &TempOpIdx) const {
const auto &ComplexPattern = ComplexPatternEquivs.find(R);
if (ComplexPattern == ComplexPatternEquivs.end())
return failedImport("SelectionDAG ComplexPattern (" + R->getName() +
") not mapped to GlobalISel");
OM.addPredicate<ComplexPatternOperandMatcher>(OM, *ComplexPattern->second);
TempOpIdx++;
return Error::success();
}
Error GlobalISelEmitter::importChildMatcher(RuleMatcher &Rule,
InstructionMatcher &InsnMatcher,
const TreePatternNode *SrcChild,
bool OperandIsAPointer,
unsigned OpIdx,
unsigned &TempOpIdx) const {
OperandMatcher &OM =
InsnMatcher.addOperand(OpIdx, SrcChild->getName(), TempOpIdx);
if (OM.isSameAsAnotherOperand())
return Error::success();
ArrayRef<TypeSetByHwMode> ChildTypes = SrcChild->getExtTypes();
if (ChildTypes.size() != 1)
return failedImport("Src pattern child has multiple results");
// Check MBB's before the type check since they are not a known type.
if (!SrcChild->isLeaf()) {
if (SrcChild->getOperator()->isSubClassOf("SDNode")) {
auto &ChildSDNI = CGP.getSDNodeInfo(SrcChild->getOperator());
if (ChildSDNI.getSDClassName() == "BasicBlockSDNode") {
OM.addPredicate<MBBOperandMatcher>();
return Error::success();
}
}
}
if (auto Error =
OM.addTypeCheckPredicate(ChildTypes.front(), OperandIsAPointer))
return failedImport(toString(std::move(Error)) + " for Src operand (" +
to_string(*SrcChild) + ")");
// Check for nested instructions.
if (!SrcChild->isLeaf()) {
if (SrcChild->getOperator()->isSubClassOf("ComplexPattern")) {
// When a ComplexPattern is used as an operator, it should do the same
// thing as when used as a leaf. However, the children of the operator
// name the sub-operands that make up the complex operand and we must
// prepare to reference them in the renderer too.
unsigned RendererID = TempOpIdx;
if (auto Error = importComplexPatternOperandMatcher(
OM, SrcChild->getOperator(), TempOpIdx))
return Error;
for (unsigned i = 0, e = SrcChild->getNumChildren(); i != e; ++i) {
auto *SubOperand = SrcChild->getChild(i);
if (!SubOperand->getName().empty())
Rule.defineComplexSubOperand(SubOperand->getName(),
SrcChild->getOperator(), RendererID, i);
}
return Error::success();
}
auto MaybeInsnOperand = OM.addPredicate<InstructionOperandMatcher>(
InsnMatcher.getRuleMatcher(), SrcChild->getName());
if (!MaybeInsnOperand.hasValue()) {
// This isn't strictly true. If the user were to provide exactly the same
// matchers as the original operand then we could allow it. However, it's
// simpler to not permit the redundant specification.
return failedImport("Nested instruction cannot be the same as another operand");
}
// Map the node to a gMIR instruction.
InstructionOperandMatcher &InsnOperand = **MaybeInsnOperand;
auto InsnMatcherOrError = createAndImportSelDAGMatcher(
Rule, InsnOperand.getInsnMatcher(), SrcChild, TempOpIdx);
if (auto Error = InsnMatcherOrError.takeError())
return Error;
return Error::success();
}
if (SrcChild->hasAnyPredicate())
return failedImport("Src pattern child has unsupported predicate");
// Check for constant immediates.
if (auto *ChildInt = dyn_cast<IntInit>(SrcChild->getLeafValue())) {
OM.addPredicate<ConstantIntOperandMatcher>(ChildInt->getValue());
return Error::success();
}
// Check for def's like register classes or ComplexPattern's.
if (auto *ChildDefInit = dyn_cast<DefInit>(SrcChild->getLeafValue())) {
auto *ChildRec = ChildDefInit->getDef();
// Check for register classes.
if (ChildRec->isSubClassOf("RegisterClass") ||
ChildRec->isSubClassOf("RegisterOperand")) {
OM.addPredicate<RegisterBankOperandMatcher>(
Target.getRegisterClass(getInitValueAsRegClass(ChildDefInit)));
return Error::success();
}
// Check for ValueType.
if (ChildRec->isSubClassOf("ValueType")) {
// We already added a type check as standard practice so this doesn't need
// to do anything.
return Error::success();
}
// Check for ComplexPattern's.
if (ChildRec->isSubClassOf("ComplexPattern"))
return importComplexPatternOperandMatcher(OM, ChildRec, TempOpIdx);
if (ChildRec->isSubClassOf("ImmLeaf")) {
return failedImport(
"Src pattern child def is an unsupported tablegen class (ImmLeaf)");
}
return failedImport(
"Src pattern child def is an unsupported tablegen class");
}
return failedImport("Src pattern child is an unsupported kind");
}
Expected<action_iterator> GlobalISelEmitter::importExplicitUseRenderer(
action_iterator InsertPt, RuleMatcher &Rule, BuildMIAction &DstMIBuilder,
TreePatternNode *DstChild) {
const auto &SubOperand = Rule.getComplexSubOperand(DstChild->getName());
if (SubOperand.hasValue()) {
DstMIBuilder.addRenderer<RenderComplexPatternOperand>(
*std::get<0>(*SubOperand), DstChild->getName(),
std::get<1>(*SubOperand), std::get<2>(*SubOperand));
return InsertPt;
}
if (!DstChild->isLeaf()) {
if (DstChild->getOperator()->isSubClassOf("SDNodeXForm")) {
auto Child = DstChild->getChild(0);
auto I = SDNodeXFormEquivs.find(DstChild->getOperator());
if (I != SDNodeXFormEquivs.end()) {
DstMIBuilder.addRenderer<CustomRenderer>(*I->second, Child->getName());
return InsertPt;
}
return failedImport("SDNodeXForm " + Child->getName() +
" has no custom renderer");
}
// We accept 'bb' here. It's an operator because BasicBlockSDNode isn't
// inline, but in MI it's just another operand.
if (DstChild->getOperator()->isSubClassOf("SDNode")) {
auto &ChildSDNI = CGP.getSDNodeInfo(DstChild->getOperator());
if (ChildSDNI.getSDClassName() == "BasicBlockSDNode") {
DstMIBuilder.addRenderer<CopyRenderer>(DstChild->getName());
return InsertPt;
}
}
// Similarly, imm is an operator in TreePatternNode's view but must be
// rendered as operands.
// FIXME: The target should be able to choose sign-extended when appropriate
// (e.g. on Mips).
if (DstChild->getOperator()->getName() == "imm") {
DstMIBuilder.addRenderer<CopyConstantAsImmRenderer>(DstChild->getName());
return InsertPt;
} else if (DstChild->getOperator()->getName() == "fpimm") {
DstMIBuilder.addRenderer<CopyFConstantAsFPImmRenderer>(
DstChild->getName());
return InsertPt;
}
if (DstChild->getOperator()->isSubClassOf("Instruction")) {
ArrayRef<TypeSetByHwMode> ChildTypes = DstChild->getExtTypes();
if (ChildTypes.size() != 1)
return failedImport("Dst pattern child has multiple results");
Optional<LLTCodeGen> OpTyOrNone = None;
if (ChildTypes.front().isMachineValueType())
OpTyOrNone =
MVTToLLT(ChildTypes.front().getMachineValueType().SimpleTy);
if (!OpTyOrNone)
return failedImport("Dst operand has an unsupported type");
unsigned TempRegID = Rule.allocateTempRegID();
InsertPt = Rule.insertAction<MakeTempRegisterAction>(
InsertPt, OpTyOrNone.getValue(), TempRegID);
DstMIBuilder.addRenderer<TempRegRenderer>(TempRegID);
auto InsertPtOrError = createAndImportSubInstructionRenderer(
++InsertPt, Rule, DstChild, TempRegID);
if (auto Error = InsertPtOrError.takeError())
return std::move(Error);
return InsertPtOrError.get();
}
return failedImport("Dst pattern child isn't a leaf node or an MBB" + llvm::to_string(*DstChild));
}
// It could be a specific immediate in which case we should just check for
// that immediate.
if (const IntInit *ChildIntInit =
dyn_cast<IntInit>(DstChild->getLeafValue())) {
DstMIBuilder.addRenderer<ImmRenderer>(ChildIntInit->getValue());
return InsertPt;
}
// Otherwise, we're looking for a bog-standard RegisterClass operand.
if (auto *ChildDefInit = dyn_cast<DefInit>(DstChild->getLeafValue())) {
auto *ChildRec = ChildDefInit->getDef();
ArrayRef<TypeSetByHwMode> ChildTypes = DstChild->getExtTypes();
if (ChildTypes.size() != 1)
return failedImport("Dst pattern child has multiple results");
Optional<LLTCodeGen> OpTyOrNone = None;
if (ChildTypes.front().isMachineValueType())
OpTyOrNone = MVTToLLT(ChildTypes.front().getMachineValueType().SimpleTy);
if (!OpTyOrNone)
return failedImport("Dst operand has an unsupported type");
if (ChildRec->isSubClassOf("Register")) {
DstMIBuilder.addRenderer<AddRegisterRenderer>(ChildRec);
return InsertPt;
}
if (ChildRec->isSubClassOf("RegisterClass") ||
ChildRec->isSubClassOf("RegisterOperand") ||
ChildRec->isSubClassOf("ValueType")) {
if (ChildRec->isSubClassOf("RegisterOperand") &&
!ChildRec->isValueUnset("GIZeroRegister")) {
DstMIBuilder.addRenderer<CopyOrAddZeroRegRenderer>(
DstChild->getName(), ChildRec->getValueAsDef("GIZeroRegister"));
return InsertPt;
}
DstMIBuilder.addRenderer<CopyRenderer>(DstChild->getName());
return InsertPt;
}
if (ChildRec->isSubClassOf("ComplexPattern")) {
const auto &ComplexPattern = ComplexPatternEquivs.find(ChildRec);
if (ComplexPattern == ComplexPatternEquivs.end())
return failedImport(
"SelectionDAG ComplexPattern not mapped to GlobalISel");
const OperandMatcher &OM = Rule.getOperandMatcher(DstChild->getName());
DstMIBuilder.addRenderer<RenderComplexPatternOperand>(
*ComplexPattern->second, DstChild->getName(),
OM.getAllocatedTemporariesBaseID());
return InsertPt;
}
return failedImport(
"Dst pattern child def is an unsupported tablegen class");
}
return failedImport("Dst pattern child is an unsupported kind");
}
Expected<BuildMIAction &> GlobalISelEmitter::createAndImportInstructionRenderer(
RuleMatcher &M, const TreePatternNode *Dst) {
auto InsertPtOrError = createInstructionRenderer(M.actions_end(), M, Dst);
if (auto Error = InsertPtOrError.takeError())
return std::move(Error);
action_iterator InsertPt = InsertPtOrError.get();
BuildMIAction &DstMIBuilder = *static_cast<BuildMIAction *>(InsertPt->get());
importExplicitDefRenderers(DstMIBuilder);
if (auto Error = importExplicitUseRenderers(InsertPt, M, DstMIBuilder, Dst)
.takeError())
return std::move(Error);
return DstMIBuilder;
}
Expected<action_iterator>
GlobalISelEmitter::createAndImportSubInstructionRenderer(
const action_iterator InsertPt, RuleMatcher &M, const TreePatternNode *Dst,
unsigned TempRegID) {
auto InsertPtOrError = createInstructionRenderer(InsertPt, M, Dst);
// TODO: Assert there's exactly one result.
if (auto Error = InsertPtOrError.takeError())
return std::move(Error);
BuildMIAction &DstMIBuilder =
*static_cast<BuildMIAction *>(InsertPtOrError.get()->get());
// Assign the result to TempReg.
DstMIBuilder.addRenderer<TempRegRenderer>(TempRegID, true);
InsertPtOrError =
importExplicitUseRenderers(InsertPtOrError.get(), M, DstMIBuilder, Dst);
if (auto Error = InsertPtOrError.takeError())
return std::move(Error);
M.insertAction<ConstrainOperandsToDefinitionAction>(InsertPt,
DstMIBuilder.getInsnID());
return InsertPtOrError.get();
}
Expected<action_iterator> GlobalISelEmitter::createInstructionRenderer(
action_iterator InsertPt, RuleMatcher &M, const TreePatternNode *Dst) {
Record *DstOp = Dst->getOperator();
if (!DstOp->isSubClassOf("Instruction")) {
if (DstOp->isSubClassOf("ValueType"))
return failedImport(
"Pattern operator isn't an instruction (it's a ValueType)");
return failedImport("Pattern operator isn't an instruction");
}
CodeGenInstruction *DstI = &Target.getInstruction(DstOp);
// COPY_TO_REGCLASS is just a copy with a ConstrainOperandToRegClassAction
// attached. Similarly for EXTRACT_SUBREG except that's a subregister copy.
if (DstI->TheDef->getName() == "COPY_TO_REGCLASS")
DstI = &Target.getInstruction(RK.getDef("COPY"));
else if (DstI->TheDef->getName() == "EXTRACT_SUBREG")
DstI = &Target.getInstruction(RK.getDef("COPY"));
else if (DstI->TheDef->getName() == "REG_SEQUENCE")
return failedImport("Unable to emit REG_SEQUENCE");
return M.insertAction<BuildMIAction>(InsertPt, M.allocateOutputInsnID(),
DstI);
}
void GlobalISelEmitter::importExplicitDefRenderers(
BuildMIAction &DstMIBuilder) {
const CodeGenInstruction *DstI = DstMIBuilder.getCGI();
for (unsigned I = 0; I < DstI->Operands.NumDefs; ++I) {
const CGIOperandList::OperandInfo &DstIOperand = DstI->Operands[I];
DstMIBuilder.addRenderer<CopyRenderer>(DstIOperand.Name);
}
}
Expected<action_iterator> GlobalISelEmitter::importExplicitUseRenderers(
action_iterator InsertPt, RuleMatcher &M, BuildMIAction &DstMIBuilder,
const llvm::TreePatternNode *Dst) {
const CodeGenInstruction *DstI = DstMIBuilder.getCGI();
CodeGenInstruction *OrigDstI = &Target.getInstruction(Dst->getOperator());
// EXTRACT_SUBREG needs to use a subregister COPY.
if (OrigDstI->TheDef->getName() == "EXTRACT_SUBREG") {
if (!Dst->getChild(0)->isLeaf())
return failedImport("EXTRACT_SUBREG child #1 is not a leaf");
if (DefInit *SubRegInit =
dyn_cast<DefInit>(Dst->getChild(1)->getLeafValue())) {
Record *RCDef = getInitValueAsRegClass(Dst->getChild(0)->getLeafValue());
if (!RCDef)
return failedImport("EXTRACT_SUBREG child #0 could not "
"be coerced to a register class");
CodeGenRegisterClass *RC = CGRegs.getRegClass(RCDef);
CodeGenSubRegIndex *SubIdx = CGRegs.getSubRegIdx(SubRegInit->getDef());
const auto &SrcRCDstRCPair =
RC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx);
if (SrcRCDstRCPair.hasValue()) {
assert(SrcRCDstRCPair->second && "Couldn't find a matching subclass");
if (SrcRCDstRCPair->first != RC)
return failedImport("EXTRACT_SUBREG requires an additional COPY");
}
DstMIBuilder.addRenderer<CopySubRegRenderer>(Dst->getChild(0)->getName(),
SubIdx);
return InsertPt;
}
return failedImport("EXTRACT_SUBREG child #1 is not a subreg index");
}
// Render the explicit uses.
unsigned DstINumUses = OrigDstI->Operands.size() - OrigDstI->Operands.NumDefs;
unsigned ExpectedDstINumUses = Dst->getNumChildren();
if (OrigDstI->TheDef->getName() == "COPY_TO_REGCLASS") {
DstINumUses--; // Ignore the class constraint.
ExpectedDstINumUses--;
}
unsigned Child = 0;
unsigned NumDefaultOps = 0;
for (unsigned I = 0; I != DstINumUses; ++I) {
const CGIOperandList::OperandInfo &DstIOperand =
DstI->Operands[DstI->Operands.NumDefs + I];
// If the operand has default values, introduce them now.
// FIXME: Until we have a decent test case that dictates we should do
// otherwise, we're going to assume that operands with default values cannot
// be specified in the patterns. Therefore, adding them will not cause us to
// end up with too many rendered operands.
if (DstIOperand.Rec->isSubClassOf("OperandWithDefaultOps")) {
DagInit *DefaultOps = DstIOperand.Rec->getValueAsDag("DefaultOps");
if (auto Error = importDefaultOperandRenderers(DstMIBuilder, DefaultOps))
return std::move(Error);
++NumDefaultOps;
continue;
}
auto InsertPtOrError = importExplicitUseRenderer(InsertPt, M, DstMIBuilder,
Dst->getChild(Child));
if (auto Error = InsertPtOrError.takeError())
return std::move(Error);
InsertPt = InsertPtOrError.get();
++Child;
}
if (NumDefaultOps + ExpectedDstINumUses != DstINumUses)
return failedImport("Expected " + llvm::to_string(DstINumUses) +
" used operands but found " +
llvm::to_string(ExpectedDstINumUses) +
" explicit ones and " + llvm::to_string(NumDefaultOps) +
" default ones");
return InsertPt;
}
Error GlobalISelEmitter::importDefaultOperandRenderers(
BuildMIAction &DstMIBuilder, DagInit *DefaultOps) const {
for (const auto *DefaultOp : DefaultOps->getArgs()) {
// Look through ValueType operators.
if (const DagInit *DefaultDagOp = dyn_cast<DagInit>(DefaultOp)) {
if (const DefInit *DefaultDagOperator =
dyn_cast<DefInit>(DefaultDagOp->getOperator())) {
if (DefaultDagOperator->getDef()->isSubClassOf("ValueType"))
DefaultOp = DefaultDagOp->getArg(0);
}
}
if (const DefInit *DefaultDefOp = dyn_cast<DefInit>(DefaultOp)) {
DstMIBuilder.addRenderer<AddRegisterRenderer>(DefaultDefOp->getDef());
continue;
}
if (const IntInit *DefaultIntOp = dyn_cast<IntInit>(DefaultOp)) {
DstMIBuilder.addRenderer<ImmRenderer>(DefaultIntOp->getValue());
continue;
}
return failedImport("Could not add default op");
}
return Error::success();
}
Error GlobalISelEmitter::importImplicitDefRenderers(
BuildMIAction &DstMIBuilder,
const std::vector<Record *> &ImplicitDefs) const {
if (!ImplicitDefs.empty())
return failedImport("Pattern defines a physical register");
return Error::success();
}
Expected<RuleMatcher> GlobalISelEmitter::runOnPattern(const PatternToMatch &P) {
// Keep track of the matchers and actions to emit.
int Score = P.getPatternComplexity(CGP);
RuleMatcher M(P.getSrcRecord()->getLoc());
RuleMatcherScores[M.getRuleID()] = Score;
M.addAction<DebugCommentAction>(llvm::to_string(*P.getSrcPattern()) +
" => " +
llvm::to_string(*P.getDstPattern()));
if (auto Error = importRulePredicates(M, P.getPredicates()))
return std::move(Error);
// Next, analyze the pattern operators.
TreePatternNode *Src = P.getSrcPattern();
TreePatternNode *Dst = P.getDstPattern();
// If the root of either pattern isn't a simple operator, ignore it.
if (auto Err = isTrivialOperatorNode(Dst))
return failedImport("Dst pattern root isn't a trivial operator (" +
toString(std::move(Err)) + ")");
if (auto Err = isTrivialOperatorNode(Src))
return failedImport("Src pattern root isn't a trivial operator (" +
toString(std::move(Err)) + ")");
// The different predicates and matchers created during
// addInstructionMatcher use the RuleMatcher M to set up their
// instruction ID (InsnVarID) that are going to be used when
// M is going to be emitted.
// However, the code doing the emission still relies on the IDs
// returned during that process by the RuleMatcher when issuing
// the recordInsn opcodes.
// Because of that:
// 1. The order in which we created the predicates
// and such must be the same as the order in which we emit them,
// and
// 2. We need to reset the generation of the IDs in M somewhere between
// addInstructionMatcher and emit
//
// FIXME: Long term, we don't want to have to rely on this implicit
// naming being the same. One possible solution would be to have
// explicit operator for operation capture and reference those.
// The plus side is that it would expose opportunities to share
// the capture accross rules. The downside is that it would
// introduce a dependency between predicates (captures must happen
// before their first use.)
InstructionMatcher &InsnMatcherTemp = M.addInstructionMatcher(Src->getName());
unsigned TempOpIdx = 0;
auto InsnMatcherOrError =
createAndImportSelDAGMatcher(M, InsnMatcherTemp, Src, TempOpIdx);
if (auto Error = InsnMatcherOrError.takeError())
return std::move(Error);
InstructionMatcher &InsnMatcher = InsnMatcherOrError.get();
if (Dst->isLeaf()) {
Record *RCDef = getInitValueAsRegClass(Dst->getLeafValue());
const CodeGenRegisterClass &RC = Target.getRegisterClass(RCDef);
if (RCDef) {
// We need to replace the def and all its uses with the specified
// operand. However, we must also insert COPY's wherever needed.
// For now, emit a copy and let the register allocator clean up.
auto &DstI = Target.getInstruction(RK.getDef("COPY"));
const auto &DstIOperand = DstI.Operands[0];
OperandMatcher &OM0 = InsnMatcher.getOperand(0);
OM0.setSymbolicName(DstIOperand.Name);
M.defineOperand(OM0.getSymbolicName(), OM0);
OM0.addPredicate<RegisterBankOperandMatcher>(RC);
auto &DstMIBuilder =
M.addAction<BuildMIAction>(M.allocateOutputInsnID(), &DstI);
DstMIBuilder.addRenderer<CopyRenderer>(DstIOperand.Name);
DstMIBuilder.addRenderer<CopyRenderer>(Dst->getName());
M.addAction<ConstrainOperandToRegClassAction>(0, 0, RC);
// We're done with this pattern! It's eligible for GISel emission; return
// it.
++NumPatternImported;
return std::move(M);
}
return failedImport("Dst pattern root isn't a known leaf");
}
// Start with the defined operands (i.e., the results of the root operator).
Record *DstOp = Dst->getOperator();
if (!DstOp->isSubClassOf("Instruction"))
return failedImport("Pattern operator isn't an instruction");
auto &DstI = Target.getInstruction(DstOp);
if (DstI.Operands.NumDefs != Src->getExtTypes().size())
return failedImport("Src pattern results and dst MI defs are different (" +
to_string(Src->getExtTypes().size()) + " def(s) vs " +
to_string(DstI.Operands.NumDefs) + " def(s))");
// The root of the match also has constraints on the register bank so that it
// matches the result instruction.
unsigned OpIdx = 0;
for (const TypeSetByHwMode &VTy : Src->getExtTypes()) {
(void)VTy;
const auto &DstIOperand = DstI.Operands[OpIdx];
Record *DstIOpRec = DstIOperand.Rec;
if (DstI.TheDef->getName() == "COPY_TO_REGCLASS") {
DstIOpRec = getInitValueAsRegClass(Dst->getChild(1)->getLeafValue());
if (DstIOpRec == nullptr)
return failedImport(
"COPY_TO_REGCLASS operand #1 isn't a register class");
} else if (DstI.TheDef->getName() == "EXTRACT_SUBREG") {
if (!Dst->getChild(0)->isLeaf())
return failedImport("EXTRACT_SUBREG operand #0 isn't a leaf");
// We can assume that a subregister is in the same bank as it's super
// register.
DstIOpRec = getInitValueAsRegClass(Dst->getChild(0)->getLeafValue());
if (DstIOpRec == nullptr)
return failedImport(
"EXTRACT_SUBREG operand #0 isn't a register class");
} else if (DstIOpRec->isSubClassOf("RegisterOperand"))
DstIOpRec = DstIOpRec->getValueAsDef("RegClass");
else if (!DstIOpRec->isSubClassOf("RegisterClass"))
return failedImport("Dst MI def isn't a register class" +
to_string(*Dst));
OperandMatcher &OM = InsnMatcher.getOperand(OpIdx);
OM.setSymbolicName(DstIOperand.Name);
M.defineOperand(OM.getSymbolicName(), OM);
OM.addPredicate<RegisterBankOperandMatcher>(
Target.getRegisterClass(DstIOpRec));
++OpIdx;
}
auto DstMIBuilderOrError = createAndImportInstructionRenderer(M, Dst);
if (auto Error = DstMIBuilderOrError.takeError())
return std::move(Error);
BuildMIAction &DstMIBuilder = DstMIBuilderOrError.get();
// Render the implicit defs.
// These are only added to the root of the result.
if (auto Error = importImplicitDefRenderers(DstMIBuilder, P.getDstRegs()))
return std::move(Error);
DstMIBuilder.chooseInsnToMutate(M);
// Constrain the registers to classes. This is normally derived from the
// emitted instruction but a few instructions require special handling.
if (DstI.TheDef->getName() == "COPY_TO_REGCLASS") {
// COPY_TO_REGCLASS does not provide operand constraints itself but the
// result is constrained to the class given by the second child.
Record *DstIOpRec =
getInitValueAsRegClass(Dst->getChild(1)->getLeafValue());
if (DstIOpRec == nullptr)
return failedImport("COPY_TO_REGCLASS operand #1 isn't a register class");
M.addAction<ConstrainOperandToRegClassAction>(
0, 0, Target.getRegisterClass(DstIOpRec));
// We're done with this pattern! It's eligible for GISel emission; return
// it.
++NumPatternImported;
return std::move(M);
}
if (DstI.TheDef->getName() == "EXTRACT_SUBREG") {
// EXTRACT_SUBREG selects into a subregister COPY but unlike most
// instructions, the result register class is controlled by the
// subregisters of the operand. As a result, we must constrain the result
// class rather than check that it's already the right one.
if (!Dst->getChild(0)->isLeaf())
return failedImport("EXTRACT_SUBREG child #1 is not a leaf");
DefInit *SubRegInit = dyn_cast<DefInit>(Dst->getChild(1)->getLeafValue());
if (!SubRegInit)
return failedImport("EXTRACT_SUBREG child #1 is not a subreg index");
// Constrain the result to the same register bank as the operand.
Record *DstIOpRec =
getInitValueAsRegClass(Dst->getChild(0)->getLeafValue());
if (DstIOpRec == nullptr)
return failedImport("EXTRACT_SUBREG operand #1 isn't a register class");
CodeGenSubRegIndex *SubIdx = CGRegs.getSubRegIdx(SubRegInit->getDef());
CodeGenRegisterClass *SrcRC = CGRegs.getRegClass(DstIOpRec);
// It would be nice to leave this constraint implicit but we're required
// to pick a register class so constrain the result to a register class
// that can hold the correct MVT.
//
// FIXME: This may introduce an extra copy if the chosen class doesn't
// actually contain the subregisters.
assert(Src->getExtTypes().size() == 1 &&
"Expected Src of EXTRACT_SUBREG to have one result type");
const auto &SrcRCDstRCPair =
SrcRC->getMatchingSubClassWithSubRegs(CGRegs, SubIdx);
assert(SrcRCDstRCPair->second && "Couldn't find a matching subclass");
M.addAction<ConstrainOperandToRegClassAction>(0, 0, *SrcRCDstRCPair->second);
M.addAction<ConstrainOperandToRegClassAction>(0, 1, *SrcRCDstRCPair->first);
// We're done with this pattern! It's eligible for GISel emission; return
// it.
++NumPatternImported;
return std::move(M);
}
M.addAction<ConstrainOperandsToDefinitionAction>(0);
// We're done with this pattern! It's eligible for GISel emission; return it.
++NumPatternImported;
return std::move(M);
}
// Emit imm predicate table and an enum to reference them with.
// The 'Predicate_' part of the name is redundant but eliminating it is more
// trouble than it's worth.
void GlobalISelEmitter::emitImmPredicates(
raw_ostream &OS, StringRef TypeIdentifier, StringRef Type,
std::function<bool(const Record *R)> Filter) {
std::vector<const Record *> MatchedRecords;
const auto &Defs = RK.getAllDerivedDefinitions("PatFrag");
std::copy_if(Defs.begin(), Defs.end(), std::back_inserter(MatchedRecords),
[&](Record *Record) {
return !Record->getValueAsString("ImmediateCode").empty() &&
Filter(Record);
});
if (!MatchedRecords.empty()) {
OS << "// PatFrag predicates.\n"
<< "enum {\n";
std::string EnumeratorSeparator =
(" = GIPFP_" + TypeIdentifier + "_Invalid + 1,\n").str();
for (const auto *Record : MatchedRecords) {
OS << " GIPFP_" << TypeIdentifier << "_Predicate_" << Record->getName()
<< EnumeratorSeparator;
EnumeratorSeparator = ",\n";
}
OS << "};\n";
}
OS << "bool " << Target.getName() << "InstructionSelector::testImmPredicate_"
<< TypeIdentifier << "(unsigned PredicateID, " << Type
<< " Imm) const {\n";
if (!MatchedRecords.empty())
OS << " switch (PredicateID) {\n";
for (const auto *Record : MatchedRecords) {
OS << " case GIPFP_" << TypeIdentifier << "_Predicate_"
<< Record->getName() << ": {\n"
<< " " << Record->getValueAsString("ImmediateCode") << "\n"
<< " llvm_unreachable(\"ImmediateCode should have returned\");\n"
<< " return false;\n"
<< " }\n";
}
if (!MatchedRecords.empty())
OS << " }\n";
OS << " llvm_unreachable(\"Unknown predicate\");\n"
<< " return false;\n"
<< "}\n";
}
template <class GroupT>
std::vector<Matcher *> GlobalISelEmitter::optimizeRules(
ArrayRef<Matcher *> Rules,
std::vector<std::unique_ptr<Matcher>> &MatcherStorage) {
std::vector<Matcher *> OptRules;
std::unique_ptr<GroupT> CurrentGroup = make_unique<GroupT>();
assert(CurrentGroup->empty() && "Newly created group isn't empty!");
unsigned NumGroups = 0;
auto ProcessCurrentGroup = [&]() {
if (CurrentGroup->empty())
// An empty group is good to be reused:
return;
// If the group isn't large enough to provide any benefit, move all the
// added rules out of it and make sure to re-create the group to properly
// re-initialize it:
if (CurrentGroup->size() < 2)
for (Matcher *M : CurrentGroup->matchers())
OptRules.push_back(M);
else {
CurrentGroup->finalize();
OptRules.push_back(CurrentGroup.get());
MatcherStorage.emplace_back(std::move(CurrentGroup));
++NumGroups;
}
CurrentGroup = make_unique<GroupT>();
};
for (Matcher *Rule : Rules) {
// Greedily add as many matchers as possible to the current group:
if (CurrentGroup->addMatcher(*Rule))
continue;
ProcessCurrentGroup();
assert(CurrentGroup->empty() && "A group wasn't properly re-initialized");
// Try to add the pending matcher to a newly created empty group:
if (!CurrentGroup->addMatcher(*Rule))
// If we couldn't add the matcher to an empty group, that group type
// doesn't support that kind of matchers at all, so just skip it:
OptRules.push_back(Rule);
}
ProcessCurrentGroup();
DEBUG(dbgs() << "NumGroups: " << NumGroups << "\n");
assert(CurrentGroup->empty() && "The last group wasn't properly processed");
return OptRules;
}
MatchTable
GlobalISelEmitter::buildMatchTable(MutableArrayRef<RuleMatcher> Rules,
bool Optimize, bool WithCoverage) {
std::vector<Matcher *> InputRules;
for (Matcher &Rule : Rules)
InputRules.push_back(&Rule);
if (!Optimize)
return MatchTable::buildTable(InputRules, WithCoverage);
+ unsigned CurrentOrdering = 0;
+ StringMap<unsigned> OpcodeOrder;
+ for (RuleMatcher &Rule : Rules) {
+ const StringRef Opcode = Rule.getOpcode();
+ assert(!Opcode.empty() && "Didn't expect an undefined opcode");
+ if (OpcodeOrder.count(Opcode) == 0)
+ OpcodeOrder[Opcode] = CurrentOrdering++;
+ }
+
+ std::stable_sort(InputRules.begin(), InputRules.end(),
+ [&OpcodeOrder](const Matcher *A, const Matcher *B) {
+ auto *L = static_cast<const RuleMatcher *>(A);
+ auto *R = static_cast<const RuleMatcher *>(B);
+ return std::make_tuple(OpcodeOrder[L->getOpcode()],
+ L->getNumOperands()) <
+ std::make_tuple(OpcodeOrder[R->getOpcode()],
+ R->getNumOperands());
+ });
+
for (Matcher *Rule : InputRules)
Rule->optimize();
std::vector<std::unique_ptr<Matcher>> MatcherStorage;
std::vector<Matcher *> OptRules =
optimizeRules<GroupMatcher>(InputRules, MatcherStorage);
for (Matcher *Rule : OptRules)
Rule->optimize();
return MatchTable::buildTable(OptRules, WithCoverage);
}
void GlobalISelEmitter::run(raw_ostream &OS) {
if (!UseCoverageFile.empty()) {
RuleCoverage = CodeGenCoverage();
auto RuleCoverageBufOrErr = MemoryBuffer::getFile(UseCoverageFile);
if (!RuleCoverageBufOrErr) {
PrintWarning(SMLoc(), "Missing rule coverage data");
RuleCoverage = None;
} else {
if (!RuleCoverage->parse(*RuleCoverageBufOrErr.get(), Target.getName())) {
PrintWarning(SMLoc(), "Ignoring invalid or missing rule coverage data");
RuleCoverage = None;
}
}
}
// Track the run-time opcode values
gatherOpcodeValues();
// Track the run-time LLT ID values
gatherTypeIDValues();
// Track the GINodeEquiv definitions.
gatherNodeEquivs();
emitSourceFileHeader(("Global Instruction Selector for the " +
Target.getName() + " target").str(), OS);
std::vector<RuleMatcher> Rules;
// Look through the SelectionDAG patterns we found, possibly emitting some.
for (const PatternToMatch &Pat : CGP.ptms()) {
++NumPatternTotal;
auto MatcherOrErr = runOnPattern(Pat);
// The pattern analysis can fail, indicating an unsupported pattern.
// Report that if we've been asked to do so.
if (auto Err = MatcherOrErr.takeError()) {
if (WarnOnSkippedPatterns) {
PrintWarning(Pat.getSrcRecord()->getLoc(),
"Skipped pattern: " + toString(std::move(Err)));
} else {
consumeError(std::move(Err));
}
++NumPatternImportsSkipped;
continue;
}
if (RuleCoverage) {
if (RuleCoverage->isCovered(MatcherOrErr->getRuleID()))
++NumPatternsTested;
else
PrintWarning(Pat.getSrcRecord()->getLoc(),
"Pattern is not covered by a test");
}
Rules.push_back(std::move(MatcherOrErr.get()));
}
// Comparison function to order records by name.
auto orderByName = [](const Record *A, const Record *B) {
return A->getName() < B->getName();
};
std::vector<Record *> ComplexPredicates =
RK.getAllDerivedDefinitions("GIComplexOperandMatcher");
llvm::sort(ComplexPredicates.begin(), ComplexPredicates.end(), orderByName);
std::vector<Record *> CustomRendererFns =
RK.getAllDerivedDefinitions("GICustomOperandRenderer");
llvm::sort(CustomRendererFns.begin(), CustomRendererFns.end(), orderByName);
unsigned MaxTemporaries = 0;
for (const auto &Rule : Rules)
MaxTemporaries = std::max(MaxTemporaries, Rule.countRendererFns());
OS << "#ifdef GET_GLOBALISEL_PREDICATE_BITSET\n"
<< "const unsigned MAX_SUBTARGET_PREDICATES = " << SubtargetFeatures.size()
<< ";\n"
<< "using PredicateBitset = "
"llvm::PredicateBitsetImpl<MAX_SUBTARGET_PREDICATES>;\n"
<< "#endif // ifdef GET_GLOBALISEL_PREDICATE_BITSET\n\n";
OS << "#ifdef GET_GLOBALISEL_TEMPORARIES_DECL\n"
<< " mutable MatcherState State;\n"
<< " typedef "
"ComplexRendererFns("
<< Target.getName()
<< "InstructionSelector::*ComplexMatcherMemFn)(MachineOperand &) const;\n"
<< " typedef void(" << Target.getName()
<< "InstructionSelector::*CustomRendererFn)(MachineInstrBuilder &, const "
"MachineInstr&) "
"const;\n"
<< " const ISelInfoTy<PredicateBitset, ComplexMatcherMemFn, "
"CustomRendererFn> "
"ISelInfo;\n";
OS << " static " << Target.getName()
<< "InstructionSelector::ComplexMatcherMemFn ComplexPredicateFns[];\n"
<< " static " << Target.getName()
<< "InstructionSelector::CustomRendererFn CustomRenderers[];\n"
<< " bool testImmPredicate_I64(unsigned PredicateID, int64_t Imm) const "
"override;\n"
<< " bool testImmPredicate_APInt(unsigned PredicateID, const APInt &Imm) "
"const override;\n"
<< " bool testImmPredicate_APFloat(unsigned PredicateID, const APFloat "
"&Imm) const override;\n"
<< " const int64_t *getMatchTable() const override;\n"
<< "#endif // ifdef GET_GLOBALISEL_TEMPORARIES_DECL\n\n";
OS << "#ifdef GET_GLOBALISEL_TEMPORARIES_INIT\n"
<< ", State(" << MaxTemporaries << "),\n"
<< "ISelInfo(TypeObjects, NumTypeObjects, FeatureBitsets"
<< ", ComplexPredicateFns, CustomRenderers)\n"
<< "#endif // ifdef GET_GLOBALISEL_TEMPORARIES_INIT\n\n";
OS << "#ifdef GET_GLOBALISEL_IMPL\n";
SubtargetFeatureInfo::emitSubtargetFeatureBitEnumeration(SubtargetFeatures,
OS);
// Separate subtarget features by how often they must be recomputed.
SubtargetFeatureInfoMap ModuleFeatures;
std::copy_if(SubtargetFeatures.begin(), SubtargetFeatures.end(),
std::inserter(ModuleFeatures, ModuleFeatures.end()),
[](const SubtargetFeatureInfoMap::value_type &X) {
return !X.second.mustRecomputePerFunction();
});
SubtargetFeatureInfoMap FunctionFeatures;
std::copy_if(SubtargetFeatures.begin(), SubtargetFeatures.end(),
std::inserter(FunctionFeatures, FunctionFeatures.end()),
[](const SubtargetFeatureInfoMap::value_type &X) {
return X.second.mustRecomputePerFunction();
});
SubtargetFeatureInfo::emitComputeAvailableFeatures(
Target.getName(), "InstructionSelector", "computeAvailableModuleFeatures",
ModuleFeatures, OS);
SubtargetFeatureInfo::emitComputeAvailableFeatures(
Target.getName(), "InstructionSelector",
"computeAvailableFunctionFeatures", FunctionFeatures, OS,
"const MachineFunction *MF");
// Emit a table containing the LLT objects needed by the matcher and an enum
// for the matcher to reference them with.
std::vector<LLTCodeGen> TypeObjects;
for (const auto &Ty : KnownTypes)
TypeObjects.push_back(Ty);
llvm::sort(TypeObjects.begin(), TypeObjects.end());
OS << "// LLT Objects.\n"
<< "enum {\n";
for (const auto &TypeObject : TypeObjects) {
OS << " ";
TypeObject.emitCxxEnumValue(OS);
OS << ",\n";
}
OS << "};\n";
OS << "const static size_t NumTypeObjects = " << TypeObjects.size() << ";\n"
<< "const static LLT TypeObjects[] = {\n";
for (const auto &TypeObject : TypeObjects) {
OS << " ";
TypeObject.emitCxxConstructorCall(OS);
OS << ",\n";
}
OS << "};\n\n";
// Emit a table containing the PredicateBitsets objects needed by the matcher
// and an enum for the matcher to reference them with.
std::vector<std::vector<Record *>> FeatureBitsets;
for (auto &Rule : Rules)
FeatureBitsets.push_back(Rule.getRequiredFeatures());
llvm::sort(
FeatureBitsets.begin(), FeatureBitsets.end(),
[&](const std::vector<Record *> &A, const std::vector<Record *> &B) {
if (A.size() < B.size())
return true;
if (A.size() > B.size())
return false;
for (const auto &Pair : zip(A, B)) {
if (std::get<0>(Pair)->getName() < std::get<1>(Pair)->getName())
return true;
if (std::get<0>(Pair)->getName() > std::get<1>(Pair)->getName())
return false;
}
return false;
});
FeatureBitsets.erase(
std::unique(FeatureBitsets.begin(), FeatureBitsets.end()),
FeatureBitsets.end());
OS << "// Feature bitsets.\n"
<< "enum {\n"
<< " GIFBS_Invalid,\n";
for (const auto &FeatureBitset : FeatureBitsets) {
if (FeatureBitset.empty())
continue;
OS << " " << getNameForFeatureBitset(FeatureBitset) << ",\n";
}
OS << "};\n"
<< "const static PredicateBitset FeatureBitsets[] {\n"
<< " {}, // GIFBS_Invalid\n";
for (const auto &FeatureBitset : FeatureBitsets) {
if (FeatureBitset.empty())
continue;
OS << " {";
for (const auto &Feature : FeatureBitset) {
const auto &I = SubtargetFeatures.find(Feature);
assert(I != SubtargetFeatures.end() && "Didn't import predicate?");
OS << I->second.getEnumBitName() << ", ";
}
OS << "},\n";
}
OS << "};\n\n";
// Emit complex predicate table and an enum to reference them with.
OS << "// ComplexPattern predicates.\n"
<< "enum {\n"
<< " GICP_Invalid,\n";
for (const auto &Record : ComplexPredicates)
OS << " GICP_" << Record->getName() << ",\n";
OS << "};\n"
<< "// See constructor for table contents\n\n";
emitImmPredicates(OS, "I64", "int64_t", [](const Record *R) {
bool Unset;
return !R->getValueAsBitOrUnset("IsAPFloat", Unset) &&
!R->getValueAsBit("IsAPInt");
});
emitImmPredicates(OS, "APFloat", "const APFloat &", [](const Record *R) {
bool Unset;
return R->getValueAsBitOrUnset("IsAPFloat", Unset);
});
emitImmPredicates(OS, "APInt", "const APInt &", [](const Record *R) {
return R->getValueAsBit("IsAPInt");
});
OS << "\n";
OS << Target.getName() << "InstructionSelector::ComplexMatcherMemFn\n"
<< Target.getName() << "InstructionSelector::ComplexPredicateFns[] = {\n"
<< " nullptr, // GICP_Invalid\n";
for (const auto &Record : ComplexPredicates)
OS << " &" << Target.getName()
<< "InstructionSelector::" << Record->getValueAsString("MatcherFn")
<< ", // " << Record->getName() << "\n";
OS << "};\n\n";
OS << "// Custom renderers.\n"
<< "enum {\n"
<< " GICR_Invalid,\n";
for (const auto &Record : CustomRendererFns)
OS << " GICR_" << Record->getValueAsString("RendererFn") << ", \n";
OS << "};\n";
OS << Target.getName() << "InstructionSelector::CustomRendererFn\n"
<< Target.getName() << "InstructionSelector::CustomRenderers[] = {\n"
<< " nullptr, // GICP_Invalid\n";
for (const auto &Record : CustomRendererFns)
OS << " &" << Target.getName()
<< "InstructionSelector::" << Record->getValueAsString("RendererFn")
<< ", // " << Record->getName() << "\n";
OS << "};\n\n";
std::stable_sort(Rules.begin(), Rules.end(), [&](const RuleMatcher &A,
const RuleMatcher &B) {
int ScoreA = RuleMatcherScores[A.getRuleID()];
int ScoreB = RuleMatcherScores[B.getRuleID()];
if (ScoreA > ScoreB)
return true;
if (ScoreB > ScoreA)
return false;
if (A.isHigherPriorityThan(B)) {
assert(!B.isHigherPriorityThan(A) && "Cannot be more important "
"and less important at "
"the same time");
return true;
}
return false;
});
OS << "bool " << Target.getName()
<< "InstructionSelector::selectImpl(MachineInstr &I, CodeGenCoverage "
"&CoverageInfo) const {\n"
<< " MachineFunction &MF = *I.getParent()->getParent();\n"
<< " MachineRegisterInfo &MRI = MF.getRegInfo();\n"
<< " // FIXME: This should be computed on a per-function basis rather "
"than per-insn.\n"
<< " AvailableFunctionFeatures = computeAvailableFunctionFeatures(&STI, "
"&MF);\n"
<< " const PredicateBitset AvailableFeatures = getAvailableFeatures();\n"
<< " NewMIVector OutMIs;\n"
<< " State.MIs.clear();\n"
<< " State.MIs.push_back(&I);\n\n"
<< " if (executeMatchTable(*this, OutMIs, State, ISelInfo"
<< ", getMatchTable(), TII, MRI, TRI, RBI, AvailableFeatures"
<< ", CoverageInfo)) {\n"
<< " return true;\n"
<< " }\n\n"
<< " return false;\n"
<< "}\n\n";
const MatchTable Table =
buildMatchTable(Rules, OptimizeMatchTable, GenerateCoverage);
OS << "const int64_t *" << Target.getName()
<< "InstructionSelector::getMatchTable() const {\n";
Table.emitDeclaration(OS);
OS << " return ";
Table.emitUse(OS);
OS << ";\n}\n";
OS << "#endif // ifdef GET_GLOBALISEL_IMPL\n";
OS << "#ifdef GET_GLOBALISEL_PREDICATES_DECL\n"
<< "PredicateBitset AvailableModuleFeatures;\n"
<< "mutable PredicateBitset AvailableFunctionFeatures;\n"
<< "PredicateBitset getAvailableFeatures() const {\n"
<< " return AvailableModuleFeatures | AvailableFunctionFeatures;\n"
<< "}\n"
<< "PredicateBitset\n"
<< "computeAvailableModuleFeatures(const " << Target.getName()
<< "Subtarget *Subtarget) const;\n"
<< "PredicateBitset\n"
<< "computeAvailableFunctionFeatures(const " << Target.getName()
<< "Subtarget *Subtarget,\n"
<< " const MachineFunction *MF) const;\n"
<< "#endif // ifdef GET_GLOBALISEL_PREDICATES_DECL\n";
OS << "#ifdef GET_GLOBALISEL_PREDICATES_INIT\n"
<< "AvailableModuleFeatures(computeAvailableModuleFeatures(&STI)),\n"
<< "AvailableFunctionFeatures()\n"
<< "#endif // ifdef GET_GLOBALISEL_PREDICATES_INIT\n";
}
void GlobalISelEmitter::declareSubtargetFeature(Record *Predicate) {
if (SubtargetFeatures.count(Predicate) == 0)
SubtargetFeatures.emplace(
Predicate, SubtargetFeatureInfo(Predicate, SubtargetFeatures.size()));
}
void RuleMatcher::optimize() {
for (auto &Item : InsnVariableIDs) {
InstructionMatcher &InsnMatcher = *Item.first;
for (auto &OM : InsnMatcher.operands()) {
// Register Banks checks rarely fail, but often crash as targets usually
// provide only partially defined RegisterBankInfo::getRegBankFromRegClass
// method. Often the problem is hidden as non-optimized MatchTable checks
// banks rather late, most notably after checking target / function /
// module features and a few opcodes. That makes these checks a)
// beneficial to delay until the very end (we don't want to perform a lot
// of checks that all pass and then fail at the very end) b) not safe to
// have as early checks.
for (auto &OP : OM->predicates())
if (isa<RegisterBankOperandMatcher>(OP) ||
isa<ComplexPatternOperandMatcher>(OP))
EpilogueMatchers.emplace_back(std::move(OP));
OM->eraseNullPredicates();
}
InsnMatcher.optimize();
}
llvm::sort(
EpilogueMatchers.begin(), EpilogueMatchers.end(),
[](const std::unique_ptr<PredicateMatcher> &L,
const std::unique_ptr<PredicateMatcher> &R) {
return std::make_tuple(L->getKind(), L->getInsnVarID(), L->getOpIdx()) <
std::make_tuple(R->getKind(), R->getInsnVarID(), R->getOpIdx());
});
}
bool RuleMatcher::hasFirstCondition() const {
if (insnmatchers_empty())
return false;
InstructionMatcher &Matcher = insnmatchers_front();
if (!Matcher.predicates_empty())
return true;
for (auto &OM : Matcher.operands())
for (auto &OP : OM->predicates())
if (!isa<InstructionOperandMatcher>(OP))
return true;
return false;
}
const PredicateMatcher &RuleMatcher::getFirstCondition() const {
assert(!insnmatchers_empty() &&
"Trying to get a condition from an empty RuleMatcher");
InstructionMatcher &Matcher = insnmatchers_front();
if (!Matcher.predicates_empty())
return **Matcher.predicates_begin();
// If there is no more predicate on the instruction itself, look at its
// operands.
for (auto &OM : Matcher.operands())
for (auto &OP : OM->predicates())
if (!isa<InstructionOperandMatcher>(OP))
return *OP;
llvm_unreachable("Trying to get a condition from an InstructionMatcher with "
"no conditions");
}
std::unique_ptr<PredicateMatcher> RuleMatcher::popFirstCondition() {
assert(!insnmatchers_empty() &&
"Trying to pop a condition from an empty RuleMatcher");
InstructionMatcher &Matcher = insnmatchers_front();
if (!Matcher.predicates_empty())
return Matcher.predicates_pop_front();
// If there is no more predicate on the instruction itself, look at its
// operands.
for (auto &OM : Matcher.operands())
for (auto &OP : OM->predicates())
if (!isa<InstructionOperandMatcher>(OP)) {
std::unique_ptr<PredicateMatcher> Result = std::move(OP);
OM->eraseNullPredicates();
return Result;
}
llvm_unreachable("Trying to pop a condition from an InstructionMatcher with "
"no conditions");
}
bool GroupMatcher::candidateConditionMatches(
const PredicateMatcher &Predicate) const {
if (empty()) {
// Sharing predicates for nested instructions is not supported yet as we
// currently don't hoist the GIM_RecordInsn's properly, therefore we can
// only work on the original root instruction (InsnVarID == 0):
if (Predicate.getInsnVarID() != 0)
return false;
// ... otherwise an empty group can handle any predicate with no specific
// requirements:
return true;
}
const Matcher &Representative = **Matchers.begin();
const auto &RepresentativeCondition = Representative.getFirstCondition();
// ... if not empty, the group can only accomodate matchers with the exact
// same first condition:
return Predicate.isIdentical(RepresentativeCondition);
}
bool GroupMatcher::addMatcher(Matcher &Candidate) {
if (!Candidate.hasFirstCondition())
return false;
const PredicateMatcher &Predicate = Candidate.getFirstCondition();
if (!candidateConditionMatches(Predicate))
return false;
Matchers.push_back(&Candidate);
return true;
}
void GroupMatcher::finalize() {
assert(Conditions.empty() && "Already finalized?");
if (empty())
return;
Matcher &FirstRule = **Matchers.begin();
Conditions.push_back(FirstRule.popFirstCondition());
for (unsigned I = 1, E = Matchers.size(); I < E; ++I)
Matchers[I]->popFirstCondition();
}
void GroupMatcher::emit(MatchTable &Table) {
unsigned LabelID = ~0U;
if (!Conditions.empty()) {
LabelID = Table.allocateLabelID();
Table << MatchTable::Opcode("GIM_Try", +1)
<< MatchTable::Comment("On fail goto")
<< MatchTable::JumpTarget(LabelID) << MatchTable::LineBreak;
}
for (auto &Condition : Conditions)
Condition->emitPredicateOpcodes(
Table, *static_cast<RuleMatcher *>(*Matchers.begin()));
for (const auto &M : Matchers)
M->emit(Table);
// Exit the group
if (!Conditions.empty())
Table << MatchTable::Opcode("GIM_Reject", -1) << MatchTable::LineBreak
<< MatchTable::Label(LabelID);
}
unsigned OperandMatcher::getInsnVarID() const { return Insn.getInsnVarID(); }
} // end anonymous namespace
//===----------------------------------------------------------------------===//
namespace llvm {
void EmitGlobalISel(RecordKeeper &RK, raw_ostream &OS) {
GlobalISelEmitter(RK).run(OS);
}
} // End llvm namespace
llvm-svn: 332999
|
||
Dan Liew
|
d7e3a3632a |
[lit] Try to make shtest-timeout.py test more reliable by using a
larger timeout value. This really isn't very good because it will still be susceptible to machine performance. While we are here also fix a bug in validation of `maxIndividualTestTime` where previously it wasn't checked if the type was an int. rdar://problem/40221572 llvm-svn: 332987 |
||
|
Dan Liew
|
bae1119a38 |
[lit] Don't run slow.py in shtest-timeout.py test.
The program used to be used in `quick_then_slow.py` but that was removed in r328702. The tests always run `slow.py` on its own but this doesn't really test additional code so we'll just drop running `slow.py` so the tests run faster. rdar://problem/40221572 llvm-svn: 332986 |
||
|
Dan Liew
|
b2006c8dcb |
[lit] Don't check output of commands used in shtest-timeout.py test.
If the system is under heavy load 1 second might not be long enough for it to produce output which could lead to spurious test failures. What matters is that the right test cases reach a timeout. rdar://problem/40221572 llvm-svn: 332985 |
||
Simon Dardis
|
b7fb5f7be7 |
[FastISel] Permit instructions to be skipped for FastISel generation.
Some ISA's such as microMIPS32(R6) have instructions which are near identical for code generation purposes, e.g. xor and xor16. These instructions take the same value types for operands and return values, have the same instruction predicates and map to the same ISD opcode. (These instructions do differ by register classes.) In such cases, the FastISel generator rejects the instruction definition. This patch borrows the 'FastIselShouldIgnore' bit from rL129692 and enables applying it to an instruction definition. Reviewers: mcrosier Differential Revision: https://reviews.llvm.org/D46953 llvm-svn: 332983 |
||
|
Roman Tereshin
|
fa30317850 |
[GlobalISel][InstructionSelect] Removing redundant num operands and nested def operands checks, perf patch 2
This patch continues a series of patches that decrease time spent by GlobalISel in its InstructionSelect pass by roughly 60% for -O0 builds for large inputs as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64. This commit specifically removes number of operands checks that are redundant if the instruction's opcode already guarantees that number of operands (or more), and also avoids any kind of checks on a def operand of a nested instruction as everything about it was already checked at its use. The expected performance implication is about 3% off InstructionSelect comparing to the baseline (before the series of patches) This patch also contains a bit of NFC changes required for further patches in the series. Every commit planned shares the same Phabricator Review. Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 332945 |
||
|
Roman Tereshin
|
dfebbe6a7d |
Reapply r332907 "[GlobalISel] Improving InstructionSelect's performance by reducing MatchTable..."
Apparently the compile time problem was caused by the fact that not all compilers / STL implementations can automatically convert std::unique_ptr<Derived> to std::unique_ptr<Base>. Fixed (hopefully) by making sure it's std::unique_ptr<Derived>&& (rvalue ref) to std::unique_ptr<Base> conversion instead. llvm-svn: 332917 |
||
|
Roman Tereshin
|
2f2e0150a3 |
Revert r332907 "[GlobalISel] Improving InstructionSelect's performance by reducing MatchTable..."
There is a compile time error I didn't see locally, investigating now. llvm-svn: 332912 |
||
|
Roman Tereshin
|
0f16aa465d |
[GlobalISel] Improving InstructionSelect's performance by reducing MatchTable, mostly NFC, perf patch 1
This patch starts a series of patches that decrease time spent by GlobalISel in its InstructionSelect pass by roughly 60% for -O0 builds for large inputs as measured on sqlite3-amalgamation (http://sqlite.org/download.html) targeting AArch64. The performance improvements are achieved solely by reducing the number of matching GIM_* opcodes executed by the MatchTable's interpreter during the selection by approx. a factor of 30, which also brings contribution of this particular part of the selection process to the overall runtime of InstructionSelect pass down from approx. 60-70% to 5-7%, thus making further improvements in this particular direction not very profitable. The improvements described above are expected for any target that doesn't have many complex patterns. The targets that do should strictly benefit from the changes, but by how much exactly is hard to estimate beforehand. It's also likely that such target WILL benefit from further improvements to MatchTable, most likely the ones that bring it closer to a perfect decision tree. This commit specifically is rather large mostly NFC commit that does necessary preparation work and refactoring, there will be a following series of small patches introducing a specific optimization each shortly after. This commit specifically is expected to cause a small compile time regression (around 2.5% of InstructionSelect pass time), which should be fixed by the next commit of the series. Every commit planned shares the same Phabricator Review. Reviewers: qcolombet, dsanders, bogner, aemerson, javed.absar Reviewed By: qcolombet Subscribers: rovka, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D44700 llvm-svn: 332907 |
||
|
Kristof Beyls
|
9eddbad56a |
Add Script to match open Phabricator reviews with potential reviewers.
At the last EuroLLVM, I gave a lightning talk about code review statistics on Phabricator reviews and what we could derive from that to try and reduce waiting-for-review bottlenecks. (see https://llvm.org/devmtg/2018-04/talks.html#Lightning_2). One of the items I pointed to is a script we've been using internally for a little while to try and match open Phabricator reviews to people who might be able to review them well. I received quite a few requests to share that script, so here it is. Warning: this is prototype quality! The script uses 2 similar heuristics to try and match open reviews with potential reviewers: If there is overlap between the lines of code touched by the patch-under-review and lines of code that a person has written, that person may be a good reviewer. If there is overlap between the files touched by the patch-under-review and the source files that a person has made changes to, that person may be a good reviewer. The script provides a percentage for each of the above heuristics and emails a summary. For example, a summary I received a few weeks ago from the script is the following: SUMMARY FOR kristof.beyls@arm.com (found 8 reviews): [3.37%/41.67%] https://reviews.llvm.org/D46018 '[GlobalISel][IRTranslator] Split aggregates during IR translation' by Amara Emerson [0.00%/100.00%] https://reviews.llvm.org/D46111 '[ARM] Enable misched for R52.' by Dave Green [0.00%/50.00%] https://reviews.llvm.org/D45770 '[AArch64] Disable spill slot scavenging when stack realignment required.' by Paul Walker [0.00%/40.00%] https://reviews.llvm.org/D42759 '[CGP] Split large data structres to sink more GEPs' by Haicheng Wu [0.00%/25.00%] https://reviews.llvm.org/D45189 '[MachineOutliner][AArch64] Keep track of functions that use a red zone in AArch64MachineFunctionInfo and use that instead of checking for noredzone in the MachineOutliner' by Jessica Paquette [0.00%/25.00%] https://reviews.llvm.org/D46107 '[AArch64] Codegen for v8.2A dot product intrinsics' by Oliver Stannard [0.00%/12.50%] https://reviews.llvm.org/D45541 '[globalisel] Update GlobalISel emitter to match new representation of extending loads' by Daniel Sanders [0.00%/6.25%] https://reviews.llvm.org/D44386 '[x86] Introduce the pconfig/enclv instructions' by Gabor Buella The first percentage in square brackets is the percentage of lines in the patch-under-review that changes lines that I wrote. The second percentage is the percentage of files that I made at least some changes to out of all of the files touched by the patch-under-review. Both the script and the heuristics are far from perfect, but I've heard positive feedback from the few colleagues the script has been sending a summary to every day - hearing that this does help them to quickly find patches-under-review they can help to review. The script takes quite some time to run (I typically see it running for 2 to 3 hours on weekdays when it gets started by a cron job early in the morning). There are 2 reasons why it takes a long time: The REST api into Phabricator isn't very efficient, i.e. a lot of uninteresting data needs to be fetched. The script tries to reduce this overhead partly by caching info it has fetched on previous runs, so as to not have to refetch lots of Phabricator state on each run. The script uses git blame to find for each line of code in the patch who wrote the original line of code being altered. git blame is sloooowww.... Anyway - to run this script: First install a virtualenv as follows (using Python2.7 - Python3 is almost certainly not going to work at the moment): $ virtualenv venv $ . ./venv/bin/activate $ pip install Phabricator Then to run the script, looking for open reviews that could be done by X.Y@company.com, run (in the venv): $ python ./find_interesting_reviews.py X.Y@company.com Please note that "X.Y@company.com" needs to be the exact email address (capitalization is important) that the git LLVM repository knows the person as. Multiple email addresses can be specified on the command line. Note that the script as is will email the results to all email addresses specified on the command line - so be careful not to spam people accidentally! Differential Revision: https://reviews.llvm.org/D46192 llvm-svn: 332711 |
||
Douglas Yung
|
d711b42b49 |
Mark test with "REQUIRES: shell" since it directly invokes "sh" and was failing on Windows.
llvm-svn: 332563 |
||
Alexander Richardson
|
7857762e4c |
Escape ]]> in xunit xml output
Summary: This sequence ends the CDATA block so any characters after that are no longer escaped. This can be fixed by replacing "]]>" with "]]]]><![CDATA[>". Reviewers: cmatthews Reviewed By: cmatthews Differential Revision: https://reviews.llvm.org/D46886 llvm-svn: 332440 |
||
|
Chris Matthews
|
e0665809b0 |
remove output xml incase it is leftover from another run
llvm-svn: 332424 |
||
|
Chris Matthews
|
30e6528953 |
Use not to catch unexpected pass as well as remove old test results
As per review feedback, make sure we rm temp files, and make the return code checking for lit more specific. llvm-svn: 332423 |
||
|
Nicola Zaghen
|
9667127c14 |
Rename DEBUG macro to LLVM_DEBUG.
The DEBUG() macro is very generic so it might clash with other projects. The renaming was done as follows: - git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g' - git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM - Manual change to APInt - Manually chage DOCS as regex doesn't match it. In the transition period the DEBUG() macro is still present and aliased to the LLVM_DEBUG() one. Differential Revision: https://reviews.llvm.org/D43624 llvm-svn: 332240 |
||
|
Chris Matthews
|
c01e6f9ecb |
Requirements can have & in them!
Lets escape those so the XML is valid! llvm-svn: 332161 |
||
|
Chris Matthews
|
5e42577a9e |
Add the message attribute to skipped
JUnit xml allows for a message attribute to be displayed on skips. Lets populate that with an analysis of why we skipped the test. llvm-svn: 332156 |
||
|
Chris Matthews
|
267c8ad156 |
Overhaul unicode handling in xunit output
I have seen a lot of errors where the xunit does not encode unicode test output correctly. Handle that explicitly now. llvm-svn: 332148 |
||
|
Chris Matthews
|
c499ebbb98 |
[LIT] replace output escapes wit a cdata block
CDATA blocks don't need to have XML stuff escaped. Makes sense to wrap output in them instead of escaping. llvm-svn: 332116 |
||
|
Chris Matthews
|
4826565393 |
Support Unsupported Tests in xunit output
We were reporting "Unsupported" tests in xunit as passes, however since they are not run, it make more sense to mark them as skipped. The Junit xml standard has support for that, so lets use it. llvm-svn: 332065 |
||
|
Chris Matthews
|
f5b7f9eb76 |
Refactor xunit test case builder to not use as much str addition
String concatenation in python is slow. Refactor to not concatenate the possibly large strings of test output and instead write them directly to the output file. llvm-svn: 332064 |
||
|
Chris Matthews
|
3dee71508e |
[LIT] Move xunit tests tests into their own location, and and add failures
Failures will increase coverage. llvm-svn: 332056 |
||
Sam Clegg
|
a01ce37a7f |
[WebAssembly] Initial Disassembler.
This implements a new table-gen emitter to create tables for a wasm disassembler, and a dissassembler to use them. Comes with 2 tests, that tests a few instructions manually. Is also able to disassemble large .wasm files with objdump reasonably. Not working so well, to be addressed in followups: - objdump appears to be passing an incorrect starting point. - since the disassembler works an instruction at a time, and it is disassembling stack instruction, it has no idea of pseudo register assignments. These registers are required for the instruction printing code that follows. For now, all such registers appear in the output as $0. Patch by Wouter van Oortmerssen Differential Revision: https://reviews.llvm.org/D45848 llvm-svn: 332052 |
||
|
Chris Matthews
|
c5f2366160 |
[LIT] Add the missing file
I forgot to commit this file. llvm-svn: 331946 |
||
|
Chris Matthews
|
0620816737 |
Refactor test incase results are backwards
Looks like results can come in either way in this file. Loosen the ordering constraints. llvm-svn: 331945 |
||
|
Chris Matthews
|
a7a8db71f5 |
[LIT] Handle xml characters in test names
Lit creates malformed xml when the test case has an & in the name. Escape those correctly. This also adds a test case which I will add other nasty encoding issues to in some followup commits. llvm-svn: 331942 |
||
Nico Weber
|
bbbb3edf87 |
Remove 'abi-breaking-checks' lit feature.
Its only two uses were removed in r311730. Effectively reverts r304851 (but that code has removed around a bit since then). https://reviews.llvm.org/D46619 clang side done in r331871. llvm-svn: 331872 |
||
|
Daniel Sanders
|
96623363e9 |
[globalisel] Update GlobalISel emitter to match new representation of extending loads
Summary: Previously, a extending load was represented at (G_*EXT (G_LOAD x)). This had a few drawbacks: * G_LOAD had to be legal for all sizes you could extend from, even if registers didn't naturally hold those sizes. * All sizes you could extend from had to be allocatable just in case the extend went missing (e.g. by optimization). * At minimum, G_*EXT and G_TRUNC had to be legal for these sizes. As we improve optimization of extends and truncates, this legality requirement would spread without considerable care w.r.t when certain combines were permitted. * The SelectionDAG importer required some ugly and fragile pattern rewriting to translate patterns into this style. This patch changes the representation to: * (G_[SZ]EXTLOAD x) * (G_LOAD x) any-extends when MMO.getSize() * 8 < ResultTy.getSizeInBits() which resolves these issues by allowing targets to work entirely in their native register sizes, and by having a more direct translation from SelectionDAG patterns. Each extending load can be lowered by the legalizer into separate extends and loads, however a target that supports s1 will need the any-extending load to extend to at least s8 since LLVM does not represent memory accesses smaller than 8 bit. The legalizer can widenScalar G_LOAD into an any-extending load but sign/zero-extending loads need help from something else like a combiner pass. A follow-up patch that adds combiner helpers for for this will follow. The new representation requires that the MMO correctly reflect the memory access so this has been corrected in a couple tests. I've also moved the extending loads to their own tests since they are (mostly) separate opcodes now. Additionally, the re-write appears to have invalidated two tests from select-with-no-legality-check.mir since the matcher table no longer contains loads that result in s1's and they aren't legal in AArch64 anymore. Depends on D45540 Reviewers: ab, aditya_nandakumar, bogner, rtereshin, volkan, rovka, javed.absar Reviewed By: rtereshin Subscribers: javed.absar, llvm-commits, kristof.beyls Differential Revision: https://reviews.llvm.org/D45541 llvm-svn: 331601 |
||
Saleem Abdulrasool
|
7dafb8329b |
lit: flesh out SubsituteCaptures further
Add overloads for `__len__` and `__getitem__` to allow use of this class on Linux as well as Windows. With these overloads, lit can be used on both hosts for the swift testsuite. llvm-svn: 331431 |
||
|
Roman Tereshin
|
8e669ed165 |
[GlobalISel][InstructionSelect] Making Coverage Info generation optional on per-match table basis
to make sure that Testgen always has access to coverage info even if the match table used by the selector itself is stripped off that information for performance reasons. Reviewers: dsanders, aemerson Reviewed By: dsanders Subscribers: rovka, kristof.beyls, llvm-commits, dsanders Differential Revision: https://reviews.llvm.org/D46098 llvm-svn: 331398 |
||
|
Roman Tereshin
|
f83ffb1ba4 |
[GlobalISel][InstructionSelect] Refactoring buildMatchTable out, NFC
to share it between the Instruction Selector in optimized and non-optimized modes both and the Testgen. Reviewers: dsanders, aemerson Reviewed By: dsanders Subscribers: rovka, kristof.beyls, llvm-commits, dsanders Differential Revision: https://reviews.llvm.org/D46097 llvm-svn: 331396 |
||
|
Roman Tereshin
|
58fdf359a9 |
[GlobalISel][InstructionSelect] Refactoring out a getMatchTable virtual method + other small NFC's
The main goal is to share getMatchTable between the Instruction Selector and the Testgen. The commit also contains some NFC only loosely related to refactoring out the getMatchTable, but strongly related to the initial Testgen patch (see https://reviews.llvm.org/D43962) Reviewers: dsanders, aemerson Reviewed By: dsanders Subscribers: rovka, kristof.beyls, llvm-commits, dsanders Differential Revision: https://reviews.llvm.org/D46096 llvm-svn: 331395 |
||
Adrian Prantl
|
0489ce9303 |
Remove @brief commands from doxygen comments, too.
This is a follow-up to r331272. We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done https://reviews.llvm.org/D46290 llvm-svn: 331275 |
||
|
Adrian Prantl
|
076a6683eb |
Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46290 llvm-svn: 331272 |
||
Gabor Buella
|
389cae1b12 |
[X86] movdiri and movdir64b instructions
Reviewers: spatel, craig.topper, RKSimon Reviewed By: craig.topper, RKSimon Differential Revision: https://reviews.llvm.org/D45983 llvm-svn: 331248 |
||
|
Craig Topper
|
4ce9451e0a |
[X86] Remove 'opaque ptr' from the intel syntax parser and printer.
Previously for instructions like fxsave we would print "opaque ptr" as part of the memory operand. Now we print nothing. We also no longer accept "opaque ptr" in the parser. We still accept any size to be specified for these instructions, but we may want to consider only parsing when no explicit size is specified. This what gas does. llvm-svn: 331243 |
||
|
Nico Weber
|
fcf0230e34 |
IWYU for llvm-config.h in llvm, additions.
See r331124 for how I made a list of files missing the include.
I then ran this Python script:
for f in open('filelist.txt'):
f = f.strip()
fl = open(f).readlines()
found = False
for i in xrange(len(fl)):
p = '#include "llvm/'
if not fl[i].startswith(p):
continue
if fl[i][len(p):] > 'Config':
fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
found = True
break
if not found:
print 'not found', f
else:
open(f, 'w').write(''.join(fl))
and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.
No intended behavior change.
llvm-svn: 331184
|
||
|
Gabor Buella
|
e2b2e2414f |
NFC - Typo fixes lib/VMCore -> lib/IR
llvm-svn: 331166 |
||
|
Andrea Di Biagio
|
46881e2def |
[Tablegen] SubtargetEmitter: move the logic that prints predicates for variant scheduling classes to helper functions. NFC
llvm-svn: 330968 |
||
|
Andrea Di Biagio
|
81668d5af5 |
[Tablegen] Simplify code in CodeGenSchedule. NFCI
llvm-svn: 330935 |
||
Sander de Smalen
|
d581504a2a |
[AsmMatcher] Extend PredicateMethod with optional DiagnosticPredicate
An optional, light-weight and backward-compatible mechanism to allow specifying that a diagnostic _only_ applies to a partial mismatch (NearMiss), rather than a full mismatch. Patch [1/2] in a series to improve assembler diagnostics for SVE. - Patch [1/2]: https://reviews.llvm.org/D45879 - Patch [2/2]: https://reviews.llvm.org/D45880 Reviewers: olista01, stoklund, craig.topper, mcrosier, rengolin, echristo, fhahn, SjoerdMeijer, evandro, javed.absar Reviewed By: olista01 Differential Revision: https://reviews.llvm.org/D45879 llvm-svn: 330930 |
||
Reid Kleckner
|
c5a26bc96b |
Revert r330755 "[lit] Report line number for failed RUN command"
It is causing many tests to fail on Windows buildbots: http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10211 llvm-svn: 330848 |
||
|
Greg Bedwell
|
650fa15cee |
[UpdateTestChecks] Change update_mca_test_checks.py file mode to match the other scripts
llvm-svn: 330815 |
||
|
Craig Topper
|
34f26c08d6 |
[TableGen] Fix bad indentation in tablegen output file.
llvm-svn: 330801 |
||
|
Joel E. Denny
|
f315d6c63e |
[lit] Report line number for failed RUN command
When debugging test failures with -vv (or -v in the case of the internal shell), this makes it easier to locate the RUN line that failed. For example, clang's test/Driver/linux-ld.c has 892 total RUN lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines after concatenation for line continuations. When reading the generated shell script, this also makes it easier to locate the RUN line that produced each command. To support reporting RUN line numbers in the case of the internal shell, this patch extends the internal shell to support the null command, ":", except pipelines are not supported. Reviewed By: asmith, delcypher Differential Revision: https://reviews.llvm.org/D44598 llvm-svn: 330755 |
||
|
Dan Liew
|
a348f9c134 |
[lit] Remove spurious - in invocation of lit in
`shtest-xunit-output.py` test. Although there is no `-` file Jeremy Morse has reported to me that it causes problems in their setup because lit tries to find it and ends up loading an out of tree lit configuration file. llvm-svn: 330728 |
||
|
Nico Weber
|
e478018779 |
Mostly revert r330672.
The test is apparently needed e.g. for check-cfi on Windows where we get 'C:/b/slave/sanitizer-windows/build/./bin/clang.exe': command not found without it. Try to fix the problem that was fixed by r330672 by also checking for isabs() instead. llvm-svn: 330673 |
||
|
Nico Weber
|
00c5136820 |
Remove code that's almost always dead, and harmful if not.
lit's util.which() would check if the passed-in path existed directly, and if so return it as-is. This is never the case when running llvm's, clang's, or lld's tests normally. But when running `./llvm-lit path/to/clang/test` with a cwd of llvm-build/bin, this if would detect that clang exists at path 'clang' and return 'clang' as the discovered clang binary -- and then lit would use the " clang " -> "*** Do not use 'clang' in tests, use '%clang'. ***" substitution to replace that with a broken test. By removing this early return, lit ends up with the usual absolute path and everything works even in this uncommon case. llvm-svn: 330672 |
||
|
Gabor Buella
|
9c90e6f6a7 |
[X86] Revert r330638 - accidental commit
llvm-svn: 330640 |
||
|
Gabor Buella
|
8b59c3cd73 |
[X86] movdiri and movdir64b instructions
Reviewers: craig.topper llvm-svn: 330638 |
||
|
Craig Topper
|
88b6ab0c0c |
[X86] Remove an unnecessary HANDLE_OPTIONAL line from the disassembler operand processing.
llvm-svn: 330534 |
||
|
Craig Topper
|
73dbe9122e |
[X86] Remove DATA32_PREFIX. Hack the printing for DATA16_PREFIX to print 'data32' in 16-bit mode. Hack the asm parser to convert 'data32' to 'data16' in 16-bit mode.
Improve the error messages to match GNU assembler. This also allows us to remove the hack from the disassembler table building. llvm-svn: 330531 |
||
|
Nico Weber
|
f8d747bd3f |
Remove llvm-build's --configure-target-def-file.
It was added 6.5 years ago in r144345, but was never hooked up and has been unused since. If _you_ do use this, feel free to revert, but add a comment on where it's used. https://reviews.llvm.org/D45262 llvm-svn: 330455 |
||
Sanjay Patel
|
f5e87f5e16 |
[utils] improve AArch64 asm parser
If we don't mark the cfi line as optional, the script won't work with 'nounwind' code. Without that attr, there may be extra noise in the asm body that we don't want to see. llvm-svn: 330453 |
||
|
Greg Bedwell
|
312638e5f5 |
[UpdateTestChecks] Fix update_mca_test_checks.py slowness issue
The script was using Python's difflib module to calculate the number of lines changed so that it could report it in its status output. It turns out this can be very very slow on large sets of lines (Python bug 6931). It's not worth the cost, so just remove the usage of difflib entirely. llvm-svn: 330419 |
||
Ilya Biryukov
|
30ee746f6f |
[Dockerfiles] Split checkout and build scripts into separate files.
Summary: This is a small refactoring to extract the svn checkout code from the build script used inside the docker image. This would give more flexibility if more than a single invocation of cmake is needed inside the docker image. User-facing interface (build_docker_image.sh) hasn't changed, only the internal scripts running inside the build container are affected. Reviewers: ioeric Reviewed By: ioeric Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D45868 llvm-svn: 330412 |
||
|
Dan Liew
|
63cbec945c |
[lit] Fix a bug where UNRESOLVED tests were not handled in the XUnit
XML printer. A test has been added that tries to comprehensively test emitting XUnit XML output for shell tests. Differential Revision: https://reviews.llvm.org/D45567 llvm-svn: 330409 |
||
Daniel Cederman
|
7415ad946f |
Add SPARC support to update_llc_test_checks.py
Reviewers: spatel, jyknight Reviewed By: spatel Subscribers: fedor.sergeev, llvm-commits Differential Revision: https://reviews.llvm.org/D45809 llvm-svn: 330401 |
||
|
Craig Topper
|
358aaafb83 |
[X86] Remove non-existant instruction name from X86DisassemblerTables.cpp.
This instruction was removed a long time so we don't need to check for it here. llvm-svn: 330363 |
||
|
Simon Pilgrim
|
0cb5a3029f |
[llvm-exegesis] Fix PfmIssueCountersTable creation
This patch ensures that the pfm issue counter tables are the correct size, accounting for the invalid resource entry at the beginning of the resource tables. It also fixes an issue with pfm failing to match event counters due to a trailing comma added to all the event names. I've also added a counter comment to each entry as it helps locate problems with the tables. Note: I don't have access to a SandyBridge test machine, which is the only model to make use of multiple event counters being mapped to a single resource. I don't know if pfm accepts a comma-seperated list or not, but that is what it was doing. Differential Revision: https://reviews.llvm.org/D45787 llvm-svn: 330317 |
||
|
Greg Bedwell
|
9dc124e019 |
[UpdateTestChecks] Add update_mca_test_checks.py script
This script can be used to regenerate tests in the test/tools/llvm-mca directory (PR36904). Regenerated a number of tests using the pattern: test/tools/llvm-mca/*/*/*.s Differential Revision: https://reviews.llvm.org/D45369 llvm-svn: 330246 |
||
Rui Ueyama
|
04fb9911c5 |
Define InitLLVM to do common initialization all at once.
We have a few functions that virtually all command wants to run on process startup/shutdown. This patch adds InitLLVM class to do that all at once, so that we don't need to copy-n-paste boilerplate code to each llvm command's main() function. Differential Revision: https://reviews.llvm.org/D45602 llvm-svn: 330046 |
||
Aaron Smith
|
9d297de1c9 |
[lit] Remove duplicate to_string method
There are two versions of to_string used by TestRunner.py. The one defined in TestRunner.py and the one defined in utils/lit/lit/util.py. The util.py version is superior to the TestRunner.py version. This change removes the duplicate to_string in TestRunner.py in favor of always using the version from util.py. Beside removing duplicate code, this makes it easier to debug TestRunner.py since only one version of to_string is used. Patch by Stella Stamenova! llvm-svn: 329972 |
||
|
Simon Pilgrim
|
5fa636dda8 |
X86FoldTableEntry - avoid unnecessary std::string creation. NFCI.
llvm-svn: 329860 |
||
|
Simon Pilgrim
|
eec640564a |
Don't repeatedly evaluate size() in the for loop. NFCI.
llvm-svn: 329853 |
||
Clement Courbet
|
ec243f03b8 |
[MC][TableGen] Fix r329675.
Caught by bots with -Wmissing-braces. llvm-svn: 329676 |
||
|
Clement Courbet
|
221e6ccd99 |
[MC][TableGen] Add optional libpfm counter names for ProcResUnits.
Summary: Subtargets can define the libpfm counter names that can be used to measure cycles and uops issued on ProcResUnits. This allows making llvm-exegesis available on more targets. Fixes PR36984. Reviewers: gchatelet, RKSimon, andreadb, craig.topper Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45360 llvm-svn: 329675 |
||
Mandeep Singh Grang
|
0068f19cfd |
[unittests] Change std::sort to llvm::sort in response to r327219
r327219 added wrappers to std::sort which randomly shuffle the container before sorting. This will help in uncovering non-determinism caused due to undefined sorting order of objects having the same key. To make use of that infrastructure we need to invoke llvm::sort instead of std::sort. Note: This patch is one of a series of patches to replace *all* std::sort to llvm::sort. Refer the comments section in D44363 for a list of all the required patches. llvm-svn: 329475 |
||
|
Aaron Smith
|
55218b8d96 |
[lit] Fix several Python 2/3 compatibility issues and tests
- In Python 2.x, basestring is the base string type, but in Python 3.x basestring is not defined and instead str includes unicode strings. - When Python is in a path that includes spaces, it needs to be specified with quotes in the test files for it to run. - The cache.ll test relies on files of a specific size being created by Python, but on some versions of Windows the files that are created by the current code are one byte larger than expected. To fix the test, update file creation to always make files of the expected size. Patch by Stella Stamenova! llvm-svn: 329466 |
||
Sameer AbuAsal
|
9cc166efe6 |
[RISCV] Tablegen-driven Instruction Compression.
Summary:
This patch implements a tablegen-driven Instruction Compression
mechanism for generating RISCV compressed instructions
(C Extension) from the expanded instruction form.
This tablegen backend processes CompressPat declarations in a
td file and generates all the compile-time and runtime checks
required to validate the declarations, validate the input
operands and generate correct instructions.
The checks include validating register operands, immediate
operands, fixed register operands and fixed immediate operands.
Example:
class CompressPat<dag input, dag output> {
dag Input = input;
dag Output = output;
list<Predicate> Predicates = [];
}
let Predicates = [HasStdExtC] in {
def : CompressPat<(ADD GPRNoX0:$rs1, GPRNoX0:$rs1, GPRNoX0:$rs2),
(C_ADD GPRNoX0:$rs1, GPRNoX0:$rs2)>;
}
The result is an auto-generated header file
'RISCVGenCompressEmitter.inc' which exports two functions for
compressing/uncompressing MCInst instructions, plus
some helper functions:
bool compressInst(MCInst& OutInst, const MCInst &MI,
const MCSubtargetInfo &STI,
MCContext &Context);
bool uncompressInst(MCInst& OutInst, const MCInst &MI,
const MCRegisterInfo &MRI,
const MCSubtargetInfo &STI);
The clients that include this auto-generated header file and
invoke these functions can compress an instruction before emitting
it, in the target-specific ASM or ELF streamer, or can uncompress
an instruction before printing it, when the expanded instruction
format aliases is favored.
The following clients were added to implement compression\uncompression
for RISCV:
1) RISCVAsmParser::MatchAndEmitInstruction:
Inserted a call to compressInst() to compresses instructions
parsed by llvm-mc coming from an ASM input.
2) RISCVAsmPrinter::EmitInstruction:
Inserted a call to compressInst() to compress instructions that
were lowered from Machine Instructions (MachineInstr).
3) RVInstPrinter::printInst:
Inserted a call to uncompressInst() to print the expanded
version of the instruction instead of the compressed one (e.g,
add s0, s0, a5 instead of c.add s0, a5) when -riscv-no-aliases
is not passed.
This patch squashes D45119, D42780 and D41932. It was reviewed in smaller patches by
asb, efriedma, apazos and mgrang.
Reviewers: asb, efriedma, apazos, llvm-commits, sabuasal
Reviewed By: sabuasal
Subscribers: mgorny, eraman, asb, rbar, johnrusso, simoncook, jordy.potman.lists, apazos, niosHD, kito-cheng, shiva0217, zzheng
Differential Revision: https://reviews.llvm.org/D45385
llvm-svn: 329455
|
||
|
Mandeep Singh Grang
|
aa9538aa70 |
[TableGen] Change std::sort to llvm::sort in response to r327219
Summary: r327219 added wrappers to std::sort which randomly shuffle the container before sorting. This will help in uncovering non-determinism caused due to undefined sorting order of objects having the same key. To make use of that infrastructure we need to invoke llvm::sort instead of std::sort. Note: This patch is one of a series of patches to replace *all* std::sort to llvm::sort. Refer the comments section in D44363 for a list of all the required patches. Reviewers: stoklund, kparzysz, dsanders Reviewed By: dsanders Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D45144 llvm-svn: 329451 |
||
|
Simon Pilgrim
|
9b724910b4 |
[UpdateTestChecks] Add update_analyze_test_checks.py for cost model analysis generation
The script allows the auto-generation of checks for cost model tests to speed up their creation and help improve coverage, which will help a lot with PR36550. If the need arises we can add support for other analyze passes as well, but the cost models was the one I needed to get done - at the moment it just warns that any other analysis mode is unsupported. I've regenerated a couple of x86 test files to show the effect. Differential Revision: https://reviews.llvm.org/D45272 llvm-svn: 329390 |
||
|
Craig Topper
|
dbad6a134b |
[X86] Disassembler support for having an ADSIZE prefix affect instructions with 0xf2 and 0xf3 prefixes.
Needed to support umonitor from D45253. llvm-svn: 329327 |
||
|
Andrea Di Biagio
|
d84c17dd93 |
Fix the buildbots after r329304.
llvm-svn: 329306 |
||
|
Andrea Di Biagio
|
01d033c489 |
[MC][Tablegen] Allow models to describe the retire control unit for llvm-mca.
This patch adds the ability to describe properties of the hardware retire control unit. Tablegen class RetireControlUnit has been added for this purpose (see TargetSchedule.td). A RetireControlUnit specifies the size of the reorder buffer, as well as the maximum number of opcodes that can be retired every cycle. A zero (or negative) value for the reorder buffer size means: "the size is unknown". If the size is unknown, then llvm-mca defaults it to the value of field SchedMachineModel::MicroOpBufferSize. A zero or negative number of opcodes retired per cycle means: "there is no restriction on the number of instructions that can be retired every cycle". Models can optionally specify an instance of RetireControlUnit. There can only be up-to one RetireControlUnit definition per scheduling model. Information related to the RCU (RetireControlUnit) is stored in (two new fields of) MCExtraProcessorInfo. llvm-mca loads that information when it initializes the DispatchUnit / RetireControlUnit (see Dispatch.h/Dispatch.cpp). This patch fixes PR36661. Differential Revision: https://reviews.llvm.org/D45259 llvm-svn: 329304 |
||
|
Andrea Di Biagio
|
217b1ab970 |
[MC] Fix spaces between values printed by EmitRegisterFileInfo.
llvm-svn: 329284 |
||
|
Simon Pilgrim
|
ae3b946d06 |
[SchedModel] Complete models shouldn't match against itineraries when they don't use them (PR35639)
For schedule models that don't use itineraries, checkCompleteness still checks that an instruction has a matching itinerary instead of skipping and going straight to matching the InstRWs. That doesn't seem to match what happens in TargetSchedule.cpp This patch causes problems for a number of models that had been incorrectly flagged as complete. Differential Revision: https://reviews.llvm.org/D43235 llvm-svn: 329280 |
||
|
Simon Pilgrim
|
0112f6cd58 |
[UpdateTestChecks] Moved core functionality of add_asm_checks into add_checks
As discussed on D45272 llvm-svn: 329270 |
||
|
Simon Pilgrim
|
26097b6579 |
[UpdateTestChecks] Split core functionality of add_ir_checks into add_checks
Cherry picked from D45272, also added some setup for add_asm_checks to use add_checks as well. llvm-svn: 329266 |
||
|
Simon Pilgrim
|
435ab9288f |
[UpdateTestChecks] Make add_asm_checks more like add_ir_checks
Towards merging them as mentioned on D45272 llvm-svn: 329265 |
||
|
Simon Pilgrim
|
d257116a5e |
[UpdateTestChecks] Remove unnecessary return from add_ir_checks
llvm-svn: 329262 |
||
|
Nico Weber
|
f497246272 |
Remove llvm-build's --write-make-fragment which looks like a remnant from the autoconf build.
llvm-svn: 329191 |
||
|
Andrea Di Biagio
|
2c2840f2eb |
[Tablegen] Slightly refactor method SubtargetEmitter::EmitExtraProcessorInfo.
This patch moves most of the logic from EmitExtraProcessorInfo to a couple of helper functions. No functional change intended. llvm-svn: 329173 |
||
|
Reid Kleckner
|
d2dd25f14b |
'cat' command for internal shell - Support Python 3
LLVM Bug Id : 36449 Revision 328563 caused tests to fail under python 3. This patch modified cat.py file to support both python 2 and 3. This patch also fixes CRLF issues on Windows. Patch by Chamal de Silva Differential Revision: https://reviews.llvm.org/D45077 llvm-svn: 329123 |
||
Eric Christopher
|
296cab8695 |
Remove a stale comment cut and pasted from another file.
llvm-svn: 329084 |
||
|
Andrea Di Biagio
|
f425ba9f0f |
[MC][Tablegen] Allow the definition of processor register files in the scheduling model for llvm-mca
This patch allows the description of register files in processor scheduling models. This addresses PR36662. A new tablegen class named 'RegisterFile' has been added to TargetSchedule.td. Targets can optionally describe register files for their processors using that class. In particular, class RegisterFile allows to specify: - The total number of physical registers. - Which target registers are accessible through the register file. - The cost of allocating a register at register renaming stage. Example (from this patch - see file X86/X86ScheduleBtVer2.td) def FpuPRF : RegisterFile<72, [VR64, VR128, VR256], [1, 1, 2]> Here, FpuPRF describes a register file for MMX/XMM/YMM registers. On Jaguar (btver2), a YMM register definition consumes 2 physical registers, while MMX/XMM register definitions only cost 1 physical register. The syntax allows to specify an empty set of register classes. An empty set of register classes means: this register file models all the registers specified by the Target. For each register class, users can specify an optional register cost. By default, register costs default to 1. A value of 0 for the number of physical registers means: "this register file has an unbounded number of physical registers". This patch is structured in two parts. * Part 1 - MC/Tablegen * A first part adds the tablegen definition of RegisterFile, and teaches the SubtargetEmitter how to emit information related to register files. Information about register files is accessible through an instance of MCExtraProcessorInfo. The idea behind this design is to logically partition the processor description which is only used by external tools (like llvm-mca) from the processor information used by the llvm machine schedulers. I think that this design would make easier for targets to get rid of the extra processor information if they don't want it. * Part 2 - llvm-mca related * The second part of this patch is related to changes to llvm-mca. The main differences are: 1) class RegisterFile now needs to take into account the "cost of a register" when allocating physical registers at register renaming stage. 2) Point 1. triggered a minor refactoring which lef to the removal of the "maximum 32 register files" restriction. 3) The BackendStatistics view has been updated so that we can print out extra details related to each register file implemented by the processor. The effect of point 3. is also visible in tests register-files-[1..5].s. Differential Revision: https://reviews.llvm.org/D44980 llvm-svn: 329067 |
||
Chandler Carruth
|
405af08695 |
[x86] Fix a pretty obvious think-o with my asm scrubbing. You have to in
fact use regular expression syntax to use regular expressions. Should restore the bots. Sorry for the noise on this test. Thanks to Philip for spotting the bug! llvm-svn: 329057 |
||
|
Chandler Carruth
|
d0826d1238 |
[x86] Extend my goofy SP offset scrubbing for llc test cases to actually
do explicit scrubbing of the offsets of stack spills and reloads. You can always turn this off in order to test specific stack slot usage. We were already hiding most of this, but the new logic hides it more generically. Notably, we should effectively hide stack slot churn in functions that have a frame pointer now, and should also hide it when changing a function from stack pointer to frame pointer. That transition already changes enough to be clearly noticed in the test case diff, showing *every* spill and reload is really noisy without benefit. See the test case I ran this on as a classic example. llvm-svn: 329055 |
||
|
Eric Christopher
|
59e1a9f353 |
Add a wrapper around llvm-objdump to look for indirect calls/jmps in x86 assembly.
Useful when looking for indirect calls/jmps the need mitigation via retpoline or other mitigations for Spectre v2. Feedback, extension, additional patches welcome. llvm-svn: 329050 |
||
|
Craig Topper
|
80960a2e45 |
[X86] Reduce number of OpPrefix bits in TSFlags to 2. NFCI
TSFlag doesn't need to disambiguate NoPrfx from PS. So shift the encodings so PS is NoPrfx|0x4. llvm-svn: 329049 |
||
|
Craig Topper
|
cc790d7e91 |
[X86][TableGen] Add a missing error check to make sure EVEX instructions use one PS/PD/XS/XD prefixes.
llvm-svn: 329048 |
||
|
Craig Topper
|
a9aa267569 |
[TableGen] Use llvm::cast instead of static_cast so that the cast will be checked. NFC
llvm-svn: 329045 |
||
|
Nico Weber
|
7cd17d6948 |
Remove utils/makellvm; it doesn't look like it works with cmake builds.
llvm-svn: 329041 |
||
|
Aaron Smith
|
7a1394d5d7 |
[lit] Prefer opening files with open (Python 2) rather than io.open which requires io.
Only rely on Python 3 (io.open) when necessary. This puts TestRunnyer.py closer to how it behaved before the changes introduced in D43165 and silences a few Windows build bot failures. Thanks to Stella Stamenova for the patch! llvm-svn: 329037 |
||
|
Aaron Smith
|
e5a6e6bd67 |
[lit] One more try at fixing TestRunner.py for D43165
llvm-svn: 329026 |
||
|
Reid Kleckner
|
8a826a7e76 |
[lit] Attempt to fix builtin diff code for Python 2
llvm-svn: 329024 |
||
|
Aaron Smith
|
b62a4d5437 |
[lit] Fix problem in how Python versions open files with different encodings
Reapply D43165 which was reverted because of different versions of python failing. The one line fix for the different python versions was commited at the same time that D43165 was reverted. If this change is giving you issues then get in touch with your python version and we will fix it. llvm-svn: 329022 |
||
|
Aaron Smith
|
4964c3831d |
[lit] Use io.open to compare two files since it supports different encodings while older versions of open do not
llvm-svn: 329020 |
||
|
Reid Kleckner
|
86303cb33d |
Revert r329012 "[lit] Fix problem in how Python versions open files with different encodings"
This doesn't work with Python 2. See this build-in-progress: http://lab.llvm.org:8011/builders/clang-x86-windows-msvc2015/builds/11105 llvm-svn: 329017 |
||
|
Aaron Smith
|
da573c6edd |
[lit] Fix problem in how Python versions open files with different encodings
Summary: This issue was found when running the clang unit test on Windows. Python 3.x cannot open some of the files that the tests are using with a simple open because of their encoding. Python 2.7+ and Python 3.x both support io.open which allows for an encoding to be specified. This change will determine whether two files being compared should be opened (and then compared) as text or binary and whether to use utf-8 or the default encoding before proceeding with a line-by-line comparison. Patch by Stella Stamenova! Reviewers: zturner, llvm-commits, rnk, MaggieYi Reviewed By: zturner Subscribers: asmith, MatzeB, stella.stamenova, delcypher, llvm-commits Differential Revision: https://reviews.llvm.org/D43165 llvm-svn: 329012 |
||
|
Nicolai Haehnle
|
eacc62ef10 |
TableGen: Support Intrinsic values in SearchableTable
Summary: We will use this in the AMDGPU backend in a subsequent patch in the stack to lookup target-specific per-intrinsic information. The generic CodeGenIntrinsic machinery is used to ensure that, even though we don't calculate actual enum values here, we do get the intrinsics in the right order for the binary search index. Change-Id: If61cd5587963a4c5a1cc53df1e59c5e4dec1f9dc Reviewers: arsenm, rampitec, b-sumner Subscribers: wdng, tpr, llvm-commits Differential Revision: https://reviews.llvm.org/D44935 llvm-svn: 328937 |
||
|
Nicolai Haehnle
|
896493fd3f |
TableGen: More helpful error messages
Summary: Change-Id: I3c23f6f6597912423762780cd8c5315870412bbe Reviewers: arsenm, rampitec, b-sumner Subscribers: wdng, llvm-commits Differential Revision: https://reviews.llvm.org/D44936 Change-Id: Ie62614a3e2d7774f46e4034478b28f57100a2c92 llvm-svn: 328936 |
||
Fangrui Song
|
cd23b12dcc |
Fix a bunch of typoes. NFC
llvm-svn: 328907 |
||
David Blaikie
|
2d78e78856 |
Add missing dependency (headers are included from MC, so a link dependency could exist easily enough)
llvm-svn: 328743 |
||
|
Dan Liew
|
2a0a84a46f |
Revert "[lit] Temporarily disable shtest-timeout.py on darwin"
This reverts commit 771829b640a5494ab65c810dd6b4330522bf3a33 (rr328598) Hopefully the test will now pass on the bots. rdar://problem/38774530 llvm-svn: 328703 |
||
|
Dan Liew
|
a120c804c6 |
[lit] Remove a timing senstive part of shtest-timeout.py
The `shtest-timeout.py` test was failing intermittently. It looks like the issue is that on a resource constrained system lit is unable to run `quick_then_slow.py` twice and print out the messages the tests expects within the one second timeout. The underlying issue is that the test is dependent on the performance of the host machine is a rather fragile way. This is due to hardcoding timeout values and having assumptions that the host machine is able to perform a certain amount of work within the hardcoded timeout values. We could increase the timeout values but that doesn't really fix the underlying issue. Instead this patch removes one of fragile assumptions in the hope that this will be enough to fix the bots. There are other fragile assumptions in this test (e.g. `quick.py` can be executed in less than 1 second). If the bots continue to fail we'll have to revisit this. rdar://problem/38774530 llvm-svn: 328702 |
||
|
Craig Topper
|
069a428012 |
[MachineScheduler] Add itinerary to schedcover.py. Make default work in the command line filter
Summary: This patch adds itinerary support to the schedcover.py script. I've been trying to use this script to figure out why SSE and AVX instructions are ending up in separate tablegen scheduler classes and sometimes its because we are using different itineraries. Rather than using None to indicate the default scheduler model, I now use the string "default". I had to hack around the sorting a little to keep "default" at the beginning. But this also makes it so you can specify "default" on the command line to just get the defaults I also fixed the regular expression code so that the no_default wasn't evaluated twice. Reviewers: RKSimon, atrick, jmolloy, javed.absar Reviewed By: javed.absar Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D44834 llvm-svn: 328608 |
||
Mircea Trofin
|
b836fe81fc |
Revert "Revert "[lit] Generalized /dev/null support on Windows.""
Summary: This reverts commit r328596. Checking if the arguments are strings before testing if they contain "/dev/null". Reviewers: rnk Reviewed By: rnk Subscribers: delcypher, llvm-commits Differential Revision: https://reviews.llvm.org/D44914 llvm-svn: 328603 |