RPCS3/llvm-mirror
1
0
Fork 0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2025-01-31 12:41:49 +01:00
Code Issues Projects Releases Wiki Activity

TableGen: Streamline the semantics of NAME

Browse Source 
Summary:
The new rules are straightforward. The main rules to keep in mind
are:

1. NAME is an implicit template argument of class and multiclass,
   and will be substituted by the name of the instantiating def/defm.

2. The name of a def/defm in a multiclass must contain a reference
   to NAME. If such a reference is not present, it is automatically
   prepended.

And for some additional subtleties, consider these:

3. defm with no name generates a unique name but has no special
   behavior otherwise.

4. def with no name generates an anonymous record, whose name is
   unique but undefined. In particular, the name won't contain a
   reference to NAME.

Keeping rules 1&2 in mind should allow a predictable behavior of
name resolution that is simple to follow.

The old "rules" were rather surprising: sometimes (but not always),
NAME would correspond to the name of the toplevel defm. They were
also plain bonkers when you pushed them to their limits, as the old
version of the TableGen test case shows.

Having NAME correspond to the name of the toplevel defm introduces
"spooky action at a distance" and breaks composability:
refactoring the upper layers of a hierarchy of nested multiclass
instantiations can cause unexpected breakage by changing the value
of NAME at a lower level of the hierarchy. The new rules don't
suffer from this problem.

Some existing .td files have to be adjusted because they ended up
depending on the details of the old implementation.

Change-Id: I694095231565b30f563e6fd0417b41ee01a12589

Reviewers: tra, simon_tatham, craig.topper, MartinO, arsenm, javed.absar

Subscribers: wdng, llvm-commits

Differential Revision: https://reviews.llvm.org/D47430

llvm-svn: 333900
This commit is contained in:
Nicolai Haehnle 2018-06-04 14:26:05 +00:00
parent 1823371fca
commit 9a21ccd3ad
16 changed files with 543 additions and 537 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

61
docs/TableGen/LangRef.rst
View File

@ -125,6 +125,7 @@ TableGen's top-level production consists of "objects".
.. productionlist::
Class: "class" `TokIdentifier` [`TemplateArgList`] `ObjectBody`
TemplateArgList: "<" `Declaration` ("," `Declaration`)* ">"
A ``class`` declaration creates a record which other records can inherit
from. A class can be parametrized by a list of "template arguments", whose
@ -145,8 +146,9 @@ forward declaration: note that records deriving from the forward-declared
class will inherit no fields from it since the record expansion is done
when the record is parsed.
.. productionlist::
TemplateArgList: "<" `Declaration` ("," `Declaration`)* ">"
Every class has an implicit template argument called ``NAME``, which is set
to the name of the instantiating ``def`` or ``defm``. The result is undefined
if the class is instantiated by an anonymous record.
Declarations
------------
@ -246,6 +248,8 @@ of:
int Baz = Bar;
}
* the implicit template argument ``NAME`` in a ``class`` or ``multiclass``
.. productionlist::
SimpleValue: `TokInteger`
@ -332,31 +336,56 @@ The ``let`` form allows overriding the value of an inherited field.
``def``
-------
.. TODO::
There can be pastes in the names here, like ``#NAME#``. Look into that
and document it (it boils down to ParseIDValue with IDParseMode ==
ParseNameMode). ParseObjectName calls into the general ParseValue, with
the only different from "arbitrary expression parsing" being IDParseMode
== Mode.
.. productionlist::
Def: "def" `TokIdentifier` `ObjectBody`
Def: "def" [`Value`] `ObjectBody`
Defines a record whose name is given by the :token:`TokIdentifier`. The
fields of the record are inherited from the base classes and defined in the
body.
Defines a record whose name is given by the optional :token:`Value`. The value
is parsed in a special mode where global identifiers (records and variables
defined by ``defset``) are not recognized, and all unrecognized identifiers
are interpreted as strings.
If no name is given, the record is anonymous. The final name of anonymous
records is undefined, but globally unique.
Special handling occurs if this ``def`` appears inside a ``multiclass`` or
a ``foreach``.
When a non-anonymous record is defined in a multiclass and the given name
does not contain a reference to the implicit template argument ``NAME``, such
a reference will automatically be prepended. That is, the following are
equivalent inside a multiclass::
def Foo;
def NAME#Foo;
``defm``
--------
.. productionlist::
Defm: "defm" [`TokIdentifier`] ":" `BaseClassListNE` ";"
Defm: "defm" [`Value`] ":" `BaseClassListNE` ";"
Note that in the :token:`BaseClassList`, all of the ``multiclass``'s must
precede any ``class``'s that appear.
The :token:`BaseClassList` is a list of at least one ``multiclass`` and any
number of ``class``'s. The ``multiclass``'s must occur before any ``class``'s.
Instantiates all records defined in all given ``multiclass``'s and adds the
given ``class``'s as superclasses.
The name is parsed in the same special mode used by ``def``. If the name is
missing, a globally unique string is used instead (but instantiated records
are not considered to be anonymous, unless they were originally defined by an
anonymous ``def``) That is, the following have different semantics::
defm : SomeMultiClass<...>; // some globally unique name
defm "" : SomeMultiClass<...>; // empty name string
When it occurs inside a multiclass, the second variant is equivalent to
``defm NAME : ...``. More generally, when ``defm`` occurs in a multiclass and
its name does not contain a reference to the implicit template argument
``NAME``, such a reference will automatically be prepended. That is, the
following are equivalent inside a multiclass::
defm Foo : SomeMultiClass<...>;
defm NAME#Foo : SomeMultiClass<...>;
``defset``
----------

7
docs/TableGen/index.rst
View File

@ -171,13 +171,6 @@ factor out the common features that instructions of its class share. A key
feature of TableGen is that it allows the end-user to define the abstractions
they prefer to use when describing their information.
Each ``def`` record has a special entry called "NAME". This is the name of the
record ("``ADD32rr``" above). In the general case ``def`` names can be formed
from various kinds of string processing expressions and ``NAME`` resolves to the
final value obtained after resolving all of those expressions. The user may
refer to ``NAME`` anywhere she desires to use the ultimate name of the ``def``.
``NAME`` should not be defined anywhere else in user code to avoid conflicts.
Syntax
======

49
include/llvm/TableGen/Record.h
View File

@ -1357,30 +1357,31 @@ class Record {
unsigned ID;
bool IsAnonymous;
bool IsClass;
void init();
void checkName();
public:
// Constructs a record.
explicit Record(Init *N, ArrayRef<SMLoc> locs, RecordKeeper &records,
bool Anonymous = false) :
Name(N), Locs(locs.begin(), locs.end()), TrackedRecords(records),
ID(LastID++), IsAnonymous(Anonymous) {
init();
bool Anonymous = false, bool Class = false)
: Name(N), Locs(locs.begin(), locs.end()), TrackedRecords(records),
ID(LastID++), IsAnonymous(Anonymous), IsClass(Class) {
checkName();
}
explicit Record(StringRef N, ArrayRef<SMLoc> locs, RecordKeeper &records)
: Record(StringInit::get(N), locs, records) {}
explicit Record(StringRef N, ArrayRef<SMLoc> locs, RecordKeeper &records,
bool Class = false)
: Record(StringInit::get(N), locs, records, false, Class) {}
// When copy-constructing a Record, we must still guarantee a globally unique
// ID number. Don't copy TheInit either since it's owned by the original
// record. All other fields can be copied normally.
Record(const Record &O) :
Name(O.Name), Locs(O.Locs), TemplateArgs(O.TemplateArgs),
Values(O.Values), SuperClasses(O.SuperClasses),
TrackedRecords(O.TrackedRecords), ID(LastID++),
IsAnonymous(O.IsAnonymous) { }
Record(const Record &O)
: Name(O.Name), Locs(O.Locs), TemplateArgs(O.TemplateArgs),
Values(O.Values), SuperClasses(O.SuperClasses),
TrackedRecords(O.TrackedRecords), ID(LastID++),
IsAnonymous(O.IsAnonymous), IsClass(O.IsClass) { }
static unsigned getNewUID() { return LastID++; }
@ -1407,6 +1408,8 @@ public:
/// get the corresponding DefInit.
DefInit *getDefInit();
bool isClass() const { return IsClass; }
ArrayRef<Init *> getTemplateArgs() const {
return TemplateArgs;
}
@ -1452,13 +1455,6 @@ public:
void addValue(const RecordVal &RV) {
assert(getValue(RV.getNameInit()) == nullptr && "Value already added!");
Values.push_back(RV);
if (Values.size() > 1)
// Keep NAME at the end of the list. It makes record dumps a
// bit prettier and allows TableGen tests to be written more
// naturally. Tests can use CHECK-NEXT to look for Record
// fields they expect to see after a def. They can't do that if
// NAME is the first Record field.
std::swap(Values[Values.size() - 2], Values[Values.size() - 1]);
}
void removeValue(Init *Name) {
@ -1905,6 +1901,21 @@ public:
Init *resolve(Init *VarName) override;
};
/// Do not resolve anything, but keep track of whether a given variable was
/// referenced.
class HasReferenceResolver final : public Resolver {
Init *VarNameToTrack;
bool Found = false;
public:
explicit HasReferenceResolver(Init *VarNameToTrack)
: Resolver(nullptr), VarNameToTrack(VarNameToTrack) {}
bool found() const { return Found; }
Init *resolve(Init *VarName) override;
};
} // end namespace llvm
#endif // LLVM_TABLEGEN_RECORD_H

21
lib/TableGen/Record.cpp
View File

@ -1572,10 +1572,8 @@ DefInit *VarDefInit::instantiate() {
Record *NewRec = NewRecOwner.get();
// Copy values from class to instance
for (const RecordVal &Val : Class->getValues()) {
if (Val.getName() != "NAME")
NewRec->addValue(Val);
}
for (const RecordVal &Val : Class->getValues())
NewRec->addValue(Val);
// Substitute and resolve template arguments
ArrayRef<Init *> TArgs = Class->getTemplateArgs();
@ -1845,14 +1843,6 @@ void RecordVal::print(raw_ostream &OS, bool PrintSem) const {
unsigned Record::LastID = 0;
void Record::init() {
checkName();
// Every record potentially has a def at the top. This value is
// replaced with the top-level def name at instantiation time.
addValue(RecordVal(StringInit::get("NAME"), StringRecTy::get(), false));
}
void Record::checkName() {
// Ensure the record name has string type.
const TypedInit *TypedName = cast<const TypedInit>(Name);
@ -2260,3 +2250,10 @@ Init *TrackUnresolvedResolver::resolve(Init *VarName) {
FoundUnresolved = true;
return I;
}
Init *HasReferenceResolver::resolve(Init *VarName)
{
if (VarName == VarNameToTrack)
Found = true;
return nullptr;
}

253
lib/TableGen/TGParser.cpp
View File

@ -110,6 +110,16 @@ static void checkConcrete(Record &R) {
}
}
/// Return the qualified version of the implicit 'NAME' template argument.
static Init *QualifiedNameOfImplicitName(Record &Rec,
MultiClass *MC = nullptr) {
return QualifyName(Rec, MC, StringInit::get("NAME"), MC ? "::" : ":");
}
static Init *QualifiedNameOfImplicitName(MultiClass *MC) {
return QualifiedNameOfImplicitName(MC->Rec, MC);
}
bool TGParser::AddValue(Record *CurRec, SMLoc Loc, const RecordVal &RV) {
if (!CurRec)
CurRec = &CurMultiClass->Rec;
@ -234,6 +244,14 @@ bool TGParser::AddSubClass(Record *CurRec, SubClassReference &SubClass) {
CurRec->removeValue(TArgs[i]);
}
Init *Name;
if (CurRec->isClass())
Name =
VarInit::get(QualifiedNameOfImplicitName(*CurRec), StringRecTy::get());
else
Name = CurRec->getNameInit();
R.set(QualifiedNameOfImplicitName(*SC), Name);
CurRec->resolveReferences(R);
// Since everything went well, we can now set the "superclass" list for the
@ -259,80 +277,45 @@ bool TGParser::AddSubClass(Record *CurRec, SubClassReference &SubClass) {
bool TGParser::AddSubMultiClass(MultiClass *CurMC,
SubMultiClassReference &SubMultiClass) {
MultiClass *SMC = SubMultiClass.MC;
Record *CurRec = &CurMC->Rec;
// Add all of the values in the subclass into the current class.
for (const auto &SMCVal : SMC->Rec.getValues())
if (AddValue(CurRec, SubMultiClass.RefRange.Start, SMCVal))
return true;
unsigned newDefStart = CurMC->DefPrototypes.size();
// Add all of the defs in the subclass into the current multiclass.
for (const std::unique_ptr<Record> &R : SMC->DefPrototypes) {
// Clone the def and add it to the current multiclass
auto NewDef = make_unique<Record>(*R);
// Add all of the values in the superclass into the current def.
for (const auto &MCVal : CurRec->getValues())
if (AddValue(NewDef.get(), SubMultiClass.RefRange.Start, MCVal))
return true;
CurMC->DefPrototypes.push_back(std::move(NewDef));
}
ArrayRef<Init *> SMCTArgs = SMC->Rec.getTemplateArgs();
// Ensure that an appropriate number of template arguments are
// specified.
if (SMCTArgs.size() < SubMultiClass.TemplateArgs.size())
return Error(SubMultiClass.RefRange.Start,
"More template args specified than expected");
// Loop over all of the template arguments, setting them to the specified
// value or leaving them as the default if necessary.
MapResolver CurRecResolver(CurRec);
// Prepare the mapping of template argument name to value, filling in default
// values if necessary.
SmallVector<std::pair<Init *, Init *>, 8> TemplateArgs;
for (unsigned i = 0, e = SMCTArgs.size(); i != e; ++i) {
if (i < SubMultiClass.TemplateArgs.size()) {
// If a value is specified for this template arg, set it in the
// superclass now.
if (SetValue(CurRec, SubMultiClass.RefRange.Start, SMCTArgs[i],
None, SubMultiClass.TemplateArgs[i]))
return true;
// If a value is specified for this template arg, set it in the
// new defs now.
for (const auto &Def :
makeArrayRef(CurMC->DefPrototypes).slice(newDefStart)) {
if (SetValue(Def.get(), SubMultiClass.RefRange.Start, SMCTArgs[i],
None, SubMultiClass.TemplateArgs[i]))
return true;
TemplateArgs.emplace_back(SMCTArgs[i], SubMultiClass.TemplateArgs[i]);
} else {
Init *Default = SMC->Rec.getValue(SMCTArgs[i])->getValue();
if (!Default->isComplete()) {
return Error(SubMultiClass.RefRange.Start,
"value not specified for template argument #" + Twine(i) +
" (" + SMCTArgs[i]->getAsUnquotedString() +
") of multiclass '" + SMC->Rec.getNameInitAsString() +
"'");
}
} else if (!CurRec->getValue(SMCTArgs[i])->getValue()->isComplete()) {
return Error(SubMultiClass.RefRange.Start,
"Value not specified for template argument #" +
Twine(i) + " (" + SMCTArgs[i]->getAsUnquotedString() +
") of subclass '" + SMC->Rec.getNameInitAsString() + "'!");
TemplateArgs.emplace_back(SMCTArgs[i], Default);
}
CurRecResolver.set(SMCTArgs[i], CurRec->getValue(SMCTArgs[i])->getValue());
CurRec->removeValue(SMCTArgs[i]);
}
CurRec->resolveReferences(CurRecResolver);
TemplateArgs.emplace_back(
QualifiedNameOfImplicitName(SMC),
VarInit::get(QualifiedNameOfImplicitName(CurMC), StringRecTy::get()));
for (const auto &Def :
makeArrayRef(CurMC->DefPrototypes).slice(newDefStart)) {
MapResolver R(Def.get());
// Add all of the defs in the subclass into the current multiclass.
for (const std::unique_ptr<Record> &Rec : SMC->DefPrototypes) {
auto NewDef = make_unique<Record>(*Rec);
for (Init *SMCTArg : SMCTArgs) {
R.set(SMCTArg, Def->getValue(SMCTArg)->getValue());
Def->removeValue(SMCTArg);
}
MapResolver R(NewDef.get());
for (const auto &TArg : TemplateArgs)
R.set(TArg.first, TArg.second);
NewDef->resolveReferences(R);
Def->resolveReferences(R);
CurMC->DefPrototypes.push_back(std::move(NewDef));
}
return false;
@ -343,18 +326,18 @@ bool TGParser::AddSubMultiClass(MultiClass *CurMC,
///
/// Apply foreach loops, resolve internal variable references, and add to the
/// current multi class or the global record keeper as appropriate.
bool TGParser::addDef(std::unique_ptr<Record> Rec, Init *DefmName) {
bool TGParser::addDef(std::unique_ptr<Record> Rec) {
IterSet IterVals;
if (Loops.empty())
return addDefOne(std::move(Rec), DefmName, IterVals);
return addDefOne(std::move(Rec), IterVals);
return addDefForeach(Rec.get(), DefmName, IterVals);
return addDefForeach(Rec.get(), IterVals);
}
/// Recursive helper function for addDef/addDefOne to resolve references to
/// foreach variables.
bool TGParser::addDefForeach(Record *Rec, Init *DefmName, IterSet &IterVals) {
bool TGParser::addDefForeach(Record *Rec, IterSet &IterVals) {
if (IterVals.size() != Loops.size()) {
assert(IterVals.size() < Loops.size());
ForeachLoop &CurLoop = Loops[IterVals.size()];
@ -363,7 +346,7 @@ bool TGParser::addDefForeach(Record *Rec, Init *DefmName, IterSet &IterVals) {
// Process each value.
for (unsigned i = 0; i < List->size(); ++i) {
IterVals.push_back(IterRecord(CurLoop.IterVar, List->getElement(i)));
if (addDefForeach(Rec, DefmName, IterVals))
if (addDefForeach(Rec, IterVals))
return true;
IterVals.pop_back();
}
@ -374,13 +357,12 @@ bool TGParser::addDefForeach(Record *Rec, Init *DefmName, IterSet &IterVals) {
// for this point in the iteration space. Instantiate a new record to
// reflect this combination of values.
auto IterRec = make_unique<Record>(*Rec);
return addDefOne(std::move(IterRec), DefmName, IterVals);
return addDefOne(std::move(IterRec), IterVals);
}
/// After resolving foreach loops, add the record as a prototype to the
/// current multiclass, or resolve fully and add to the record keeper.
bool TGParser::addDefOne(std::unique_ptr<Record> Rec, Init *DefmName,
IterSet &IterVals) {
bool TGParser::addDefOne(std::unique_ptr<Record> Rec, IterSet &IterVals) {
MapResolver R(Rec.get());
for (IterRecord &IR : IterVals)
@ -389,32 +371,21 @@ bool TGParser::addDefOne(std::unique_ptr<Record> Rec, Init *DefmName,
Rec->resolveReferences(R);
if (CurMultiClass) {
for (const auto &Proto : CurMultiClass->DefPrototypes) {
if (Proto->getNameInit() == Rec->getNameInit()) {
if (!Rec->isAnonymous()) {
if (!Rec->isAnonymous()) {
for (const auto &Proto : CurMultiClass->DefPrototypes) {
if (Proto->getNameInit() == Rec->getNameInit()) {
PrintError(Rec->getLoc(),
Twine("def '") + Rec->getNameInitAsString() +
"' already defined in this multiclass!");
PrintNote(Proto->getLoc(), "location of previous definition");
return true;
}
Rec->setName(Records.getNewAnonymousName());
break;
}
}
CurMultiClass->DefPrototypes.emplace_back(std::move(Rec));
return false;
}
// Name construction is an incoherent mess. Unfortunately, existing .td
// files rely on pretty much all the quirks and implementation details of
// this.
if (DefmName) {
MapResolver R(Rec.get());
R.set(StringInit::get("NAME"), DefmName);
Rec->resolveReferences(R);
}
if (Record *Prev = Records.getDef(Rec->getNameInitAsString())) {
if (!Rec->isAnonymous()) {
PrintError(Rec->getLoc(),
@ -488,7 +459,20 @@ Init *TGParser::ParseObjectName(MultiClass *CurMultiClass) {
if (CurMultiClass)
CurRec = &CurMultiClass->Rec;
return ParseValue(CurRec, StringRecTy::get(), ParseNameMode);
Init *Name = ParseValue(CurRec, StringRecTy::get(), ParseNameMode);
if (!Name)
return nullptr;
if (CurMultiClass) {
Init *NameStr = QualifiedNameOfImplicitName(CurMultiClass);
HasReferenceResolver R(NameStr);
Name->resolveReferences(R);
if (!R.found())
Name = BinOpInit::getStrConcat(VarInit::get(NameStr, StringRecTy::get()),
Name);
}
return Name;
}
/// ParseClassID - Parse and resolve a reference to a class name. This returns
@ -798,30 +782,23 @@ Init *TGParser::ParseIDValue(Record *CurRec, StringInit *Name, SMLoc NameLoc,
if (CurRec) {
if (const RecordVal *RV = CurRec->getValue(Name))
return VarInit::get(Name, RV->getType());
Init *TemplateArgName = QualifyName(*CurRec, CurMultiClass, Name, ":");
if (CurMultiClass)
TemplateArgName = QualifyName(CurMultiClass->Rec, CurMultiClass, Name,
"::");
if (CurRec->isTemplateArg(TemplateArgName)) {
const RecordVal *RV = CurRec->getValue(TemplateArgName);
assert(RV && "Template arg doesn't exist??");
return VarInit::get(TemplateArgName, RV->getType());
}
}
if (CurMultiClass) {
if (Name->getValue() == "NAME")
return VarInit::get(Name, StringRecTy::get());
if ((CurRec && CurRec->isClass()) || CurMultiClass) {
Init *TemplateArgName;
if (CurMultiClass) {
TemplateArgName =
QualifyName(CurMultiClass->Rec, CurMultiClass, Name, "::");
} else
TemplateArgName = QualifyName(*CurRec, CurMultiClass, Name, ":");
Init *MCName = QualifyName(CurMultiClass->Rec, CurMultiClass, Name, "::");
if (CurMultiClass->Rec.isTemplateArg(MCName)) {
const RecordVal *RV = CurMultiClass->Rec.getValue(MCName);
Record *TemplateRec = CurMultiClass ? &CurMultiClass->Rec : CurRec;
if (TemplateRec->isTemplateArg(TemplateArgName)) {
const RecordVal *RV = TemplateRec->getValue(TemplateArgName);
assert(RV && "Template arg doesn't exist??");
return VarInit::get(MCName, RV->getType());
return VarInit::get(TemplateArgName, RV->getType());
} else if (Name->getValue() == "NAME") {
return VarInit::get(TemplateArgName, StringRecTy::get());
}
}
@ -840,15 +817,12 @@ Init *TGParser::ParseIDValue(Record *CurRec, StringInit *Name, SMLoc NameLoc,
// Allow self-references of concrete defs, but delay the lookup so that we
// get the correct type.
if (CurRec && !CurMultiClass && CurRec->getNameInit() == Name)
if (CurRec && !CurRec->isClass() && !CurMultiClass &&
CurRec->getNameInit() == Name)
return UnOpInit::get(UnOpInit::CAST, Name, CurRec->getType());
if (Mode == ParseValueMode) {
Error(NameLoc, "Variable not defined: '" + Name->getValue() + "'");
return nullptr;
}
return Name;
Error(NameLoc, "Variable not defined: '" + Name->getValue() + "'");
return nullptr;
}
/// ParseOperation - Parse an operator. This returns null on error.
@ -2169,8 +2143,14 @@ Init *TGParser::ParseDeclaration(Record *CurRec,
return nullptr;
}
std::string Str = Lex.getCurStrVal();
if (Str == "NAME") {
TokError("'" + Str + "' is a reserved variable name");
return nullptr;
}
SMLoc IdLoc = Lex.getLoc();
Init *DeclName = StringInit::get(Lex.getCurStrVal());
Init *DeclName = StringInit::get(Str);
Lex.Lex();
if (ParsingTemplateArgs) {
@ -2471,7 +2451,7 @@ bool TGParser::ParseDef(MultiClass *CurMultiClass) {
if (ParseObjectBody(CurRec.get()))
return true;
return addDef(std::move(CurRec), nullptr);
return addDef(std::move(CurRec));
}
/// ParseDefset - Parse a defset statement.
@ -2585,7 +2565,7 @@ bool TGParser::ParseClass() {
Record *CurRec = Records.getClass(Lex.getCurStrVal());
if (CurRec) {
// If the body was previously defined, this is an error.
if (CurRec->getValues().size() > 1 || // Account for NAME.
if (!CurRec->getValues().empty() ||
!CurRec->getSuperClasses().empty() ||
!CurRec->getTemplateArgs().empty())
return TokError("Class '" + CurRec->getNameInitAsString() +
@ -2593,7 +2573,8 @@ bool TGParser::ParseClass() {
} else {
// If this is the first reference to this class, create and add it.
auto NewRec =
llvm::make_unique<Record>(Lex.getCurStrVal(), Lex.getLoc(), Records);
llvm::make_unique<Record>(Lex.getCurStrVal(), Lex.getLoc(), Records,
/*Class=*/true);
CurRec = NewRec.get();
Records.addClass(std::move(NewRec));
}
@ -2604,7 +2585,6 @@ bool TGParser::ParseClass() {
if (ParseTemplateArgList(CurRec))
return true;
// Finally, parse the object body.
return ParseObjectBody(CurRec);
}
@ -2802,8 +2782,14 @@ bool TGParser::ParseDefm(MultiClass *CurMultiClass) {
Init *DefmName = ParseObjectName(CurMultiClass);
if (!DefmName)
return true;
if (isa<UnsetInit>(DefmName))
if (isa<UnsetInit>(DefmName)) {
DefmName = Records.getNewAnonymousName();
if (CurMultiClass)
DefmName = BinOpInit::getStrConcat(
VarInit::get(QualifiedNameOfImplicitName(CurMultiClass),
StringRecTy::get()),
DefmName);
}
if (Lex.getCode() != tgtok::colon)
return TokError("expected ':' after defm identifier");
@ -2836,10 +2822,10 @@ bool TGParser::ParseDefm(MultiClass *CurMultiClass) {
return Error(SubClassLoc,
"more template args specified than multiclass expects");
DenseMap<Init *, Init *> TemplateArgs;
SmallVector<std::pair<Init *, Init *>, 8> TemplateArgs;
for (unsigned i = 0, e = TArgs.size(); i != e; ++i) {
if (i < TemplateVals.size()) {
TemplateArgs.insert({TArgs[i], TemplateVals[i]});
TemplateArgs.emplace_back(TArgs[i], TemplateVals[i]);
} else {
Init *Default = MC->Rec.getValue(TArgs[i])->getValue();
if (!Default->isComplete()) {
@ -2849,45 +2835,20 @@ bool TGParser::ParseDefm(MultiClass *CurMultiClass) {
") of multiclass '" + MC->Rec.getNameInitAsString() +
"'");
}
TemplateArgs.insert({TArgs[i], Default});
TemplateArgs.emplace_back(TArgs[i], Default);
}
}
TemplateArgs.emplace_back(QualifiedNameOfImplicitName(MC), DefmName);
// Loop over all the def's in the multiclass, instantiating each one.
for (const std::unique_ptr<Record> &DefProto : MC->DefPrototypes) {
bool ResolveName = true;
auto CurRec = make_unique<Record>(*DefProto);
CurRec->appendLoc(SubClassLoc);
if (StringInit *NameString =
dyn_cast<StringInit>(CurRec->getNameInit())) {
// We have a fully expanded string so there are no operators to
// resolve. We should concatenate the given prefix and name.
//
// TODO: This MUST happen before template argument resolution. This
// does not make sense and should be changed, but at the time of
// writing, there are existing .td files which rely on this
// implementation detail. It's a bad idea and should be fixed.
// See test/TableGen/name-resolution-consistency.td for some
// examples.
CurRec->setName(BinOpInit::getStrConcat(DefmName, NameString));
ResolveName = false;
}
MapResolver R(CurRec.get());
if (ResolveName) {
// If the proto's name wasn't resolved, we probably have a reference to
// NAME and need to replace it.
//
// TODO: Whether the name is resolved is basically determined by magic.
// Unfortunately, existing .td files depend on it.
R.set(StringInit::get("NAME"), DefmName);
}
for (const auto &TArg : TemplateArgs)
R.set(TArg.first, TArg.second);
CurRec->resolveReferences(R);
NewRecDefs.emplace_back(std::move(CurRec));
@ -2937,7 +2898,7 @@ bool TGParser::ParseDefm(MultiClass *CurMultiClass) {
if (ApplyLetStack(CurRec.get()))
return true;
addDef(std::move(CurRec), DefmName);
addDef(std::move(CurRec));
}
if (Lex.getCode() != tgtok::semi)

7
lib/TableGen/TGParser.h
View File

@ -126,10 +126,9 @@ private: // Semantic analysis methods.
// iteration space.
typedef std::vector<IterRecord> IterSet;
bool addDefOne(std::unique_ptr<Record> Rec, Init *DefmName,
IterSet &IterVals);
bool addDefForeach(Record *Rec, Init *DefmName, IterSet &IterVals);
bool addDef(std::unique_ptr<Record> Rec, Init *DefmName);
bool addDefOne(std::unique_ptr<Record> Rec, IterSet &IterVals);
bool addDefForeach(Record *Rec, IterSet &IterVals);
bool addDef(std::unique_ptr<Record> Rec);
private: // Parser methods.
bool ParseObjectList(MultiClass *MC = nullptr);

144
lib/Target/AArch64/AArch64InstrFormats.td
View File

@ -8655,14 +8655,14 @@ class BaseSIMDLdStPost<bit Q, bit L, bits<4> opcode, bits<2> size,
// The immediate form of AdvSIMD post-indexed addressing is encoded with
// register post-index addressing from the zero register.
multiclass SIMDLdStAliases<string asm, string layout, string Count,
multiclass SIMDLdStAliases<string BaseName, string asm, string layout, string Count,
int Offset, int Size> {
// E.g. "ld1 { v0.8b, v1.8b }, [x1], #16"
// "ld1\t$Vt, [$Rn], #16"
// may get mapped to
// (LD1Twov8b_POST VecListTwo8b:$Vt, GPR64sp:$Rn, XZR)
def : InstAlias<asm # "\t$Vt, [$Rn], #" # Offset,
(!cast<Instruction>(NAME # Count # "v" # layout # "_POST")
(!cast<Instruction>(BaseName # Count # "v" # layout # "_POST")
GPR64sp:$Rn,
!cast<RegisterOperand>("VecList" # Count # layout):$Vt,
XZR), 1>;
@ -8672,7 +8672,7 @@ multiclass SIMDLdStAliases<string asm, string layout, string Count,
// may get mapped to
// (LD1Twov8b_POST VecListTwo64:$Vt, GPR64sp:$Rn, XZR)
def : InstAlias<asm # "." # layout # "\t$Vt, [$Rn], #" # Offset,
(!cast<Instruction>(NAME # Count # "v" # layout # "_POST")
(!cast<Instruction>(BaseName # Count # "v" # layout # "_POST")
GPR64sp:$Rn,
!cast<RegisterOperand>("VecList" # Count # Size):$Vt,
XZR), 0>;
@ -8682,7 +8682,7 @@ multiclass SIMDLdStAliases<string asm, string layout, string Count,
// may get mapped to
// (LD1Twov8b VecListTwo64:$Vt, GPR64sp:$Rn)
def : InstAlias<asm # "." # layout # "\t$Vt, [$Rn]",
(!cast<Instruction>(NAME # Count # "v" # layout)
(!cast<Instruction>(BaseName # Count # "v" # layout)
!cast<RegisterOperand>("VecList" # Count # Size):$Vt,
GPR64sp:$Rn), 0>;
@ -8691,14 +8691,14 @@ multiclass SIMDLdStAliases<string asm, string layout, string Count,
// may get mapped to
// (LD1Twov8b_POST VecListTwo64:$Vt, GPR64sp:$Rn, GPR64pi8:$Xm)
def : InstAlias<asm # "." # layout # "\t$Vt, [$Rn], $Xm",
(!cast<Instruction>(NAME # Count # "v" # layout # "_POST")
(!cast<Instruction>(BaseName # Count # "v" # layout # "_POST")
GPR64sp:$Rn,
!cast<RegisterOperand>("VecList" # Count # Size):$Vt,
!cast<RegisterOperand>("GPR64pi" # Offset):$Xm), 0>;
}
multiclass BaseSIMDLdN<string Count, string asm, string veclist, int Offset128,
int Offset64, bits<4> opcode> {
multiclass BaseSIMDLdN<string BaseName, string Count, string asm, string veclist,
int Offset128, int Offset64, bits<4> opcode> {
let hasSideEffects = 0, mayLoad = 1, mayStore = 0 in {
def v16b: BaseSIMDLdSt<1, 1, opcode, 0b00, asm,
(outs !cast<RegisterOperand>(veclist # "16b"):$Vt),
@ -8760,18 +8760,18 @@ multiclass BaseSIMDLdN<string Count, string asm, string veclist, int Offset128,
!cast<RegisterOperand>("GPR64pi" # Offset64):$Xm)>;
}
defm : SIMDLdStAliases<asm, "16b", Count, Offset128, 128>;
defm : SIMDLdStAliases<asm, "8h", Count, Offset128, 128>;
defm : SIMDLdStAliases<asm, "4s", Count, Offset128, 128>;
defm : SIMDLdStAliases<asm, "2d", Count, Offset128, 128>;
defm : SIMDLdStAliases<asm, "8b", Count, Offset64, 64>;
defm : SIMDLdStAliases<asm, "4h", Count, Offset64, 64>;
defm : SIMDLdStAliases<asm, "2s", Count, Offset64, 64>;
defm : SIMDLdStAliases<BaseName, asm, "16b", Count, Offset128, 128>;
defm : SIMDLdStAliases<BaseName, asm, "8h", Count, Offset128, 128>;
defm : SIMDLdStAliases<BaseName, asm, "4s", Count, Offset128, 128>;
defm : SIMDLdStAliases<BaseName, asm, "2d", Count, Offset128, 128>;
defm : SIMDLdStAliases<BaseName, asm, "8b", Count, Offset64, 64>;
defm : SIMDLdStAliases<BaseName, asm, "4h", Count, Offset64, 64>;
defm : SIMDLdStAliases<BaseName, asm, "2s", Count, Offset64, 64>;
}
// Only ld1/st1 has a v1d version.
multiclass BaseSIMDStN<string Count, string asm, string veclist, int Offset128,
int Offset64, bits<4> opcode> {
multiclass BaseSIMDStN<string BaseName, string Count, string asm, string veclist,
int Offset128, int Offset64, bits<4> opcode> {
let hasSideEffects = 0, mayStore = 1, mayLoad = 0 in {
def v16b : BaseSIMDLdSt<1, 0, opcode, 0b00, asm, (outs),
(ins !cast<RegisterOperand>(veclist # "16b"):$Vt,
@ -8832,18 +8832,18 @@ multiclass BaseSIMDStN<string Count, string asm, string veclist, int Offset128,
!cast<RegisterOperand>("GPR64pi" # Offset64):$Xm)>;
}
defm : SIMDLdStAliases<asm, "16b", Count, Offset128, 128>;
defm : SIMDLdStAliases<asm, "8h", Count, Offset128, 128>;
defm : SIMDLdStAliases<asm, "4s", Count, Offset128, 128>;
defm : SIMDLdStAliases<asm, "2d", Count, Offset128, 128>;
defm : SIMDLdStAliases<asm, "8b", Count, Offset64, 64>;
defm : SIMDLdStAliases<asm, "4h", Count, Offset64, 64>;
defm : SIMDLdStAliases<asm, "2s", Count, Offset64, 64>;
defm : SIMDLdStAliases<BaseName, asm, "16b", Count, Offset128, 128>;
defm : SIMDLdStAliases<BaseName, asm, "8h", Count, Offset128, 128>;
defm : SIMDLdStAliases<BaseName, asm, "4s", Count, Offset128, 128>;
defm : SIMDLdStAliases<BaseName, asm, "2d", Count, Offset128, 128>;
defm : SIMDLdStAliases<BaseName, asm, "8b", Count, Offset64, 64>;
defm : SIMDLdStAliases<BaseName, asm, "4h", Count, Offset64, 64>;
defm : SIMDLdStAliases<BaseName, asm, "2s", Count, Offset64, 64>;
}
multiclass BaseSIMDLd1<string Count, string asm, string veclist,
multiclass BaseSIMDLd1<string BaseName, string Count, string asm, string veclist,
int Offset128, int Offset64, bits<4> opcode>
: BaseSIMDLdN<Count, asm, veclist, Offset128, Offset64, opcode> {
: BaseSIMDLdN<BaseName, Count, asm, veclist, Offset128, Offset64, opcode> {
// LD1 instructions have extra "1d" variants.
let hasSideEffects = 0, mayLoad = 1, mayStore = 0 in {
@ -8858,12 +8858,12 @@ multiclass BaseSIMDLd1<string Count, string asm, string veclist,
!cast<RegisterOperand>("GPR64pi" # Offset64):$Xm)>;
}
defm : SIMDLdStAliases<asm, "1d", Count, Offset64, 64>;
defm : SIMDLdStAliases<BaseName, asm, "1d", Count, Offset64, 64>;
}
multiclass BaseSIMDSt1<string Count, string asm, string veclist,
multiclass BaseSIMDSt1<string BaseName, string Count, string asm, string veclist,
int Offset128, int Offset64, bits<4> opcode>
: BaseSIMDStN<Count, asm, veclist, Offset128, Offset64, opcode> {
: BaseSIMDStN<BaseName, Count, asm, veclist, Offset128, Offset64, opcode> {
// ST1 instructions have extra "1d" variants.
let hasSideEffects = 0, mayLoad = 0, mayStore = 1 in {
@ -8878,45 +8878,45 @@ multiclass BaseSIMDSt1<string Count, string asm, string veclist,
!cast<RegisterOperand>("GPR64pi" # Offset64):$Xm)>;
}
defm : SIMDLdStAliases<asm, "1d", Count, Offset64, 64>;
defm : SIMDLdStAliases<BaseName, asm, "1d", Count, Offset64, 64>;
}
multiclass SIMDLd1Multiple<string asm> {
defm One : BaseSIMDLd1<"One", asm, "VecListOne", 16, 8, 0b0111>;
defm Two : BaseSIMDLd1<"Two", asm, "VecListTwo", 32, 16, 0b1010>;
defm Three : BaseSIMDLd1<"Three", asm, "VecListThree", 48, 24, 0b0110>;
defm Four : BaseSIMDLd1<"Four", asm, "VecListFour", 64, 32, 0b0010>;
defm One : BaseSIMDLd1<NAME, "One", asm, "VecListOne", 16, 8, 0b0111>;
defm Two : BaseSIMDLd1<NAME, "Two", asm, "VecListTwo", 32, 16, 0b1010>;
defm Three : BaseSIMDLd1<NAME, "Three", asm, "VecListThree", 48, 24, 0b0110>;
defm Four : BaseSIMDLd1<NAME, "Four", asm, "VecListFour", 64, 32, 0b0010>;
}
multiclass SIMDSt1Multiple<string asm> {
defm One : BaseSIMDSt1<"One", asm, "VecListOne", 16, 8, 0b0111>;
defm Two : BaseSIMDSt1<"Two", asm, "VecListTwo", 32, 16, 0b1010>;
defm Three : BaseSIMDSt1<"Three", asm, "VecListThree", 48, 24, 0b0110>;
defm Four : BaseSIMDSt1<"Four", asm, "VecListFour", 64, 32, 0b0010>;
defm One : BaseSIMDSt1<NAME, "One", asm, "VecListOne", 16, 8, 0b0111>;
defm Two : BaseSIMDSt1<NAME, "Two", asm, "VecListTwo", 32, 16, 0b1010>;
defm Three : BaseSIMDSt1<NAME, "Three", asm, "VecListThree", 48, 24, 0b0110>;
defm Four : BaseSIMDSt1<NAME, "Four", asm, "VecListFour", 64, 32, 0b0010>;
}
multiclass SIMDLd2Multiple<string asm> {
defm Two : BaseSIMDLdN<"Two", asm, "VecListTwo", 32, 16, 0b1000>;
defm Two : BaseSIMDLdN<NAME, "Two", asm, "VecListTwo", 32, 16, 0b1000>;
}
multiclass SIMDSt2Multiple<string asm> {
defm Two : BaseSIMDStN<"Two", asm, "VecListTwo", 32, 16, 0b1000>;
defm Two : BaseSIMDStN<NAME, "Two", asm, "VecListTwo", 32, 16, 0b1000>;
}
multiclass SIMDLd3Multiple<string asm> {
defm Three : BaseSIMDLdN<"Three", asm, "VecListThree", 48, 24, 0b0100>;
defm Three : BaseSIMDLdN<NAME, "Three", asm, "VecListThree", 48, 24, 0b0100>;
}
multiclass SIMDSt3Multiple<string asm> {
defm Three : BaseSIMDStN<"Three", asm, "VecListThree", 48, 24, 0b0100>;
defm Three : BaseSIMDStN<NAME, "Three", asm, "VecListThree", 48, 24, 0b0100>;
}
multiclass SIMDLd4Multiple<string asm> {
defm Four : BaseSIMDLdN<"Four", asm, "VecListFour", 64, 32, 0b0000>;
defm Four : BaseSIMDLdN<NAME, "Four", asm, "VecListFour", 64, 32, 0b0000>;
}
multiclass SIMDSt4Multiple<string asm> {
defm Four : BaseSIMDStN<"Four", asm, "VecListFour", 64, 32, 0b0000>;
defm Four : BaseSIMDStN<NAME, "Four", asm, "VecListFour", 64, 32, 0b0000>;
}
//---
@ -8981,14 +8981,14 @@ class BaseSIMDLdRPost<bit Q, bit R, bits<3> opcode, bit S, bits<2> size,
let Inst{11-10} = size;
}
multiclass SIMDLdrAliases<string asm, string layout, string Count,
multiclass SIMDLdrAliases<string BaseName, string asm, string layout, string Count,
int Offset, int Size> {
// E.g. "ld1r { v0.8b }, [x1], #1"
// "ld1r.8b\t$Vt, [$Rn], #1"
// may get mapped to
// (LD1Rv8b_POST VecListOne8b:$Vt, GPR64sp:$Rn, XZR)
def : InstAlias<asm # "\t$Vt, [$Rn], #" # Offset,
(!cast<Instruction>(NAME # "v" # layout # "_POST")
(!cast<Instruction>(BaseName # "v" # layout # "_POST")
GPR64sp:$Rn,
!cast<RegisterOperand>("VecList" # Count # layout):$Vt,
XZR), 1>;
@ -8998,7 +8998,7 @@ multiclass SIMDLdrAliases<string asm, string layout, string Count,
// may get mapped to
// (LD1Rv8b_POST VecListOne64:$Vt, GPR64sp:$Rn, XZR)
def : InstAlias<asm # "." # layout # "\t$Vt, [$Rn], #" # Offset,
(!cast<Instruction>(NAME # "v" # layout # "_POST")
(!cast<Instruction>(BaseName # "v" # layout # "_POST")
GPR64sp:$Rn,
!cast<RegisterOperand>("VecList" # Count # Size):$Vt,
XZR), 0>;
@ -9008,7 +9008,7 @@ multiclass SIMDLdrAliases<string asm, string layout, string Count,
// may get mapped to
// (LD1Rv8b VecListOne64:$Vt, GPR64sp:$Rn)
def : InstAlias<asm # "." # layout # "\t$Vt, [$Rn]",
(!cast<Instruction>(NAME # "v" # layout)
(!cast<Instruction>(BaseName # "v" # layout)
!cast<RegisterOperand>("VecList" # Count # Size):$Vt,
GPR64sp:$Rn), 0>;
@ -9017,7 +9017,7 @@ multiclass SIMDLdrAliases<string asm, string layout, string Count,
// may get mapped to
// (LD1Rv8b_POST VecListOne64:$Vt, GPR64sp:$Rn, GPR64pi1:$Xm)
def : InstAlias<asm # "." # layout # "\t$Vt, [$Rn], $Xm",
(!cast<Instruction>(NAME # "v" # layout # "_POST")
(!cast<Instruction>(BaseName # "v" # layout # "_POST")
GPR64sp:$Rn,
!cast<RegisterOperand>("VecList" # Count # Size):$Vt,
!cast<RegisterOperand>("GPR64pi" # Offset):$Xm), 0>;
@ -9067,14 +9067,14 @@ multiclass SIMDLdR<bit R, bits<3> opcode, bit S, string asm, string Count,
!cast<DAGOperand>("VecList" # Count # "2d"),
!cast<DAGOperand>("GPR64pi" # Offset8)>;
defm : SIMDLdrAliases<asm, "8b", Count, Offset1, 64>;
defm : SIMDLdrAliases<asm, "16b", Count, Offset1, 128>;
defm : SIMDLdrAliases<asm, "4h", Count, Offset2, 64>;
defm : SIMDLdrAliases<asm, "8h", Count, Offset2, 128>;
defm : SIMDLdrAliases<asm, "2s", Count, Offset4, 64>;
defm : SIMDLdrAliases<asm, "4s", Count, Offset4, 128>;
defm : SIMDLdrAliases<asm, "1d", Count, Offset8, 64>;
defm : SIMDLdrAliases<asm, "2d", Count, Offset8, 128>;
defm : SIMDLdrAliases<NAME, asm, "8b", Count, Offset1, 64>;
defm : SIMDLdrAliases<NAME, asm, "16b", Count, Offset1, 128>;
defm : SIMDLdrAliases<NAME, asm, "4h", Count, Offset2, 64>;
defm : SIMDLdrAliases<NAME, asm, "8h", Count, Offset2, 128>;
defm : SIMDLdrAliases<NAME, asm, "2s", Count, Offset4, 64>;
defm : SIMDLdrAliases<NAME, asm, "4s", Count, Offset4, 128>;
defm : SIMDLdrAliases<NAME, asm, "1d", Count, Offset8, 64>;
defm : SIMDLdrAliases<NAME, asm, "2d", Count, Offset8, 128>;
}
class SIMDLdStSingleB<bit L, bit R, bits<3> opcode, string asm,
@ -9432,31 +9432,31 @@ multiclass SIMDLdStSingleAliases<string asm, string layout, string Type,
}
multiclass SIMDLdSt1SingleAliases<string asm> {
defm : SIMDLdStSingleAliases<asm, "b", "i8", "One", 1, VectorIndexB>;
defm : SIMDLdStSingleAliases<asm, "h", "i16", "One", 2, VectorIndexH>;
defm : SIMDLdStSingleAliases<asm, "s", "i32", "One", 4, VectorIndexS>;
defm : SIMDLdStSingleAliases<asm, "d", "i64", "One", 8, VectorIndexD>;
defm "" : SIMDLdStSingleAliases<asm, "b", "i8", "One", 1, VectorIndexB>;
defm "" : SIMDLdStSingleAliases<asm, "h", "i16", "One", 2, VectorIndexH>;
defm "" : SIMDLdStSingleAliases<asm, "s", "i32", "One", 4, VectorIndexS>;
defm "" : SIMDLdStSingleAliases<asm, "d", "i64", "One", 8, VectorIndexD>;
}
multiclass SIMDLdSt2SingleAliases<string asm> {
defm : SIMDLdStSingleAliases<asm, "b", "i8", "Two", 2, VectorIndexB>;
defm : SIMDLdStSingleAliases<asm, "h", "i16", "Two", 4, VectorIndexH>;
defm : SIMDLdStSingleAliases<asm, "s", "i32", "Two", 8, VectorIndexS>;
defm : SIMDLdStSingleAliases<asm, "d", "i64", "Two", 16, VectorIndexD>;
defm "" : SIMDLdStSingleAliases<asm, "b", "i8", "Two", 2, VectorIndexB>;
defm "" : SIMDLdStSingleAliases<asm, "h", "i16", "Two", 4, VectorIndexH>;
defm "" : SIMDLdStSingleAliases<asm, "s", "i32", "Two", 8, VectorIndexS>;
defm "" : SIMDLdStSingleAliases<asm, "d", "i64", "Two", 16, VectorIndexD>;
}
multiclass SIMDLdSt3SingleAliases<string asm> {
defm : SIMDLdStSingleAliases<asm, "b", "i8", "Three", 3, VectorIndexB>;
defm : SIMDLdStSingleAliases<asm, "h", "i16", "Three", 6, VectorIndexH>;
defm : SIMDLdStSingleAliases<asm, "s", "i32", "Three", 12, VectorIndexS>;
defm : SIMDLdStSingleAliases<asm, "d", "i64", "Three", 24, VectorIndexD>;
defm "" : SIMDLdStSingleAliases<asm, "b", "i8", "Three", 3, VectorIndexB>;
defm "" : SIMDLdStSingleAliases<asm, "h", "i16", "Three", 6, VectorIndexH>;
defm "" : SIMDLdStSingleAliases<asm, "s", "i32", "Three", 12, VectorIndexS>;
defm "" : SIMDLdStSingleAliases<asm, "d", "i64", "Three", 24, VectorIndexD>;
}
multiclass SIMDLdSt4SingleAliases<string asm> {
defm : SIMDLdStSingleAliases<asm, "b", "i8", "Four", 4, VectorIndexB>;
defm : SIMDLdStSingleAliases<asm, "h", "i16", "Four", 8, VectorIndexH>;
defm : SIMDLdStSingleAliases<asm, "s", "i32", "Four", 16, VectorIndexS>;
defm : SIMDLdStSingleAliases<asm, "d", "i64", "Four", 32, VectorIndexD>;
defm "" : SIMDLdStSingleAliases<asm, "b", "i8", "Four", 4, VectorIndexB>;
defm "" : SIMDLdStSingleAliases<asm, "h", "i16", "Four", 8, VectorIndexH>;
defm "" : SIMDLdStSingleAliases<asm, "s", "i32", "Four", 16, VectorIndexS>;
defm "" : SIMDLdStSingleAliases<asm, "d", "i64", "Four", 32, VectorIndexD>;
}
} // end of 'let Predicates = [HasNEON]'

72
lib/Target/AMDGPU/BUFInstructions.td
View File

@ -52,19 +52,19 @@ class getAddrName<int addrKind> {
"")))));
}
class MUBUFAddr64Table <bit is_addr64, string suffix = ""> {
class MUBUFAddr64Table <bit is_addr64, string Name> {
bit IsAddr64 = is_addr64;
string OpName = NAME # suffix;
string OpName = Name;
}
class MUBUFLdsTable <bit is_lds, string suffix> {
class MUBUFLdsTable <bit is_lds, string Name> {
bit IsLds = is_lds;
string OpName = NAME # suffix;
string OpName = Name;
}
class MTBUFAddr64Table <bit is_addr64, string suffix = ""> {
class MTBUFAddr64Table <bit is_addr64, string Name> {
bit IsAddr64 = is_addr64;
string OpName = NAME # suffix;
string OpName = Name;
}
//===----------------------------------------------------------------------===//
@ -219,13 +219,13 @@ multiclass MTBUF_Pseudo_Loads<string opName, RegisterClass vdataClass,
[(set load_vt:$vdata,
(ld (MUBUFOffset v4i32:$srsrc, i32:$soffset, i16:$offset, i8:$dfmt,
i8:$nfmt, i1:$glc, i1:$slc, i1:$tfe)))]>,
MTBUFAddr64Table<0>;
MTBUFAddr64Table<0, NAME>;
def _ADDR64 : MTBUF_Load_Pseudo <opName, BUFAddrKind.Addr64, vdataClass,
[(set load_vt:$vdata,
(ld (MUBUFAddr64 v4i32:$srsrc, i64:$vaddr, i32:$soffset, i16:$offset,
i8:$dfmt, i8:$nfmt, i1:$glc, i1:$slc, i1:$tfe)))]>,
MTBUFAddr64Table<1>;
MTBUFAddr64Table<1, NAME>;
def _OFFEN : MTBUF_Load_Pseudo <opName, BUFAddrKind.OffEn, vdataClass>;
def _IDXEN : MTBUF_Load_Pseudo <opName, BUFAddrKind.IdxEn, vdataClass>;
@ -265,13 +265,13 @@ multiclass MTBUF_Pseudo_Stores<string opName, RegisterClass vdataClass,
[(st store_vt:$vdata, (MUBUFOffset v4i32:$srsrc, i32:$soffset,
i16:$offset, i8:$dfmt, i8:$nfmt, i1:$glc,
i1:$slc, i1:$tfe))]>,
MTBUFAddr64Table<0>;
MTBUFAddr64Table<0, NAME>;
def _ADDR64 : MTBUF_Store_Pseudo <opName, BUFAddrKind.Addr64, vdataClass,
[(st store_vt:$vdata, (MUBUFAddr64 v4i32:$srsrc, i64:$vaddr, i32:$soffset,
i16:$offset, i8:$dfmt, i8:$nfmt, i1:$glc,
i1:$slc, i1:$tfe))]>,
MTBUFAddr64Table<1>;
MTBUFAddr64Table<1, NAME>;
def _OFFEN : MTBUF_Store_Pseudo <opName, BUFAddrKind.OffEn, vdataClass>;
def _IDXEN : MTBUF_Store_Pseudo <opName, BUFAddrKind.IdxEn, vdataClass>;
@ -474,7 +474,7 @@ multiclass MUBUF_Pseudo_Loads<string opName, RegisterClass vdataClass,
[],
[(set load_vt:$vdata,
(ld (MUBUFOffset v4i32:$srsrc, i32:$soffset, i16:$offset, i1:$glc, i1:$slc, i1:$tfe)))])>,
MUBUFAddr64Table<0, !if(isLds, "_LDS", "")>;
MUBUFAddr64Table<0, NAME # !if(isLds, "_LDS", "")>;
def _ADDR64 : MUBUF_Load_Pseudo <opName, BUFAddrKind.Addr64, vdataClass,
TiedDest, isLds,
@ -482,7 +482,7 @@ multiclass MUBUF_Pseudo_Loads<string opName, RegisterClass vdataClass,
[],
[(set load_vt:$vdata,
(ld (MUBUFAddr64 v4i32:$srsrc, i64:$vaddr, i32:$soffset, i16:$offset, i1:$glc, i1:$slc, i1:$tfe)))])>,
MUBUFAddr64Table<1, !if(isLds, "_LDS", "")>;
MUBUFAddr64Table<1, NAME # !if(isLds, "_LDS", "")>;
def _OFFEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.OffEn, vdataClass, TiedDest, isLds>;
def _IDXEN : MUBUF_Load_Pseudo <opName, BUFAddrKind.IdxEn, vdataClass, TiedDest, isLds>;
@ -530,12 +530,12 @@ multiclass MUBUF_Pseudo_Stores<string opName, RegisterClass vdataClass,
def _OFFSET : MUBUF_Store_Pseudo <opName, BUFAddrKind.Offset, vdataClass,
[(st store_vt:$vdata, (MUBUFOffset v4i32:$srsrc, i32:$soffset,
i16:$offset, i1:$glc, i1:$slc, i1:$tfe))]>,
MUBUFAddr64Table<0>;
MUBUFAddr64Table<0, NAME>;
def _ADDR64 : MUBUF_Store_Pseudo <opName, BUFAddrKind.Addr64, vdataClass,
[(st store_vt:$vdata, (MUBUFAddr64 v4i32:$srsrc, i64:$vaddr, i32:$soffset,
i16:$offset, i1:$glc, i1:$slc, i1:$tfe))]>,
MUBUFAddr64Table<1>;
MUBUFAddr64Table<1, NAME>;
def _OFFEN : MUBUF_Store_Pseudo <opName, BUFAddrKind.OffEn, vdataClass>;
def _IDXEN : MUBUF_Store_Pseudo <opName, BUFAddrKind.IdxEn, vdataClass>;
@ -666,9 +666,9 @@ multiclass MUBUF_Pseudo_Atomics <string opName,
SDPatternOperator atomic> {
def _OFFSET : MUBUF_AtomicNoRet_Pseudo <opName, BUFAddrKind.Offset, vdataClass>,
MUBUFAddr64Table <0>;
MUBUFAddr64Table <0, NAME>;
def _ADDR64 : MUBUF_AtomicNoRet_Pseudo <opName, BUFAddrKind.Addr64, vdataClass>,
MUBUFAddr64Table <1>;
MUBUFAddr64Table <1, NAME>;
def _OFFEN : MUBUF_AtomicNoRet_Pseudo <opName, BUFAddrKind.OffEn, vdataClass>;
def _IDXEN : MUBUF_AtomicNoRet_Pseudo <opName, BUFAddrKind.IdxEn, vdataClass>;
def _BOTHEN : MUBUF_AtomicNoRet_Pseudo <opName, BUFAddrKind.BothEn, vdataClass>;
@ -677,13 +677,13 @@ multiclass MUBUF_Pseudo_Atomics <string opName,
[(set vdataType:$vdata,
(atomic (MUBUFOffsetAtomic v4i32:$srsrc, i32:$soffset, i16:$offset, i1:$slc),
vdataType:$vdata_in))]>,
MUBUFAddr64Table <0, "_RTN">;
MUBUFAddr64Table <0, NAME # "_RTN">;
def _ADDR64_RTN : MUBUF_AtomicRet_Pseudo <opName, BUFAddrKind.Addr64, vdataClass,
[(set vdataType:$vdata,
(atomic (MUBUFAddr64Atomic v4i32:$srsrc, i64:$vaddr, i32:$soffset, i16:$offset, i1:$slc),
vdataType:$vdata_in))]>,
MUBUFAddr64Table <1, "_RTN">;
MUBUFAddr64Table <1, NAME # "_RTN">;
def _OFFEN_RTN : MUBUF_AtomicRet_Pseudo <opName, BUFAddrKind.OffEn, vdataClass>;
def _IDXEN_RTN : MUBUF_AtomicRet_Pseudo <opName, BUFAddrKind.IdxEn, vdataClass>;
@ -1663,26 +1663,26 @@ multiclass MUBUF_Real_AllAddr_si<bits<7> op> {
multiclass MUBUF_Real_AllAddr_Lds_si<bits<7> op> {
def _OFFSET_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>,
MUBUFLdsTable<0, "_OFFSET_si">;
MUBUFLdsTable<0, NAME # "_OFFSET_si">;
def _ADDR64_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_ADDR64")>,
MUBUFLdsTable<0, "_ADDR64_si">;
MUBUFLdsTable<0, NAME # "_ADDR64_si">;
def _OFFEN_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>,
MUBUFLdsTable<0, "_OFFEN_si">;
MUBUFLdsTable<0, NAME # "_OFFEN_si">;
def _IDXEN_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>,
MUBUFLdsTable<0, "_IDXEN_si">;
MUBUFLdsTable<0, NAME # "_IDXEN_si">;
def _BOTHEN_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>,
MUBUFLdsTable<0, "_BOTHEN_si">;
MUBUFLdsTable<0, NAME # "_BOTHEN_si">;
def _LDS_OFFSET_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>,
MUBUFLdsTable<1, "_OFFSET_si">;
MUBUFLdsTable<1, NAME # "_OFFSET_si">;
def _LDS_ADDR64_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_ADDR64")>,
MUBUFLdsTable<1, "_ADDR64_si">;
MUBUFLdsTable<1, NAME # "_ADDR64_si">;
def _LDS_OFFEN_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>,
MUBUFLdsTable<1, "_OFFEN_si">;
MUBUFLdsTable<1, NAME # "_OFFEN_si">;
def _LDS_IDXEN_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>,
MUBUFLdsTable<1, "_IDXEN_si">;
MUBUFLdsTable<1, NAME # "_IDXEN_si">;
def _LDS_BOTHEN_si : MUBUF_Real_si <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>,
MUBUFLdsTable<1, "_BOTHEN_si">;
MUBUFLdsTable<1, NAME # "_BOTHEN_si">;
}
multiclass MUBUF_Real_Atomic_si<bits<7> op> : MUBUF_Real_AllAddr_si<op> {
@ -1846,22 +1846,22 @@ multiclass MUBUF_Real_AllAddr_vi<bits<7> op> {
multiclass MUBUF_Real_AllAddr_Lds_vi<bits<7> op> {
def _OFFSET_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_OFFSET")>,
MUBUFLdsTable<0, "_OFFSET_vi">;
MUBUFLdsTable<0, NAME # "_OFFSET_vi">;
def _OFFEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_OFFEN")>,
MUBUFLdsTable<0, "_OFFEN_vi">;
MUBUFLdsTable<0, NAME # "_OFFEN_vi">;
def _IDXEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_IDXEN")>,
MUBUFLdsTable<0, "_IDXEN_vi">;
MUBUFLdsTable<0, NAME # "_IDXEN_vi">;
def _BOTHEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_BOTHEN")>,
MUBUFLdsTable<0, "_BOTHEN_vi">;
MUBUFLdsTable<0, NAME # "_BOTHEN_vi">;
def _LDS_OFFSET_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFSET")>,
MUBUFLdsTable<1, "_OFFSET_vi">;
MUBUFLdsTable<1, NAME # "_OFFSET_vi">;
def _LDS_OFFEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_OFFEN")>,
MUBUFLdsTable<1, "_OFFEN_vi">;
MUBUFLdsTable<1, NAME # "_OFFEN_vi">;
def _LDS_IDXEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_IDXEN")>,
MUBUFLdsTable<1, "_IDXEN_vi">;
MUBUFLdsTable<1, NAME # "_IDXEN_vi">;
def _LDS_BOTHEN_vi : MUBUF_Real_vi <op, !cast<MUBUF_Pseudo>(NAME#"_LDS_BOTHEN")>,
MUBUFLdsTable<1, "_BOTHEN_vi">;
MUBUFLdsTable<1, NAME # "_BOTHEN_vi">;
}
class MUBUF_Real_gfx80 <bits<7> op, MUBUF_Pseudo ps> :

12
lib/Target/AMDGPU/MIMGInstructions.td
View File

@ -280,15 +280,15 @@ multiclass MIMG_Sampler_Src_Helper_Helper <bits<7> op, string asm,
multiclass MIMG_Sampler_Src_Helper <bits<7> op, string asm,
RegisterClass dst_rc,
int channels, bit wqm> {
defm : MIMG_Sampler_Src_Helper_Helper <op, asm, dst_rc, channels, wqm, 0, "">;
defm "" : MIMG_Sampler_Src_Helper_Helper <op, asm, dst_rc, channels, wqm, 0, "">;
let d16 = 1 in {
let SubtargetPredicate = HasPackedD16VMem in {
defm : MIMG_Sampler_Src_Helper_Helper <op, asm, dst_rc, channels, wqm, 1, "_D16">;
defm "" : MIMG_Sampler_Src_Helper_Helper <op, asm, dst_rc, channels, wqm, 1, "_D16">;
} // End HasPackedD16VMem.
let SubtargetPredicate = HasUnpackedD16VMem, DecoderNamespace = "GFX80_UNPACKED" in {
defm : MIMG_Sampler_Src_Helper_Helper <op, asm, dst_rc, channels, wqm, 1, "_D16_gfx80">;
defm "" : MIMG_Sampler_Src_Helper_Helper <op, asm, dst_rc, channels, wqm, 1, "_D16_gfx80">;
} // End HasUnpackedD16VMem.
} // End d16 = 1.
}
@ -348,15 +348,15 @@ multiclass MIMG_Gather_Src_Helper <bits<7> op, string asm,
}
multiclass MIMG_Gather <bits<7> op, string asm, bit wqm=0> {
defm : MIMG_Gather_Src_Helper<op, asm, VReg_128, wqm, 0, "_V4", "">;
defm "" : MIMG_Gather_Src_Helper<op, asm, VReg_128, wqm, 0, "_V4", "">;
let d16 = 1 in {
let AssemblerPredicate = HasPackedD16VMem in {
defm : MIMG_Gather_Src_Helper<op, asm, VReg_64, wqm, 1, "_V2", "_D16">;
defm "" : MIMG_Gather_Src_Helper<op, asm, VReg_64, wqm, 1, "_V2", "_D16">;
} // End HasPackedD16VMem.
let AssemblerPredicate = HasUnpackedD16VMem, DecoderNamespace = "GFX80_UNPACKED" in {
defm : MIMG_Gather_Src_Helper<op, asm, VReg_128, wqm, 1, "_V4", "_D16_gfx80">;
defm "" : MIMG_Gather_Src_Helper<op, asm, VReg_128, wqm, 1, "_V4", "_D16_gfx80">;
} // End HasUnpackedD16VMem.
} // End d16 = 1.
}

329
lib/Target/X86/X86InstrAVX512.td
View File

@ -1130,26 +1130,28 @@ def VEXTRACTPSZmr : AVX512AIi8<0x17, MRMDestMem, (outs),
//---
// broadcast with a scalar argument.
multiclass avx512_broadcast_scalar<bits<8> opc, string OpcodeStr,
string Name,
X86VectorVTInfo DestInfo, X86VectorVTInfo SrcInfo> {
def : Pat<(DestInfo.VT (X86VBroadcast SrcInfo.FRC:$src)),
(!cast<Instruction>(NAME#DestInfo.ZSuffix#r)
(!cast<Instruction>(Name#DestInfo.ZSuffix#r)
(COPY_TO_REGCLASS SrcInfo.FRC:$src, SrcInfo.RC))>;
def : Pat<(DestInfo.VT (vselect DestInfo.KRCWM:$mask,
(X86VBroadcast SrcInfo.FRC:$src),
DestInfo.RC:$src0)),
(!cast<Instruction>(NAME#DestInfo.ZSuffix#rk)
(!cast<Instruction>(Name#DestInfo.ZSuffix#rk)
DestInfo.RC:$src0, DestInfo.KRCWM:$mask,
(COPY_TO_REGCLASS SrcInfo.FRC:$src, SrcInfo.RC))>;
def : Pat<(DestInfo.VT (vselect DestInfo.KRCWM:$mask,
(X86VBroadcast SrcInfo.FRC:$src),
DestInfo.ImmAllZerosV)),
(!cast<Instruction>(NAME#DestInfo.ZSuffix#rkz)
(!cast<Instruction>(Name#DestInfo.ZSuffix#rkz)
DestInfo.KRCWM:$mask, (COPY_TO_REGCLASS SrcInfo.FRC:$src, SrcInfo.RC))>;
}
// Split version to allow mask and broadcast node to be different types. This
// helps support the 32x2 broadcasts.
multiclass avx512_broadcast_rm_split<bits<8> opc, string OpcodeStr,
string Name,
SchedWrite SchedRR, SchedWrite SchedRM,
X86VectorVTInfo MaskInfo,
X86VectorVTInfo DestInfo,
@ -1189,7 +1191,7 @@ multiclass avx512_broadcast_rm_split<bits<8> opc, string OpcodeStr,
(DestInfo.VT (UnmaskedOp
(SrcInfo.VT (scalar_to_vector
(SrcInfo.ScalarLdFrag addr:$src))))))),
(!cast<Instruction>(NAME#MaskInfo.ZSuffix#m) addr:$src)>;
(!cast<Instruction>(Name#MaskInfo.ZSuffix#m) addr:$src)>;
def : Pat<(MaskInfo.VT (vselect MaskInfo.KRCWM:$mask,
(bitconvert
(DestInfo.VT
@ -1197,7 +1199,7 @@ multiclass avx512_broadcast_rm_split<bits<8> opc, string OpcodeStr,
(SrcInfo.VT (scalar_to_vector
(SrcInfo.ScalarLdFrag addr:$src)))))),
MaskInfo.RC:$src0)),
(!cast<Instruction>(NAME#DestInfo.ZSuffix#mk)
(!cast<Instruction>(Name#DestInfo.ZSuffix#mk)
MaskInfo.RC:$src0, MaskInfo.KRCWM:$mask, addr:$src)>;
def : Pat<(MaskInfo.VT (vselect MaskInfo.KRCWM:$mask,
(bitconvert
@ -1206,53 +1208,58 @@ multiclass avx512_broadcast_rm_split<bits<8> opc, string OpcodeStr,
(SrcInfo.VT (scalar_to_vector
(SrcInfo.ScalarLdFrag addr:$src)))))),
MaskInfo.ImmAllZerosV)),
(!cast<Instruction>(NAME#MaskInfo.ZSuffix#mkz)
(!cast<Instruction>(Name#MaskInfo.ZSuffix#mkz)
MaskInfo.KRCWM:$mask, addr:$src)>;
}
// Helper class to force mask and broadcast result to same type.
multiclass avx512_broadcast_rm<bits<8> opc, string OpcodeStr,
multiclass avx512_broadcast_rm<bits<8> opc, string OpcodeStr, string Name,
SchedWrite SchedRR, SchedWrite SchedRM,
X86VectorVTInfo DestInfo,
X86VectorVTInfo SrcInfo> :
avx512_broadcast_rm_split<opc, OpcodeStr, SchedRR, SchedRM,
avx512_broadcast_rm_split<opc, OpcodeStr, Name, SchedRR, SchedRM,
DestInfo, DestInfo, SrcInfo>;
multiclass avx512_fp_broadcast_sd<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in {
defm Z : avx512_broadcast_rm<opc, OpcodeStr, WriteFShuffle256,
defm Z : avx512_broadcast_rm<opc, OpcodeStr, NAME, WriteFShuffle256,
WriteFShuffle256Ld, _.info512, _.info128>,
avx512_broadcast_scalar<opc, OpcodeStr, _.info512, _.info128>,
EVEX_V512;
avx512_broadcast_scalar<opc, OpcodeStr, NAME, _.info512,
_.info128>,
EVEX_V512;
}
let Predicates = [HasVLX] in {
defm Z256 : avx512_broadcast_rm<opc, OpcodeStr, WriteFShuffle256,
defm Z256 : avx512_broadcast_rm<opc, OpcodeStr, NAME, WriteFShuffle256,
WriteFShuffle256Ld, _.info256, _.info128>,
avx512_broadcast_scalar<opc, OpcodeStr, _.info256, _.info128>,
EVEX_V256;
avx512_broadcast_scalar<opc, OpcodeStr, NAME, _.info256,
_.info128>,
EVEX_V256;
}
}
multiclass avx512_fp_broadcast_ss<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in {
defm Z : avx512_broadcast_rm<opc, OpcodeStr, WriteFShuffle256,
defm Z : avx512_broadcast_rm<opc, OpcodeStr, NAME, WriteFShuffle256,
WriteFShuffle256Ld, _.info512, _.info128>,
avx512_broadcast_scalar<opc, OpcodeStr, _.info512, _.info128>,
EVEX_V512;
avx512_broadcast_scalar<opc, OpcodeStr, NAME, _.info512,
_.info128>,
EVEX_V512;
}
let Predicates = [HasVLX] in {
defm Z256 : avx512_broadcast_rm<opc, OpcodeStr, WriteFShuffle256,
defm Z256 : avx512_broadcast_rm<opc, OpcodeStr, NAME, WriteFShuffle256,
WriteFShuffle256Ld, _.info256, _.info128>,
avx512_broadcast_scalar<opc, OpcodeStr, _.info256, _.info128>,
EVEX_V256;
defm Z128 : avx512_broadcast_rm<opc, OpcodeStr, WriteFShuffle256,
avx512_broadcast_scalar<opc, OpcodeStr, NAME, _.info256,
_.info128>,
EVEX_V256;
defm Z128 : avx512_broadcast_rm<opc, OpcodeStr, NAME, WriteFShuffle256,
WriteFShuffle256Ld, _.info128, _.info128>,
avx512_broadcast_scalar<opc, OpcodeStr, _.info128, _.info128>,
EVEX_V128;
avx512_broadcast_scalar<opc, OpcodeStr, NAME, _.info128,
_.info128>,
EVEX_V128;
}
}
defm VBROADCASTSS : avx512_fp_broadcast_ss<0x18, "vbroadcastss",
@ -1335,29 +1342,30 @@ defm VPBROADCASTQr : avx512_int_broadcast_reg_vl<0x7C, avx512vl_i64_info,
// Provide aliases for broadcast from the same register class that
// automatically does the extract.
multiclass avx512_int_broadcast_rm_lowering<X86VectorVTInfo DestInfo,
multiclass avx512_int_broadcast_rm_lowering<string Name,
X86VectorVTInfo DestInfo,
X86VectorVTInfo SrcInfo> {
def : Pat<(DestInfo.VT (X86VBroadcast (SrcInfo.VT SrcInfo.RC:$src))),
(!cast<Instruction>(NAME#DestInfo.ZSuffix#"r")
(!cast<Instruction>(Name#DestInfo.ZSuffix#"r")
(EXTRACT_SUBREG (SrcInfo.VT SrcInfo.RC:$src), sub_xmm))>;
}
multiclass avx512_int_broadcast_rm_vl<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _, Predicate prd> {
let Predicates = [prd] in {
defm Z : avx512_broadcast_rm<opc, OpcodeStr, WriteShuffle256,
defm Z : avx512_broadcast_rm<opc, OpcodeStr, NAME, WriteShuffle256,
WriteShuffle256Ld, _.info512, _.info128>,
avx512_int_broadcast_rm_lowering<_.info512, _.info256>,
avx512_int_broadcast_rm_lowering<NAME, _.info512, _.info256>,
EVEX_V512;
// Defined separately to avoid redefinition.
defm Z_Alt : avx512_int_broadcast_rm_lowering<_.info512, _.info512>;
defm Z_Alt : avx512_int_broadcast_rm_lowering<NAME, _.info512, _.info512>;
}
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_broadcast_rm<opc, OpcodeStr, WriteShuffle256,
defm Z256 : avx512_broadcast_rm<opc, OpcodeStr, NAME, WriteShuffle256,
WriteShuffle256Ld, _.info256, _.info128>,
avx512_int_broadcast_rm_lowering<_.info256, _.info256>,
avx512_int_broadcast_rm_lowering<NAME, _.info256, _.info256>,
EVEX_V256;
defm Z128 : avx512_broadcast_rm<opc, OpcodeStr, WriteShuffle,
defm Z128 : avx512_broadcast_rm<opc, OpcodeStr, NAME, WriteShuffle,
WriteShuffleXLd, _.info128, _.info128>,
EVEX_V128;
}
@ -1656,12 +1664,12 @@ def : Pat<(vselect VK8WM:$mask,
multiclass avx512_common_broadcast_32x2<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _Dst, AVX512VLVectorVTInfo _Src> {
let Predicates = [HasDQI] in
defm Z : avx512_broadcast_rm_split<opc, OpcodeStr, WriteShuffle256,
defm Z : avx512_broadcast_rm_split<opc, OpcodeStr, NAME, WriteShuffle256,
WriteShuffle256Ld, _Dst.info512,
_Src.info512, _Src.info128, null_frag>,
EVEX_V512;
let Predicates = [HasDQI, HasVLX] in
defm Z256 : avx512_broadcast_rm_split<opc, OpcodeStr, WriteShuffle256,
defm Z256 : avx512_broadcast_rm_split<opc, OpcodeStr, NAME, WriteShuffle256,
WriteShuffle256Ld, _Dst.info256,
_Src.info256, _Src.info128, null_frag>,
EVEX_V256;
@ -1672,7 +1680,7 @@ multiclass avx512_common_broadcast_i32x2<bits<8> opc, string OpcodeStr,
avx512_common_broadcast_32x2<opc, OpcodeStr, _Dst, _Src> {
let Predicates = [HasDQI, HasVLX] in
defm Z128 : avx512_broadcast_rm_split<opc, OpcodeStr, WriteShuffle,
defm Z128 : avx512_broadcast_rm_split<opc, OpcodeStr, NAME, WriteShuffle,
WriteShuffleXLd, _Dst.info128,
_Src.info128, _Src.info128, null_frag>,
EVEX_V128;
@ -2279,7 +2287,8 @@ def CommutePCMPCC : SDNodeXForm<imm, [{
}]>;
multiclass avx512_icmp_cc<bits<8> opc, string Suffix, SDNode OpNode,
X86FoldableSchedWrite sched, X86VectorVTInfo _> {
X86FoldableSchedWrite sched, X86VectorVTInfo _,
string Name> {
let isCommutable = 1 in
def rri : AVX512AIi8<opc, MRMSrcReg,
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2, AVX512ICC:$cc),
@ -2351,19 +2360,20 @@ multiclass avx512_icmp_cc<bits<8> opc, string Suffix, SDNode OpNode,
def : Pat<(OpNode (bitconvert (_.LdFrag addr:$src2)),
(_.VT _.RC:$src1), imm:$cc),
(!cast<Instruction>(NAME#_.ZSuffix#"rmi") _.RC:$src1, addr:$src2,
(!cast<Instruction>(Name#_.ZSuffix#"rmi") _.RC:$src1, addr:$src2,
(CommutePCMPCC imm:$cc))>;
def : Pat<(and _.KRCWM:$mask, (OpNode (bitconvert (_.LdFrag addr:$src2)),
(_.VT _.RC:$src1), imm:$cc)),
(!cast<Instruction>(NAME#_.ZSuffix#"rmik") _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#"rmik") _.KRCWM:$mask,
_.RC:$src1, addr:$src2,
(CommutePCMPCC imm:$cc))>;
}
multiclass avx512_icmp_cc_rmb<bits<8> opc, string Suffix, SDNode OpNode,
X86FoldableSchedWrite sched, X86VectorVTInfo _> :
avx512_icmp_cc<opc, Suffix, OpNode, sched, _> {
X86FoldableSchedWrite sched, X86VectorVTInfo _,
string Name> :
avx512_icmp_cc<opc, Suffix, OpNode, sched, _, Name> {
def rmib : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.RC:$src1, _.ScalarMemOp:$src2,
AVX512ICC:$cc),
@ -2406,13 +2416,13 @@ multiclass avx512_icmp_cc_rmb<bits<8> opc, string Suffix, SDNode OpNode,
def : Pat<(OpNode (X86VBroadcast (_.ScalarLdFrag addr:$src2)),
(_.VT _.RC:$src1), imm:$cc),
(!cast<Instruction>(NAME#_.ZSuffix#"rmib") _.RC:$src1, addr:$src2,
(!cast<Instruction>(Name#_.ZSuffix#"rmib") _.RC:$src1, addr:$src2,
(CommutePCMPCC imm:$cc))>;
def : Pat<(and _.KRCWM:$mask, (OpNode (X86VBroadcast
(_.ScalarLdFrag addr:$src2)),
(_.VT _.RC:$src1), imm:$cc)),
(!cast<Instruction>(NAME#_.ZSuffix#"rmibk") _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#"rmibk") _.KRCWM:$mask,
_.RC:$src1, addr:$src2,
(CommutePCMPCC imm:$cc))>;
}
@ -2421,14 +2431,16 @@ multiclass avx512_icmp_cc_vl<bits<8> opc, string Suffix, SDNode OpNode,
X86SchedWriteWidths sched,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_cc<opc, Suffix, OpNode, sched.ZMM, VTInfo.info512>,
EVEX_V512;
defm Z : avx512_icmp_cc<opc, Suffix, OpNode, sched.ZMM, VTInfo.info512, NAME>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_cc<opc, Suffix, OpNode, sched.YMM, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_icmp_cc<opc, Suffix, OpNode, sched.XMM, VTInfo.info128>,
EVEX_V128;
defm Z256 : avx512_icmp_cc<opc, Suffix, OpNode, sched.YMM, VTInfo.info256,
NAME>,
EVEX_V256;
defm Z128 : avx512_icmp_cc<opc, Suffix, OpNode, sched.XMM, VTInfo.info128,
NAME>,
EVEX_V128;
}
}
@ -2437,13 +2449,13 @@ multiclass avx512_icmp_cc_rmb_vl<bits<8> opc, string Suffix, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_cc_rmb<opc, Suffix, OpNode, sched.ZMM,
VTInfo.info512>, EVEX_V512;
VTInfo.info512, NAME>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_cc_rmb<opc, Suffix, OpNode, sched.YMM,
VTInfo.info256>, EVEX_V256;
VTInfo.info256, NAME>, EVEX_V256;
defm Z128 : avx512_icmp_cc_rmb<opc, Suffix, OpNode, sched.XMM,
VTInfo.info128>, EVEX_V128;
VTInfo.info128, NAME>, EVEX_V128;
}
}
@ -2474,7 +2486,8 @@ defm VPCMPUQ : avx512_icmp_cc_rmb_vl<0x1E, "uq", X86cmpmu, SchedWriteVecALU,
avx512vl_i64_info, HasAVX512>,
VEX_W, EVEX_CD8<64, CD8VF>;
multiclass avx512_vcmp_common<X86FoldableSchedWrite sched, X86VectorVTInfo _> {
multiclass avx512_vcmp_common<X86FoldableSchedWrite sched, X86VectorVTInfo _,
string Name> {
defm rri : AVX512_maskable_cmp<0xC2, MRMSrcReg, _,
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2,AVXCC:$cc),
"vcmp${cc}"#_.Suffix,
@ -2533,26 +2546,26 @@ multiclass avx512_vcmp_common<X86FoldableSchedWrite sched, X86VectorVTInfo _> {
// Patterns for selecting with loads in other operand.
def : Pat<(X86cmpm (_.LdFrag addr:$src2), (_.VT _.RC:$src1),
CommutableCMPCC:$cc),
(!cast<Instruction>(NAME#_.ZSuffix#"rmi") _.RC:$src1, addr:$src2,
(!cast<Instruction>(Name#_.ZSuffix#"rmi") _.RC:$src1, addr:$src2,
imm:$cc)>;
def : Pat<(and _.KRCWM:$mask, (X86cmpm (_.LdFrag addr:$src2),
(_.VT _.RC:$src1),
CommutableCMPCC:$cc)),
(!cast<Instruction>(NAME#_.ZSuffix#"rmik") _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#"rmik") _.KRCWM:$mask,
_.RC:$src1, addr:$src2,
imm:$cc)>;
def : Pat<(X86cmpm (X86VBroadcast (_.ScalarLdFrag addr:$src2)),
(_.VT _.RC:$src1), CommutableCMPCC:$cc),
(!cast<Instruction>(NAME#_.ZSuffix#"rmbi") _.RC:$src1, addr:$src2,
(!cast<Instruction>(Name#_.ZSuffix#"rmbi") _.RC:$src1, addr:$src2,
imm:$cc)>;
def : Pat<(and _.KRCWM:$mask, (X86cmpm (X86VBroadcast
(_.ScalarLdFrag addr:$src2)),
(_.VT _.RC:$src1),
CommutableCMPCC:$cc)),
(!cast<Instruction>(NAME#_.ZSuffix#"rmbik") _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#"rmbik") _.KRCWM:$mask,
_.RC:$src1, addr:$src2,
imm:$cc)>;
}
@ -2582,13 +2595,13 @@ multiclass avx512_vcmp_sae<X86FoldableSchedWrite sched, X86VectorVTInfo _> {
multiclass avx512_vcmp<X86SchedWriteWidths sched, AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in {
defm Z : avx512_vcmp_common<sched.ZMM, _.info512>,
defm Z : avx512_vcmp_common<sched.ZMM, _.info512, NAME>,
avx512_vcmp_sae<sched.ZMM, _.info512>, EVEX_V512;
}
let Predicates = [HasAVX512,HasVLX] in {
defm Z128 : avx512_vcmp_common<sched.XMM, _.info128>, EVEX_V128;
defm Z256 : avx512_vcmp_common<sched.YMM, _.info256>, EVEX_V256;
defm Z128 : avx512_vcmp_common<sched.XMM, _.info128, NAME>, EVEX_V128;
defm Z256 : avx512_vcmp_common<sched.YMM, _.info256, NAME>, EVEX_V256;
}
}
@ -3236,7 +3249,7 @@ defm : operation_subvector_mask_lowering<VK32, v32i1, VK64, v64i1>;
// AVX-512 - Aligned and unaligned load and store
//
multiclass avx512_load<bits<8> opc, string OpcodeStr,
multiclass avx512_load<bits<8> opc, string OpcodeStr, string Name,
X86VectorVTInfo _, PatFrag ld_frag, PatFrag mload,
X86SchedWriteMoveLS Sched, bit NoRMPattern = 0,
SDPatternOperator SelectOprr = vselect> {
@ -3290,13 +3303,13 @@ multiclass avx512_load<bits<8> opc, string OpcodeStr,
_.ExeDomain>, EVEX, EVEX_KZ, Sched<[Sched.RM]>;
}
def : Pat<(_.VT (mload addr:$ptr, _.KRCWM:$mask, undef)),
(!cast<Instruction>(NAME#_.ZSuffix##rmkz) _.KRCWM:$mask, addr:$ptr)>;
(!cast<Instruction>(Name#_.ZSuffix##rmkz) _.KRCWM:$mask, addr:$ptr)>;
def : Pat<(_.VT (mload addr:$ptr, _.KRCWM:$mask, _.ImmAllZerosV)),
(!cast<Instruction>(NAME#_.ZSuffix##rmkz) _.KRCWM:$mask, addr:$ptr)>;
(!cast<Instruction>(Name#_.ZSuffix##rmkz) _.KRCWM:$mask, addr:$ptr)>;
def : Pat<(_.VT (mload addr:$ptr, _.KRCWM:$mask, (_.VT _.RC:$src0))),
(!cast<Instruction>(NAME#_.ZSuffix##rmk) _.RC:$src0,
(!cast<Instruction>(Name#_.ZSuffix##rmk) _.RC:$src0,
_.KRCWM:$mask, addr:$ptr)>;
}
@ -3305,15 +3318,15 @@ multiclass avx512_alignedload_vl<bits<8> opc, string OpcodeStr,
X86SchedWriteMoveLSWidths Sched,
bit NoRMPattern = 0> {
let Predicates = [prd] in
defm Z : avx512_load<opc, OpcodeStr, _.info512,
defm Z : avx512_load<opc, OpcodeStr, NAME, _.info512,
_.info512.AlignedLdFrag, masked_load_aligned512,
Sched.ZMM, NoRMPattern>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_load<opc, OpcodeStr, _.info256,
defm Z256 : avx512_load<opc, OpcodeStr, NAME, _.info256,
_.info256.AlignedLdFrag, masked_load_aligned256,
Sched.YMM, NoRMPattern>, EVEX_V256;
defm Z128 : avx512_load<opc, OpcodeStr, _.info128,
defm Z128 : avx512_load<opc, OpcodeStr, NAME, _.info128,
_.info128.AlignedLdFrag, masked_load_aligned128,
Sched.XMM, NoRMPattern>, EVEX_V128;
}
@ -3325,21 +3338,21 @@ multiclass avx512_load_vl<bits<8> opc, string OpcodeStr,
bit NoRMPattern = 0,
SDPatternOperator SelectOprr = vselect> {
let Predicates = [prd] in
defm Z : avx512_load<opc, OpcodeStr, _.info512, _.info512.LdFrag,
defm Z : avx512_load<opc, OpcodeStr, NAME, _.info512, _.info512.LdFrag,
masked_load_unaligned, Sched.ZMM, NoRMPattern,
SelectOprr>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_load<opc, OpcodeStr, _.info256, _.info256.LdFrag,
defm Z256 : avx512_load<opc, OpcodeStr, NAME, _.info256, _.info256.LdFrag,
masked_load_unaligned, Sched.YMM, NoRMPattern,
SelectOprr>, EVEX_V256;
defm Z128 : avx512_load<opc, OpcodeStr, _.info128, _.info128.LdFrag,
defm Z128 : avx512_load<opc, OpcodeStr, NAME, _.info128, _.info128.LdFrag,
masked_load_unaligned, Sched.XMM, NoRMPattern,
SelectOprr>, EVEX_V128;
}
}
multiclass avx512_store<bits<8> opc, string OpcodeStr,
multiclass avx512_store<bits<8> opc, string OpcodeStr, string BaseName,
X86VectorVTInfo _, PatFrag st_frag, PatFrag mstore,
string Name, X86SchedWriteMoveLS Sched,
bit NoMRPattern = 0> {
@ -3375,8 +3388,8 @@ multiclass avx512_store<bits<8> opc, string OpcodeStr,
[], _.ExeDomain>, EVEX, EVEX_K, Sched<[Sched.MR]>;
def: Pat<(mstore addr:$ptr, _.KRCWM:$mask, (_.VT _.RC:$src)),
(!cast<Instruction>(NAME#_.ZSuffix##mrk) addr:$ptr,
_.KRCWM:$mask, _.RC:$src)>;
(!cast<Instruction>(BaseName#_.ZSuffix##mrk) addr:$ptr,
_.KRCWM:$mask, _.RC:$src)>;
}
multiclass avx512_store_vl< bits<8> opc, string OpcodeStr,
@ -3384,14 +3397,14 @@ multiclass avx512_store_vl< bits<8> opc, string OpcodeStr,
string Name, X86SchedWriteMoveLSWidths Sched,
bit NoMRPattern = 0> {
let Predicates = [prd] in
defm Z : avx512_store<opc, OpcodeStr, _.info512, store,
defm Z : avx512_store<opc, OpcodeStr, NAME, _.info512, store,
masked_store_unaligned, Name#Z, Sched.ZMM,
NoMRPattern>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_store<opc, OpcodeStr, _.info256, store,
defm Z256 : avx512_store<opc, OpcodeStr, NAME, _.info256, store,
masked_store_unaligned, Name#Z256, Sched.YMM,
NoMRPattern>, EVEX_V256;
defm Z128 : avx512_store<opc, OpcodeStr, _.info128, store,
defm Z128 : avx512_store<opc, OpcodeStr, NAME, _.info128, store,
masked_store_unaligned, Name#Z128, Sched.XMM,
NoMRPattern>, EVEX_V128;
}
@ -3402,15 +3415,15 @@ multiclass avx512_alignedstore_vl<bits<8> opc, string OpcodeStr,
string Name, X86SchedWriteMoveLSWidths Sched,
bit NoMRPattern = 0> {
let Predicates = [prd] in
defm Z : avx512_store<opc, OpcodeStr, _.info512, alignedstore,
defm Z : avx512_store<opc, OpcodeStr, NAME, _.info512, alignedstore,
masked_store_aligned512, Name#Z, Sched.ZMM,
NoMRPattern>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_store<opc, OpcodeStr, _.info256, alignedstore,
defm Z256 : avx512_store<opc, OpcodeStr, NAME, _.info256, alignedstore,
masked_store_aligned256, Name#Z256, Sched.YMM,
NoMRPattern>, EVEX_V256;
defm Z128 : avx512_store<opc, OpcodeStr, _.info128, alignedstore,
defm Z128 : avx512_store<opc, OpcodeStr, NAME, _.info128, alignedstore,
masked_store_aligned128, Name#Z128, Sched.XMM,
NoMRPattern>, EVEX_V128;
}
@ -5479,7 +5492,7 @@ defm VSCALEF : avx512_fp_scalef_all<0x2C, 0x2D, "vscalef", X86scalef, X86scalefs
multiclass avx512_vptest<bits<8> opc, string OpcodeStr, PatFrag OpNode,
X86FoldableSchedWrite sched, X86VectorVTInfo _,
string Suffix> {
string Name> {
let ExeDomain = _.ExeDomain in {
let isCommutable = 1 in
defm rr : AVX512_maskable_cmp<opc, MRMSrcReg, _, (outs _.KRC:$dst),
@ -5501,11 +5514,11 @@ multiclass avx512_vptest<bits<8> opc, string OpcodeStr, PatFrag OpNode,
// Patterns for compare with 0 that just use the same source twice.
def : Pat<(_.KVT (OpNode _.RC:$src, _.ImmAllZerosV)),
(_.KVT (!cast<Instruction>(NAME # Suffix # _.ZSuffix # "rr")
(_.KVT (!cast<Instruction>(Name # _.ZSuffix # "rr")
_.RC:$src, _.RC:$src))>;
def : Pat<(_.KVT (and _.KRC:$mask, (OpNode _.RC:$src, _.ImmAllZerosV))),
(_.KVT (!cast<Instruction>(NAME # Suffix # _.ZSuffix # "rrk")
(_.KVT (!cast<Instruction>(Name # _.ZSuffix # "rrk")
_.KRC:$mask, _.RC:$src, _.RC:$src))>;
}
@ -5526,11 +5539,11 @@ multiclass avx512_vptest_mb<bits<8> opc, string OpcodeStr, PatFrag OpNode,
// Use 512bit version to implement 128/256 bit in case NoVLX.
multiclass avx512_vptest_lowering<PatFrag OpNode, X86VectorVTInfo ExtendInfo,
X86VectorVTInfo _, string Suffix> {
X86VectorVTInfo _, string Name> {
def : Pat<(_.KVT (OpNode (bitconvert (_.i64VT (and _.RC:$src1, _.RC:$src2))),
_.ImmAllZerosV)),
(_.KVT (COPY_TO_REGCLASS
(!cast<Instruction>(NAME # Suffix # "Zrr")
(!cast<Instruction>(Name # "Zrr")
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
_.RC:$src1, _.SubRegIdx),
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
@ -5541,7 +5554,7 @@ multiclass avx512_vptest_lowering<PatFrag OpNode, X86VectorVTInfo ExtendInfo,
(OpNode (bitconvert (_.i64VT (and _.RC:$src1, _.RC:$src2))),
_.ImmAllZerosV))),
(COPY_TO_REGCLASS
(!cast<Instruction>(NAME # Suffix # "Zrrk")
(!cast<Instruction>(Name # "Zrrk")
(COPY_TO_REGCLASS _.KRC:$mask, ExtendInfo.KRC),
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
_.RC:$src1, _.SubRegIdx),
@ -5551,7 +5564,7 @@ multiclass avx512_vptest_lowering<PatFrag OpNode, X86VectorVTInfo ExtendInfo,
def : Pat<(_.KVT (OpNode _.RC:$src, _.ImmAllZerosV)),
(_.KVT (COPY_TO_REGCLASS
(!cast<Instruction>(NAME # Suffix # "Zrr")
(!cast<Instruction>(Name # "Zrr")
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
_.RC:$src, _.SubRegIdx),
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
@ -5560,7 +5573,7 @@ multiclass avx512_vptest_lowering<PatFrag OpNode, X86VectorVTInfo ExtendInfo,
def : Pat<(_.KVT (and _.KRC:$mask, (OpNode _.RC:$src, _.ImmAllZerosV))),
(COPY_TO_REGCLASS
(!cast<Instruction>(NAME # Suffix # "Zrrk")
(!cast<Instruction>(Name # "Zrrk")
(COPY_TO_REGCLASS _.KRC:$mask, ExtendInfo.KRC),
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
_.RC:$src, _.SubRegIdx),
@ -5570,57 +5583,56 @@ multiclass avx512_vptest_lowering<PatFrag OpNode, X86VectorVTInfo ExtendInfo,
}
multiclass avx512_vptest_dq_sizes<bits<8> opc, string OpcodeStr, PatFrag OpNode,
X86SchedWriteWidths sched, AVX512VLVectorVTInfo _,
string Suffix> {
X86SchedWriteWidths sched, AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in
defm Z : avx512_vptest<opc, OpcodeStr, OpNode, sched.ZMM, _.info512, Suffix>,
defm Z : avx512_vptest<opc, OpcodeStr, OpNode, sched.ZMM, _.info512, NAME>,
avx512_vptest_mb<opc, OpcodeStr, OpNode, sched.ZMM, _.info512>, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in {
defm Z256 : avx512_vptest<opc, OpcodeStr, OpNode, sched.YMM, _.info256, Suffix>,
defm Z256 : avx512_vptest<opc, OpcodeStr, OpNode, sched.YMM, _.info256, NAME>,
avx512_vptest_mb<opc, OpcodeStr, OpNode, sched.YMM, _.info256>, EVEX_V256;
defm Z128 : avx512_vptest<opc, OpcodeStr, OpNode, sched.XMM, _.info128, Suffix>,
defm Z128 : avx512_vptest<opc, OpcodeStr, OpNode, sched.XMM, _.info128, NAME>,
avx512_vptest_mb<opc, OpcodeStr, OpNode, sched.XMM, _.info128>, EVEX_V128;
}
let Predicates = [HasAVX512, NoVLX] in {
defm Z256_Alt : avx512_vptest_lowering< OpNode, _.info512, _.info256, Suffix>;
defm Z128_Alt : avx512_vptest_lowering< OpNode, _.info512, _.info128, Suffix>;
defm Z256_Alt : avx512_vptest_lowering< OpNode, _.info512, _.info256, NAME>;
defm Z128_Alt : avx512_vptest_lowering< OpNode, _.info512, _.info128, NAME>;
}
}
multiclass avx512_vptest_dq<bits<8> opc, string OpcodeStr, PatFrag OpNode,
X86SchedWriteWidths sched> {
defm D : avx512_vptest_dq_sizes<opc, OpcodeStr#"d", OpNode, sched,
avx512vl_i32_info, "D">;
avx512vl_i32_info>;
defm Q : avx512_vptest_dq_sizes<opc, OpcodeStr#"q", OpNode, sched,
avx512vl_i64_info, "Q">, VEX_W;
avx512vl_i64_info>, VEX_W;
}
multiclass avx512_vptest_wb<bits<8> opc, string OpcodeStr,
PatFrag OpNode, X86SchedWriteWidths sched> {
let Predicates = [HasBWI] in {
defm WZ: avx512_vptest<opc, OpcodeStr#"w", OpNode, sched.ZMM,
v32i16_info, "W">, EVEX_V512, VEX_W;
v32i16_info, NAME#"W">, EVEX_V512, VEX_W;
defm BZ: avx512_vptest<opc, OpcodeStr#"b", OpNode, sched.ZMM,
v64i8_info, "B">, EVEX_V512;
v64i8_info, NAME#"B">, EVEX_V512;
}
let Predicates = [HasVLX, HasBWI] in {
defm WZ256: avx512_vptest<opc, OpcodeStr#"w", OpNode, sched.YMM,
v16i16x_info, "W">, EVEX_V256, VEX_W;
v16i16x_info, NAME#"W">, EVEX_V256, VEX_W;
defm WZ128: avx512_vptest<opc, OpcodeStr#"w", OpNode, sched.XMM,
v8i16x_info, "W">, EVEX_V128, VEX_W;
v8i16x_info, NAME#"W">, EVEX_V128, VEX_W;
defm BZ256: avx512_vptest<opc, OpcodeStr#"b", OpNode, sched.YMM,
v32i8x_info, "B">, EVEX_V256;
v32i8x_info, NAME#"B">, EVEX_V256;
defm BZ128: avx512_vptest<opc, OpcodeStr#"b", OpNode, sched.XMM,
v16i8x_info, "B">, EVEX_V128;
v16i8x_info, NAME#"B">, EVEX_V128;
}
let Predicates = [HasAVX512, NoVLX] in {
defm BZ256_Alt : avx512_vptest_lowering<OpNode, v64i8_info, v32i8x_info, "B">;
defm BZ128_Alt : avx512_vptest_lowering<OpNode, v64i8_info, v16i8x_info, "B">;
defm WZ256_Alt : avx512_vptest_lowering<OpNode, v32i16_info, v16i16x_info, "W">;
defm WZ128_Alt : avx512_vptest_lowering<OpNode, v32i16_info, v8i16x_info, "W">;
defm BZ256_Alt : avx512_vptest_lowering<OpNode, v64i8_info, v32i8x_info, NAME#"B">;
defm BZ128_Alt : avx512_vptest_lowering<OpNode, v64i8_info, v16i8x_info, NAME#"B">;
defm WZ256_Alt : avx512_vptest_lowering<OpNode, v32i16_info, v16i16x_info, NAME#"W">;
defm WZ128_Alt : avx512_vptest_lowering<OpNode, v32i16_info, v8i16x_info, NAME#"W">;
}
}
@ -8414,7 +8426,7 @@ multiclass avx512_sqrt_packed_all_round<bits<8> opc, string OpcodeStr,
}
multiclass avx512_sqrt_scalar<bits<8> opc, string OpcodeStr, X86FoldableSchedWrite sched,
X86VectorVTInfo _, string SUFF, Intrinsic Intr> {
X86VectorVTInfo _, string Name, Intrinsic Intr> {
let ExeDomain = _.ExeDomain in {
defm r_Int : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
@ -8453,31 +8465,31 @@ multiclass avx512_sqrt_scalar<bits<8> opc, string OpcodeStr, X86FoldableSchedWri
let Predicates = [HasAVX512] in {
def : Pat<(_.EltVT (fsqrt _.FRC:$src)),
(!cast<Instruction>(NAME#SUFF#Zr)
(!cast<Instruction>(Name#Zr)
(_.EltVT (IMPLICIT_DEF)), _.FRC:$src)>;
def : Pat<(Intr VR128X:$src),
(!cast<Instruction>(NAME#SUFF#Zr_Int) VR128X:$src,
(!cast<Instruction>(Name#Zr_Int) VR128X:$src,
VR128X:$src)>;
}
let Predicates = [HasAVX512, OptForSize] in {
def : Pat<(_.EltVT (fsqrt (load addr:$src))),
(!cast<Instruction>(NAME#SUFF#Zm)
(!cast<Instruction>(Name#Zm)
(_.EltVT (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(Intr _.ScalarIntMemCPat:$src2),
(!cast<Instruction>(NAME#SUFF#Zm_Int)
(!cast<Instruction>(Name#Zm_Int)
(_.VT (IMPLICIT_DEF)), addr:$src2)>;
}
}
multiclass avx512_sqrt_scalar_all<bits<8> opc, string OpcodeStr,
X86SchedWriteSizes sched> {
defm SSZ : avx512_sqrt_scalar<opc, OpcodeStr#"ss", sched.PS.Scl, f32x_info, "SS",
defm SSZ : avx512_sqrt_scalar<opc, OpcodeStr#"ss", sched.PS.Scl, f32x_info, NAME#"SS",
int_x86_sse_sqrt_ss>,
EVEX_CD8<32, CD8VT1>, EVEX_4V, XS, NotMemoryFoldable;
defm SDZ : avx512_sqrt_scalar<opc, OpcodeStr#"sd", sched.PD.Scl, f64x_info, "SD",
defm SDZ : avx512_sqrt_scalar<opc, OpcodeStr#"sd", sched.PD.Scl, f64x_info, NAME#"SD",
int_x86_sse2_sqrt_sd>,
EVEX_CD8<64, CD8VT1>, EVEX_4V, XD, VEX_W,
NotMemoryFoldable;
@ -8639,15 +8651,16 @@ multiclass avx512_trunc_common<bits<8> opc, string OpcodeStr, SDNode OpNode,
multiclass avx512_trunc_mr_lowering<X86VectorVTInfo SrcInfo,
X86VectorVTInfo DestInfo,
PatFrag truncFrag, PatFrag mtruncFrag > {
PatFrag truncFrag, PatFrag mtruncFrag,
string Name> {
def : Pat<(truncFrag (SrcInfo.VT SrcInfo.RC:$src), addr:$dst),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##mr)
(!cast<Instruction>(Name#SrcInfo.ZSuffix##mr)
addr:$dst, SrcInfo.RC:$src)>;
def : Pat<(mtruncFrag addr:$dst, SrcInfo.KRCWM:$mask,
(SrcInfo.VT SrcInfo.RC:$src)),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##mrk)
(!cast<Instruction>(Name#SrcInfo.ZSuffix##mrk)
addr:$dst, SrcInfo.KRCWM:$mask, SrcInfo.RC:$src)>;
}
@ -8664,18 +8677,18 @@ multiclass avx512_trunc<bits<8> opc, string OpcodeStr, SDNode OpNode128,
defm Z128: avx512_trunc_common<opc, OpcodeStr, OpNode128, sched,
VTSrcInfo.info128, DestInfoZ128, x86memopZ128>,
avx512_trunc_mr_lowering<VTSrcInfo.info128, DestInfoZ128,
truncFrag, mtruncFrag>, EVEX_V128;
truncFrag, mtruncFrag, NAME>, EVEX_V128;
defm Z256: avx512_trunc_common<opc, OpcodeStr, OpNode256, sched,
VTSrcInfo.info256, DestInfoZ256, x86memopZ256>,
avx512_trunc_mr_lowering<VTSrcInfo.info256, DestInfoZ256,
truncFrag, mtruncFrag>, EVEX_V256;
truncFrag, mtruncFrag, NAME>, EVEX_V256;
}
let Predicates = [prd] in
defm Z: avx512_trunc_common<opc, OpcodeStr, OpNode512, sched,
VTSrcInfo.info512, DestInfoZ, x86memopZ>,
avx512_trunc_mr_lowering<VTSrcInfo.info512, DestInfoZ,
truncFrag, mtruncFrag>, EVEX_V512;
truncFrag, mtruncFrag, NAME>, EVEX_V512;
}
multiclass avx512_trunc_qb<bits<8> opc, string OpcodeStr, SDNode OpNode,
@ -9292,11 +9305,12 @@ multiclass convert_vector_to_mask_common<bits<8> opc, X86VectorVTInfo _, string
// Use 512bit version to implement 128/256 bit in case NoVLX.
multiclass convert_vector_to_mask_lowering<X86VectorVTInfo ExtendInfo,
X86VectorVTInfo _> {
X86VectorVTInfo _,
string Name> {
def : Pat<(_.KVT (X86pcmpgtm _.ImmAllZerosV, (_.VT _.RC:$src))),
(_.KVT (COPY_TO_REGCLASS
(!cast<Instruction>(NAME#"Zrr")
(!cast<Instruction>(Name#"Zrr")
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
_.RC:$src, _.SubRegIdx)),
_.KRC))>;
@ -9315,8 +9329,8 @@ multiclass avx512_convert_vector_to_mask<bits<8> opc, string OpcodeStr,
EVEX_V128;
}
let Predicates = [prd, NoVLX] in {
defm Z256_Alt : convert_vector_to_mask_lowering<VTInfo.info512, VTInfo.info256>;
defm Z128_Alt : convert_vector_to_mask_lowering<VTInfo.info512, VTInfo.info128>;
defm Z256_Alt : convert_vector_to_mask_lowering<VTInfo.info512, VTInfo.info256, NAME>;
defm Z128_Alt : convert_vector_to_mask_lowering<VTInfo.info512, VTInfo.info128, NAME>;
}
}
@ -9365,10 +9379,10 @@ multiclass compress_by_vec_width_common<bits<8> opc, X86VectorVTInfo _,
Sched<[sched.Folded]>;
}
multiclass compress_by_vec_width_lowering<X86VectorVTInfo _ > {
multiclass compress_by_vec_width_lowering<X86VectorVTInfo _, string Name> {
def : Pat<(X86mCompressingStore addr:$dst, _.KRCWM:$mask,
(_.VT _.RC:$src)),
(!cast<Instruction>(NAME#_.ZSuffix##mrk)
(!cast<Instruction>(Name#_.ZSuffix##mrk)
addr:$dst, _.KRCWM:$mask, _.RC:$src)>;
}
@ -9378,13 +9392,13 @@ multiclass compress_by_elt_width<bits<8> opc, string OpcodeStr,
Predicate Pred = HasAVX512> {
let Predicates = [Pred] in
defm Z : compress_by_vec_width_common<opc, VTInfo.info512, OpcodeStr, sched>,
compress_by_vec_width_lowering<VTInfo.info512>, EVEX_V512;
compress_by_vec_width_lowering<VTInfo.info512, NAME>, EVEX_V512;
let Predicates = [Pred, HasVLX] in {
defm Z256 : compress_by_vec_width_common<opc, VTInfo.info256, OpcodeStr, sched>,
compress_by_vec_width_lowering<VTInfo.info256>, EVEX_V256;
compress_by_vec_width_lowering<VTInfo.info256, NAME>, EVEX_V256;
defm Z128 : compress_by_vec_width_common<opc, VTInfo.info128, OpcodeStr, sched>,
compress_by_vec_width_lowering<VTInfo.info128>, EVEX_V128;
compress_by_vec_width_lowering<VTInfo.info128, NAME>, EVEX_V128;
}
}
@ -9414,19 +9428,19 @@ multiclass expand_by_vec_width<bits<8> opc, X86VectorVTInfo _,
Sched<[sched.Folded, ReadAfterLd]>;
}
multiclass expand_by_vec_width_lowering<X86VectorVTInfo _ > {
multiclass expand_by_vec_width_lowering<X86VectorVTInfo _, string Name> {
def : Pat<(_.VT (X86mExpandingLoad addr:$src, _.KRCWM:$mask, undef)),
(!cast<Instruction>(NAME#_.ZSuffix##rmkz)
(!cast<Instruction>(Name#_.ZSuffix##rmkz)
_.KRCWM:$mask, addr:$src)>;
def : Pat<(_.VT (X86mExpandingLoad addr:$src, _.KRCWM:$mask, _.ImmAllZerosV)),
(!cast<Instruction>(NAME#_.ZSuffix##rmkz)
(!cast<Instruction>(Name#_.ZSuffix##rmkz)
_.KRCWM:$mask, addr:$src)>;
def : Pat<(_.VT (X86mExpandingLoad addr:$src, _.KRCWM:$mask,
(_.VT _.RC:$src0))),
(!cast<Instruction>(NAME#_.ZSuffix##rmk)
(!cast<Instruction>(Name#_.ZSuffix##rmk)
_.RC:$src0, _.KRCWM:$mask, addr:$src)>;
}
@ -9436,13 +9450,13 @@ multiclass expand_by_elt_width<bits<8> opc, string OpcodeStr,
Predicate Pred = HasAVX512> {
let Predicates = [Pred] in
defm Z : expand_by_vec_width<opc, VTInfo.info512, OpcodeStr, sched>,
expand_by_vec_width_lowering<VTInfo.info512>, EVEX_V512;
expand_by_vec_width_lowering<VTInfo.info512, NAME>, EVEX_V512;
let Predicates = [Pred, HasVLX] in {
defm Z256 : expand_by_vec_width<opc, VTInfo.info256, OpcodeStr, sched>,
expand_by_vec_width_lowering<VTInfo.info256>, EVEX_V256;
expand_by_vec_width_lowering<VTInfo.info256, NAME>, EVEX_V256;
defm Z128 : expand_by_vec_width<opc, VTInfo.info128, OpcodeStr, sched>,
expand_by_vec_width_lowering<VTInfo.info128>, EVEX_V128;
expand_by_vec_width_lowering<VTInfo.info128, NAME>, EVEX_V128;
}
}
@ -10584,7 +10598,8 @@ def VPTERNLOG312_imm8 : SDNodeXForm<imm, [{
}]>;
multiclass avx512_ternlog<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86FoldableSchedWrite sched, X86VectorVTInfo _>{
X86FoldableSchedWrite sched, X86VectorVTInfo _,
string Name>{
let Constraints = "$src1 = $dst", ExeDomain = _.ExeDomain in {
defm rri : AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, u8imm:$src4),
@ -10619,23 +10634,23 @@ multiclass avx512_ternlog<bits<8> opc, string OpcodeStr, SDNode OpNode,
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode _.RC:$src3, _.RC:$src2, _.RC:$src1, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rrik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rrik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, _.RC:$src3, (VPTERNLOG321_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode _.RC:$src2, _.RC:$src1, _.RC:$src3, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rrik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rrik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, _.RC:$src3, (VPTERNLOG213_imm8 imm:$src4))>;
// Additional patterns for matching loads in other positions.
def : Pat<(_.VT (OpNode (bitconvert (_.LdFrag addr:$src3)),
_.RC:$src2, _.RC:$src1, (i8 imm:$src4))),
(!cast<Instruction>(NAME#_.ZSuffix#rmi) _.RC:$src1, _.RC:$src2,
(!cast<Instruction>(Name#_.ZSuffix#rmi) _.RC:$src1, _.RC:$src2,
addr:$src3, (VPTERNLOG321_imm8 imm:$src4))>;
def : Pat<(_.VT (OpNode _.RC:$src1,
(bitconvert (_.LdFrag addr:$src3)),
_.RC:$src2, (i8 imm:$src4))),
(!cast<Instruction>(NAME#_.ZSuffix#rmi) _.RC:$src1, _.RC:$src2,
(!cast<Instruction>(Name#_.ZSuffix#rmi) _.RC:$src1, _.RC:$src2,
addr:$src3, (VPTERNLOG132_imm8 imm:$src4))>;
// Additional patterns for matching zero masking with loads in other
@ -10644,13 +10659,13 @@ multiclass avx512_ternlog<bits<8> opc, string OpcodeStr, SDNode OpNode,
(OpNode (bitconvert (_.LdFrag addr:$src3)),
_.RC:$src2, _.RC:$src1, (i8 imm:$src4)),
_.ImmAllZerosV)),
(!cast<Instruction>(NAME#_.ZSuffix#rmikz) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmikz) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG321_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode _.RC:$src1, (bitconvert (_.LdFrag addr:$src3)),
_.RC:$src2, (i8 imm:$src4)),
_.ImmAllZerosV)),
(!cast<Instruction>(NAME#_.ZSuffix#rmikz) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmikz) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG132_imm8 imm:$src4))>;
// Additional patterns for matching masked loads with different
@ -10659,42 +10674,42 @@ multiclass avx512_ternlog<bits<8> opc, string OpcodeStr, SDNode OpNode,
(OpNode _.RC:$src1, (bitconvert (_.LdFrag addr:$src3)),
_.RC:$src2, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG132_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode (bitconvert (_.LdFrag addr:$src3)),
_.RC:$src2, _.RC:$src1, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG321_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode _.RC:$src2, _.RC:$src1,
(bitconvert (_.LdFrag addr:$src3)), (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG213_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode _.RC:$src2, (bitconvert (_.LdFrag addr:$src3)),
_.RC:$src1, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG231_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode (bitconvert (_.LdFrag addr:$src3)),
_.RC:$src1, _.RC:$src2, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG312_imm8 imm:$src4))>;
// Additional patterns for matching broadcasts in other positions.
def : Pat<(_.VT (OpNode (X86VBroadcast (_.ScalarLdFrag addr:$src3)),
_.RC:$src2, _.RC:$src1, (i8 imm:$src4))),
(!cast<Instruction>(NAME#_.ZSuffix#rmbi) _.RC:$src1, _.RC:$src2,
(!cast<Instruction>(Name#_.ZSuffix#rmbi) _.RC:$src1, _.RC:$src2,
addr:$src3, (VPTERNLOG321_imm8 imm:$src4))>;
def : Pat<(_.VT (OpNode _.RC:$src1,
(X86VBroadcast (_.ScalarLdFrag addr:$src3)),
_.RC:$src2, (i8 imm:$src4))),
(!cast<Instruction>(NAME#_.ZSuffix#rmbi) _.RC:$src1, _.RC:$src2,
(!cast<Instruction>(Name#_.ZSuffix#rmbi) _.RC:$src1, _.RC:$src2,
addr:$src3, (VPTERNLOG132_imm8 imm:$src4))>;
// Additional patterns for matching zero masking with broadcasts in other
@ -10703,7 +10718,7 @@ multiclass avx512_ternlog<bits<8> opc, string OpcodeStr, SDNode OpNode,
(OpNode (X86VBroadcast (_.ScalarLdFrag addr:$src3)),
_.RC:$src2, _.RC:$src1, (i8 imm:$src4)),
_.ImmAllZerosV)),
(!cast<Instruction>(NAME#_.ZSuffix#rmbikz) _.RC:$src1,
(!cast<Instruction>(Name#_.ZSuffix#rmbikz) _.RC:$src1,
_.KRCWM:$mask, _.RC:$src2, addr:$src3,
(VPTERNLOG321_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
@ -10711,7 +10726,7 @@ multiclass avx512_ternlog<bits<8> opc, string OpcodeStr, SDNode OpNode,
(X86VBroadcast (_.ScalarLdFrag addr:$src3)),
_.RC:$src2, (i8 imm:$src4)),
_.ImmAllZerosV)),
(!cast<Instruction>(NAME#_.ZSuffix#rmbikz) _.RC:$src1,
(!cast<Instruction>(Name#_.ZSuffix#rmbikz) _.RC:$src1,
_.KRCWM:$mask, _.RC:$src2, addr:$src3,
(VPTERNLOG132_imm8 imm:$src4))>;
@ -10722,32 +10737,32 @@ multiclass avx512_ternlog<bits<8> opc, string OpcodeStr, SDNode OpNode,
(X86VBroadcast (_.ScalarLdFrag addr:$src3)),
_.RC:$src2, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG132_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode (X86VBroadcast (_.ScalarLdFrag addr:$src3)),
_.RC:$src2, _.RC:$src1, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG321_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode _.RC:$src2, _.RC:$src1,
(X86VBroadcast (_.ScalarLdFrag addr:$src3)),
(i8 imm:$src4)), _.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG213_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode _.RC:$src2,
(X86VBroadcast (_.ScalarLdFrag addr:$src3)),
_.RC:$src1, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG231_imm8 imm:$src4))>;
def : Pat<(_.VT (vselect _.KRCWM:$mask,
(OpNode (X86VBroadcast (_.ScalarLdFrag addr:$src3)),
_.RC:$src1, _.RC:$src2, (i8 imm:$src4)),
_.RC:$src1)),
(!cast<Instruction>(NAME#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
(!cast<Instruction>(Name#_.ZSuffix#rmbik) _.RC:$src1, _.KRCWM:$mask,
_.RC:$src2, addr:$src3, (VPTERNLOG312_imm8 imm:$src4))>;
}
@ -10755,12 +10770,12 @@ multiclass avx512_common_ternlog<string OpcodeStr, X86SchedWriteWidths sched,
AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in
defm Z : avx512_ternlog<0x25, OpcodeStr, X86vpternlog, sched.ZMM,
_.info512>, EVEX_V512;
_.info512, NAME>, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in {
defm Z128 : avx512_ternlog<0x25, OpcodeStr, X86vpternlog, sched.XMM,
_.info128>, EVEX_V128;
_.info128, NAME>, EVEX_V128;
defm Z256 : avx512_ternlog<0x25, OpcodeStr, X86vpternlog, sched.YMM,
_.info256>, EVEX_V256;
_.info256, NAME>, EVEX_V256;
}
}

25
lib/Target/X86/X86InstrSSE.td
View File

@ -2740,7 +2740,7 @@ multiclass sse_fp_unop_s<bits<8> opc, string OpcodeStr, RegisterClass RC,
Operand intmemop, ComplexPattern int_cpat,
Intrinsic Intr, SDNode OpNode, Domain d,
X86FoldableSchedWrite sched,
Predicate target, string Suffix> {
Predicate target> {
let hasSideEffects = 0 in {
def r : I<opc, MRMSrcReg, (outs RC:$dst), (ins RC:$src1),
!strconcat(OpcodeStr, "\t{$src1, $dst|$dst, $src1}"),
@ -2768,7 +2768,7 @@ multiclass sse_fp_unop_s<bits<8> opc, string OpcodeStr, RegisterClass RC,
// These are unary operations, but they are modeled as having 2 source operands
// because the high elements of the destination are unchanged in SSE.
def : Pat<(Intr VR128:$src),
(!cast<Instruction>(NAME#Suffix##r_Int) VR128:$src, VR128:$src)>;
(!cast<Instruction>(NAME#r_Int) VR128:$src, VR128:$src)>;
}
// We don't want to fold scalar loads into these instructions unless
// optimizing for size. This is because the folded instruction will have a
@ -2779,7 +2779,7 @@ multiclass sse_fp_unop_s<bits<8> opc, string OpcodeStr, RegisterClass RC,
// rcpss mem, %xmm0
let Predicates = [target, OptForSize] in {
def : Pat<(Intr int_cpat:$src2),
(!cast<Instruction>(NAME#Suffix##m_Int)
(!cast<Instruction>(NAME#m_Int)
(vt (IMPLICIT_DEF)), addr:$src2)>;
}
}
@ -2789,8 +2789,7 @@ multiclass avx_fp_unop_s<bits<8> opc, string OpcodeStr, RegisterClass RC,
X86MemOperand x86memop,
Operand intmemop, ComplexPattern int_cpat,
Intrinsic Intr, SDNode OpNode, Domain d,
X86FoldableSchedWrite sched, Predicate target,
string Suffix> {
X86FoldableSchedWrite sched, Predicate target> {
let hasSideEffects = 0 in {
def r : I<opc, MRMSrcReg, (outs RC:$dst), (ins RC:$src1, RC:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
@ -2822,18 +2821,18 @@ multiclass avx_fp_unop_s<bits<8> opc, string OpcodeStr, RegisterClass RC,
// TODO: In theory, we could fold the load, and avoid the stall caused by
// the partial register store, either in BreakFalseDeps or with smarter RA.
let Predicates = [target] in {
def : Pat<(OpNode RC:$src), (!cast<Instruction>("V"#NAME#Suffix##r)
def : Pat<(OpNode RC:$src), (!cast<Instruction>(NAME#r)
(ScalarVT (IMPLICIT_DEF)), RC:$src)>;
def : Pat<(Intr VR128:$src),
(!cast<Instruction>("V"#NAME#Suffix##r_Int) VR128:$src,
(!cast<Instruction>(NAME#r_Int) VR128:$src,
VR128:$src)>;
}
let Predicates = [target, OptForSize] in {
def : Pat<(Intr int_cpat:$src2),
(!cast<Instruction>("V"#NAME#Suffix##m_Int)
(!cast<Instruction>(NAME#m_Int)
(vt (IMPLICIT_DEF)), addr:$src2)>;
def : Pat<(ScalarVT (OpNode (load addr:$src))),
(!cast<Instruction>("V"#NAME#Suffix##m) (ScalarVT (IMPLICIT_DEF)),
(!cast<Instruction>(NAME#m) (ScalarVT (IMPLICIT_DEF)),
addr:$src)>;
}
}
@ -2915,11 +2914,11 @@ multiclass sse1_fp_unop_s<bits<8> opc, string OpcodeStr, SDNode OpNode,
defm SS : sse_fp_unop_s<opc, OpcodeStr##ss, FR32, v4f32, f32, f32mem,
ssmem, sse_load_f32,
!cast<Intrinsic>("int_x86_sse_"##OpcodeStr##_ss), OpNode,
SSEPackedSingle, sched.Scl, UseSSE1, "SS">, XS;
SSEPackedSingle, sched.Scl, UseSSE1>, XS;
defm V#NAME#SS : avx_fp_unop_s<opc, "v"#OpcodeStr##ss, FR32, v4f32, f32,
f32mem, ssmem, sse_load_f32,
!cast<Intrinsic>("int_x86_sse_"##OpcodeStr##_ss), OpNode,
SSEPackedSingle, sched.Scl, AVXTarget, "SS">, XS, VEX_4V,
SSEPackedSingle, sched.Scl, AVXTarget>, XS, VEX_4V,
VEX_LIG, VEX_WIG, NotMemoryFoldable;
}
@ -2928,11 +2927,11 @@ multiclass sse2_fp_unop_s<bits<8> opc, string OpcodeStr, SDNode OpNode,
defm SD : sse_fp_unop_s<opc, OpcodeStr##sd, FR64, v2f64, f64, f64mem,
sdmem, sse_load_f64,
!cast<Intrinsic>("int_x86_sse2_"##OpcodeStr##_sd),
OpNode, SSEPackedDouble, sched.Scl, UseSSE2, "SD">, XD;
OpNode, SSEPackedDouble, sched.Scl, UseSSE2>, XD;
defm V#NAME#SD : avx_fp_unop_s<opc, "v"#OpcodeStr##sd, FR64, v2f64, f64,
f64mem, sdmem, sse_load_f64,
!cast<Intrinsic>("int_x86_sse2_"##OpcodeStr##_sd),
OpNode, SSEPackedDouble, sched.Scl, AVXTarget, "SD">,
OpNode, SSEPackedDouble, sched.Scl, AVXTarget>,
XD, VEX_4V, VEX_LIG, VEX_WIG, NotMemoryFoldable;
}

2
test/TableGen/Dag.td
View File

@ -60,7 +60,6 @@ def VAL3 : bar<foo1, somedef1>;
// CHECK-NEXT: dag Dag1 = (somedef1 1);
// CHECK-NEXT: dag Dag2 = (somedef1 2);
// CHECK-NEXT: dag Dag3 = (somedef1 2);
// CHECK-NEXT: NAME = ?
// CHECK-NEXT: }
@ -69,7 +68,6 @@ def VAL4 : bar<foo2, somedef2>;
// CHECK-NEXT: dag Dag1 = (somedef1 1);
// CHECK-NEXT: dag Dag2 = (somedef2 2);
// CHECK-NEXT: dag Dag3 = (somedef2 2);
// CHECK-NEXT: NAME = ?
// CHECK-NEXT: }
def VAL5 : bar<foo2, somedef2> {

2
test/TableGen/MultiClassDefName.td
View File

@ -43,7 +43,7 @@ multiclass MC3<string s> {
def ZFizz#s : C<s>;
}
defm : MC3<"Buzz">;
defm "" : MC3<"Buzz">;
// CHECK: def ZFizzBuzz
// CHECK: string name = "Buzz";

2
test/TableGen/lisp.td
View File

@ -5,14 +5,12 @@
// CHECK-NEXT: string element = "Jeffrey Sinclair";
// CHECK-NEXT: list<string> rest = [];
// CHECK-NEXT: int null = 1;
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: def Three {
// CHECK-NEXT: list<string> names = ["Tom", "Dick", "Harry"];
// CHECK-NEXT: string element = "Tom";
// CHECK-NEXT: list<string> rest = ["Dick", "Harry"];
// CHECK-NEXT: int null = 0;
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
class List<list<string> n> {

83
test/TableGen/name-resolution-consistency.td
View File

@ -1,57 +1,56 @@
// RUN: llvm-tblgen %s | FileCheck %s
// XFAIL: vg_leak
// This test demonstrates a number of inconsistencies in how NAME is resolved
// and record names are constructed.
//
// The TODO lines describe a suggested consistent behavior that would result
// from:
// (1) Treating NAME as an implicit multiclass template argument and
// (2) always storing the name of (non-anonymous) prototype records in
// multiclasses with at least one explicit reference to NAME.
//
// Unfortunately, several backends (including X86) rely quite heavily on the
// current inconsistent behavior and would have to be fixed.
// CHECK: def B0a {
// CHECK: string e = "B0";
// CHECK: }
// CHECK: def B0ba {
// TODO: expect "B0b" here
// CHECK: string a = "B0";
// CHECK: string a = "B0b";
// CHECK: string b = "B0";
// CHECK: }
// CHECK: def B0bys {
// CHECK: string f = "B0b";
// CHECK: string g = "B0";
// CHECK: }
// CHECK: def B0cza {
// TODO: expect "B0cz" here
// CHECK: string a = "B0";
// CHECK: string a = "B0cz";
// CHECK: string b = "B0";
// CHECK: }
// TODO: expect this to be named 'xB0b'
// CHECK: def B0xb {
// TODO: expect "B0b" here
// CHECK: string c = "b";
// CHECK: string d = "b";
// CHECK: def B0czyt {
// CHECK: string f = "B0cz";
// CHECK: string g = "B0";
// CHECK: }
// TODO: expect this to be named B0bys
// CHECK: def B0ys {
// TODO: expect "B0b" here
// CHECK: string f = "b";
// CHECK: string g = "b";
// CHECK: def C0 {
// CHECK: string a = "C0";
// CHECK: string b = "C0";
// CHECK: string c = "a";
// CHECK: }
// CHECK: def D0a {
// CHECK: string a = "D0a";
// CHECK: string b = "D0a";
// CHECK: string c = "D0";
// CHECK: }
// CHECK: def D0b {
// CHECK: string a = "D0b";
// CHECK: string b = "D0b";
// CHECK: string c = "a";
// CHECK: }
// CHECK: def xB0b {
// CHECK: string c = "B0b";
// CHECK: string d = "B0";
// CHECK: }
// CHECK: def xB0cz {
// CHECK: string c = "B0cz";
// CHECK: string d = "B0cz";
// CHECK: }
// TODO: expect this to be named B0czyt
// CHECK: def yt {
// CHECK: string f = "B0cz";
// CHECK: string g = "B0cz";
// CHECK: string d = "B0";
// CHECK: }
multiclass A<string p, string q> {
@ -82,3 +81,21 @@ multiclass B<string name, string t> {
}
defm B0 : B<"z", "t">;
class Cbase {
string a = NAME;
}
class C<string arg> : Cbase {
string b = NAME;
string c = arg;
}
def C0 : C<"a">;
multiclass D<string arg> {
def a : C<NAME>;
def b : C<arg>;
}
defm D0 : D<"a">;

11
test/TableGen/subst.td
View File

@ -28,51 +28,40 @@ def JimmyJohnson : AName<"Jimmy Johnson", Mr>;
// CHECK-NEXT: class AName<string AName:name = ?, Honorific AName:honorific = ?> {
// CHECK-NEXT: string name = !subst("FIRST", "John", !subst("LAST", "Smith", AName:name));
// CHECK-NEXT: Honorific honorific = !subst(TVAR, Mr, AName:honorific);
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: class Honorific<string Honorific:t = ?> {
// CHECK-NEXT: string honorific = Honorific:t;
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: class Name<string Name:n = ?, Honorific Name:t = ?> {
// CHECK-NEXT: string name = Name:n;
// CHECK-NEXT: Honorific honorific = Name:t;
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: ------------- Defs -----------------
// CHECK-NEXT: def JaneSmith {
// CHECK-NEXT: string name = "Jane Smith";
// CHECK-NEXT: Honorific honorific = Ms;
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: def JimmyJohnson {
// CHECK-NEXT: string name = "Jimmy Johnson";
// CHECK-NEXT: Honorific honorific = Mr;
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: def JohnSmith {
// CHECK-NEXT: string name = "John Smith";
// CHECK-NEXT: Honorific honorific = Mr;
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: def JohnSmithJones {
// CHECK-NEXT: string name = "John Smith-Jones";
// CHECK-NEXT: Honorific honorific = Mr;
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: def Mr
// CHECK-NEXT: string honorific = "Mr.";
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: def Mrs {
// CHECK-NEXT: string honorific = "Mrs.";
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: def Ms {
// CHECK-NEXT: string honorific = "Ms.";
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }
// CHECK-NEXT: def TVAR {
// CHECK-NEXT: string honorific = "Bogus";
// CHECK-NEXT: string NAME = ?;
// CHECK-NEXT: }