RPCS3/llvm-mirror
1
0
Fork 0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2025-01-31 12:41:49 +01:00
Code Issues Projects Releases Wiki Activity

[IR] redefine 'UnsafeAlgebra' / 'reassoc' fast-math-flags and add 'trans' fast-math-flag

Browse Source 
As discussed on llvm-dev:
http://lists.llvm.org/pipermail/llvm-dev/2016-November/107104.html
and again more recently:
http://lists.llvm.org/pipermail/llvm-dev/2017-October/118118.html

...this is a step in cleaning up our fast-math-flags implementation in IR to better match
the capabilities of both clang's user-visible flags and the backend's flags for SDNode.

As proposed in the above threads, we're replacing the 'UnsafeAlgebra' bit (which had the 
'umbrella' meaning that all flags are set) with a new bit that only applies to algebraic 
reassociation - 'AllowReassoc'.

We're also adding a bit to allow approximations for library functions called 'ApproxFunc' 
(this was initially proposed as 'libm' or similar).

...and we're out of bits. 7 bits ought to be enough for anyone, right? :) FWIW, I did 
look at getting this out of SubclassOptionalData via SubclassData (spacious 16-bits), 
but that's apparently already used for other purposes. Also, I don't think we can just 
add a field to FPMathOperator because Operator is not intended to be instantiated. 
We'll defer movement of FMF to another day.

We keep the 'fast' keyword. I thought about removing that, but seeing IR like this:
%f.fast = fadd reassoc nnan ninf nsz arcp contract afn float %op1, %op2
...made me think we want to keep the shortcut synonym.

Finally, this change is binary incompatible with existing IR as seen in the 
compatibility tests. This statement:
"Newer releases can ignore features from older releases, but they cannot miscompile 
them. For example, if nsw is ever replaced with something else, dropping it would be 
a valid way to upgrade the IR." 
( http://llvm.org/docs/DeveloperPolicy.html#ir-backwards-compatibility )
...provides the flexibility we want to make this change without requiring a new IR 
version. Ie, we're not loosening the FP strictness of existing IR. At worst, we will 
fail to optimize some previously 'fast' code because it's no longer recognized as 
'fast'. This should get fixed as we audit/squash all of the uses of 'isFast()'.

Note: an inter-dependent clang commit to use the new API name should closely follow 
commit.

Differential Revision: https://reviews.llvm.org/D39304

llvm-svn: 317488
This commit is contained in:
Sanjay Patel 2017-11-06 16:27:15 +00:00
parent 3a0dfeb76d
commit fd69991264
32 changed files with 387 additions and 191 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

142
docs/LangRef.rst
View File

@ -2272,11 +2272,11 @@ seq\_cst total orderings of other operations that are not marked
Fast-Math Flags Fast-Math Flags
--------------- ---------------
LLVM IR floating-point binary ops (:ref:`fadd <i_fadd>`, LLVM IR floating-point operations (:ref:`fadd <i_fadd>`,
:ref:`fsub <i_fsub>`, :ref:`fmul <i_fmul>`, :ref:`fdiv <i_fdiv>`, :ref:`fsub <i_fsub>`, :ref:`fmul <i_fmul>`, :ref:`fdiv <i_fdiv>`,
:ref:`frem <i_frem>`, :ref:`fcmp <i_fcmp>`) and :ref:`call <i_call>` :ref:`frem <i_frem>`, :ref:`fcmp <i_fcmp>`) and :ref:`call <i_call>`
instructions have the following flags that can be set to enable may use the following flags to enable otherwise unsafe
otherwise unsafe floating point transformations. floating-point transformations.
``nnan`` ``nnan``
No NaNs - Allow optimizations to assume the arguments and result are not No NaNs - Allow optimizations to assume the arguments and result are not
@ -2300,10 +2300,17 @@ otherwise unsafe floating point transformations.
Allow floating-point contraction (e.g. fusing a multiply followed by an Allow floating-point contraction (e.g. fusing a multiply followed by an
addition into a fused multiply-and-add). addition into a fused multiply-and-add).
``afn``
Approximate functions - Allow substitution of approximate calculations for
functions (sin, log, sqrt, etc). See floating-point intrinsic definitions
for places where this can apply to LLVM's intrinsic math functions.
``reassoc``
Allow reassociation transformations for floating-point instructions.
This may dramatically change results in floating point.
``fast`` ``fast``
Fast - Allow algebraically equivalent transformations that may This flag implies all of the others.
dramatically change results in floating point (e.g. reassociate). This
flag implies all the others.
.. _uselistorder: .. _uselistorder:
@ -10483,7 +10490,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.sqrt`` on any This is an overloaded intrinsic. You can use ``llvm.sqrt`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10497,20 +10504,22 @@ all types however.
Overview: Overview:
""""""""" """""""""
The '``llvm.sqrt``' intrinsics return the square root of the specified value, The '``llvm.sqrt``' intrinsics return the square root of the specified value.
returning the same value as the libm '``sqrt``' functions would, but without
trapping or setting ``errno``.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same type. The argument and return value are floating-point numbers of the same type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the square root of the operand if it is a nonnegative Return the same value as a corresponding libm '``sqrt``' function but without
floating point number. trapping or setting ``errno``. For types specified by IEEE-754, the result
matches a conforming libm implementation.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.powi.*``' Intrinsic '``llvm.powi.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10557,7 +10566,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.sin`` on any This is an overloaded intrinsic. You can use ``llvm.sin`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10576,14 +10585,16 @@ The '``llvm.sin.*``' intrinsics return the sine of the operand.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same type. The argument and return value are floating-point numbers of the same type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the sine of the specified operand, returning the Return the same value as a corresponding libm '``sin``' function but without
same values as the libm ``sin`` functions would, and handles error trapping or setting ``errno``.
conditions in the same way.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.cos.*``' Intrinsic '``llvm.cos.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10592,7 +10603,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.cos`` on any This is an overloaded intrinsic. You can use ``llvm.cos`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10611,14 +10622,16 @@ The '``llvm.cos.*``' intrinsics return the cosine of the operand.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same type. The argument and return value are floating-point numbers of the same type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the cosine of the specified operand, returning the Return the same value as a corresponding libm '``cos``' function but without
same values as the libm ``cos`` functions would, and handles error trapping or setting ``errno``.
conditions in the same way.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.pow.*``' Intrinsic '``llvm.pow.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10627,7 +10640,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.pow`` on any This is an overloaded intrinsic. You can use ``llvm.pow`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10647,15 +10660,16 @@ specified (positive or negative) power.
Arguments: Arguments:
"""""""""" """"""""""
The second argument is a floating point power, and the first is a value The arguments and return value are floating-point numbers of the same type.
to raise to that power.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the first value raised to the second power, Return the same value as a corresponding libm '``pow``' function but without
returning the same values as the libm ``pow`` functions would, and trapping or setting ``errno``.
handles error conditions in the same way.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.exp.*``' Intrinsic '``llvm.exp.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10664,7 +10678,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.exp`` on any This is an overloaded intrinsic. You can use ``llvm.exp`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10684,13 +10698,16 @@ value.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same type. The argument and return value are floating-point numbers of the same type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the same values as the libm ``exp`` functions Return the same value as a corresponding libm '``exp``' function but without
would, and handles error conditions in the same way. trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.exp2.*``' Intrinsic '``llvm.exp2.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10699,7 +10716,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.exp2`` on any This is an overloaded intrinsic. You can use ``llvm.exp2`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10719,13 +10736,16 @@ specified value.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same type. The argument and return value are floating-point numbers of the same type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the same values as the libm ``exp2`` functions Return the same value as a corresponding libm '``exp2``' function but without
would, and handles error conditions in the same way. trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.log.*``' Intrinsic '``llvm.log.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10734,7 +10754,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.log`` on any This is an overloaded intrinsic. You can use ``llvm.log`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10754,13 +10774,16 @@ value.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same type. The argument and return value are floating-point numbers of the same type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the same values as the libm ``log`` functions Return the same value as a corresponding libm '``log``' function but without
would, and handles error conditions in the same way. trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.log10.*``' Intrinsic '``llvm.log10.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10769,7 +10792,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.log10`` on any This is an overloaded intrinsic. You can use ``llvm.log10`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10789,13 +10812,16 @@ specified value.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same type. The argument and return value are floating-point numbers of the same type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the same values as the libm ``log10`` functions Return the same value as a corresponding libm '``log10``' function but without
would, and handles error conditions in the same way. trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.log2.*``' Intrinsic '``llvm.log2.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10804,7 +10830,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.log2`` on any This is an overloaded intrinsic. You can use ``llvm.log2`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10824,13 +10850,16 @@ value.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same type. The argument and return value are floating-point numbers of the same type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the same values as the libm ``log2`` functions Return the same value as a corresponding libm '``log2``' function but without
would, and handles error conditions in the same way. trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.fma.*``' Intrinsic '``llvm.fma.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -10839,7 +10868,7 @@ Syntax:
""""""" """""""
This is an overloaded intrinsic. You can use ``llvm.fma`` on any This is an overloaded intrinsic. You can use ``llvm.fma`` on any
floating point or vector of floating point type. Not all targets support floating-point or vector of floating-point type. Not all targets support
all types however. all types however.
:: ::
@ -10853,20 +10882,21 @@ all types however.
Overview: Overview:
""""""""" """""""""
The '``llvm.fma.*``' intrinsics perform the fused multiply-add The '``llvm.fma.*``' intrinsics perform the fused multiply-add operation.
operation.
Arguments: Arguments:
"""""""""" """"""""""
The argument and return value are floating point numbers of the same The arguments and return value are floating-point numbers of the same type.
type.
Semantics: Semantics:
"""""""""" """"""""""
This function returns the same values as the libm ``fma`` functions Return the same value as a corresponding libm '``fma``' function but without
would, and does not set errno. trapping or setting ``errno``.
When specified with the fast-math-flag 'afn', the result may be approximated
using a less accurate calculation.
'``llvm.fabs.*``' Intrinsic '``llvm.fabs.*``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^

24
include/llvm/IR/Instruction.h
View File

@ -308,10 +308,15 @@ public:
/// Determine whether the exact flag is set. /// Determine whether the exact flag is set.
bool isExact() const; bool isExact() const;
/// Set or clear the unsafe-algebra flag on this instruction, which must be an /// Set or clear all fast-math-flags on this instruction, which must be an
/// operator which supports this flag. See LangRef.html for the meaning of /// operator which supports this flag. See LangRef.html for the meaning of
/// this flag. /// this flag.
void setHasUnsafeAlgebra(bool B); void setFast(bool B);
/// Set or clear the reassociation flag on this instruction, which must be
/// an operator which supports this flag. See LangRef.html for the meaning of
/// this flag.
void setHasAllowReassoc(bool B);
/// Set or clear the no-nans flag on this instruction, which must be an /// Set or clear the no-nans flag on this instruction, which must be an
/// operator which supports this flag. See LangRef.html for the meaning of /// operator which supports this flag. See LangRef.html for the meaning of
@ -333,6 +338,11 @@ public:
/// this flag. /// this flag.
void setHasAllowReciprocal(bool B); void setHasAllowReciprocal(bool B);
/// Set or clear the approximate-math-functions flag on this instruction,
/// which must be an operator which supports this flag. See LangRef.html for
/// the meaning of this flag.
void setHasApproxFunc(bool B);
/// Convenience function for setting multiple fast-math flags on this /// Convenience function for setting multiple fast-math flags on this
/// instruction, which must be an operator which supports these flags. See /// instruction, which must be an operator which supports these flags. See
/// LangRef.html for the meaning of these flags. /// LangRef.html for the meaning of these flags.
@ -343,8 +353,11 @@ public:
/// LangRef.html for the meaning of these flags. /// LangRef.html for the meaning of these flags.
void copyFastMathFlags(FastMathFlags FMF); void copyFastMathFlags(FastMathFlags FMF);
/// Determine whether the unsafe-algebra flag is set. /// Determine whether all fast-math-flags are set.
bool hasUnsafeAlgebra() const; bool isFast() const;
/// Determine whether the allow-reassociation flag is set.
bool hasAllowReassoc() const;
/// Determine whether the no-NaNs flag is set. /// Determine whether the no-NaNs flag is set.
bool hasNoNaNs() const; bool hasNoNaNs() const;
@ -361,6 +374,9 @@ public:
/// Determine whether the allow-contract flag is set. /// Determine whether the allow-contract flag is set.
bool hasAllowContract() const; bool hasAllowContract() const;
/// Determine whether the approximate-math-functions flag is set.
bool hasApproxFunc() const;
/// Convenience function for getting all the fast-math flags, which must be an /// Convenience function for getting all the fast-math flags, which must be an
/// operator which supports these flags. See LangRef.html for the meaning of /// operator which supports these flags. See LangRef.html for the meaning of
/// these flags. /// these flags.

111
include/llvm/IR/Operator.h
View File

@ -163,52 +163,61 @@ private:
unsigned Flags = 0; unsigned Flags = 0;
FastMathFlags(unsigned F) : Flags(F) { } FastMathFlags(unsigned F) {
// If all 7 bits are set, turn this into -1. If the number of bits grows,
// this must be updated. This is intended to provide some forward binary
// compatibility insurance for the meaning of 'fast' in case bits are added.
if (F == 0x7F) Flags = ~0U;
else Flags = F;
}
public: public:
/// This is how the bits are used in Value::SubclassOptionalData so they // This is how the bits are used in Value::SubclassOptionalData so they
/// should fit there too. // should fit there too.
// WARNING: We're out of space. SubclassOptionalData only has 7 bits. New
// functionality will require a change in how this information is stored.
enum { enum {
UnsafeAlgebra = (1 << 0), AllowReassoc = (1 << 0),
NoNaNs = (1 << 1), NoNaNs = (1 << 1),
NoInfs = (1 << 2), NoInfs = (1 << 2),
NoSignedZeros = (1 << 3), NoSignedZeros = (1 << 3),
AllowReciprocal = (1 << 4), AllowReciprocal = (1 << 4),
AllowContract = (1 << 5) AllowContract = (1 << 5),
ApproxFunc = (1 << 6)
}; };
FastMathFlags() = default; FastMathFlags() = default;
/// Whether any flag is set
bool any() const { return Flags != 0; } bool any() const { return Flags != 0; }
bool none() const { return Flags == 0; }
bool all() const { return Flags == ~0U; }
/// Set all the flags to false
void clear() { Flags = 0; } void clear() { Flags = 0; }
void set() { Flags = ~0U; }
/// Flag queries /// Flag queries
bool allowReassoc() const { return 0 != (Flags & AllowReassoc); }
bool noNaNs() const { return 0 != (Flags & NoNaNs); } bool noNaNs() const { return 0 != (Flags & NoNaNs); }
bool noInfs() const { return 0 != (Flags & NoInfs); } bool noInfs() const { return 0 != (Flags & NoInfs); }
bool noSignedZeros() const { return 0 != (Flags & NoSignedZeros); } bool noSignedZeros() const { return 0 != (Flags & NoSignedZeros); }
bool allowReciprocal() const { return 0 != (Flags & AllowReciprocal); } bool allowReciprocal() const { return 0 != (Flags & AllowReciprocal); }
bool allowContract() const { return 0 != (Flags & AllowContract); } bool allowContract() const { return 0 != (Flags & AllowContract); }
bool unsafeAlgebra() const { return 0 != (Flags & UnsafeAlgebra); } bool approxFunc() const { return 0 != (Flags & ApproxFunc); }
/// 'Fast' means all bits are set.
bool isFast() const { return all(); }
/// Flag setters /// Flag setters
void setAllowReassoc() { Flags |= AllowReassoc; }
void setNoNaNs() { Flags |= NoNaNs; } void setNoNaNs() { Flags |= NoNaNs; }
void setNoInfs() { Flags |= NoInfs; } void setNoInfs() { Flags |= NoInfs; }
void setNoSignedZeros() { Flags |= NoSignedZeros; } void setNoSignedZeros() { Flags |= NoSignedZeros; }
void setAllowReciprocal() { Flags |= AllowReciprocal; } void setAllowReciprocal() { Flags |= AllowReciprocal; }
// TODO: Change the other set* functions to take a parameter?
void setAllowContract(bool B) { void setAllowContract(bool B) {
Flags = (Flags & ~AllowContract) | B * AllowContract; Flags = (Flags & ~AllowContract) | B * AllowContract;
} }
void setUnsafeAlgebra() { void setApproxFunc() { Flags |= ApproxFunc; }
Flags |= UnsafeAlgebra; void setFast() { set(); }
setNoNaNs();
setNoInfs();
setNoSignedZeros();
setAllowReciprocal();
setAllowContract(true);
}
void operator&=(const FastMathFlags &OtherFlags) { void operator&=(const FastMathFlags &OtherFlags) {
Flags &= OtherFlags.Flags; Flags &= OtherFlags.Flags;
@ -221,18 +230,21 @@ class FPMathOperator : public Operator {
private: private:
friend class Instruction; friend class Instruction;
void setHasUnsafeAlgebra(bool B) { /// 'Fast' means all bits are set.
SubclassOptionalData = void setFast(bool B) {
(SubclassOptionalData & ~FastMathFlags::UnsafeAlgebra) | setHasAllowReassoc(B);
(B * FastMathFlags::UnsafeAlgebra); setHasNoNaNs(B);
setHasNoInfs(B);
// Unsafe algebra implies all the others setHasNoSignedZeros(B);
if (B) { setHasAllowReciprocal(B);
setHasNoNaNs(true); setHasAllowContract(B);
setHasNoInfs(true); setHasApproxFunc(B);
setHasNoSignedZeros(true);
setHasAllowReciprocal(true);
} }
void setHasAllowReassoc(bool B) {
SubclassOptionalData =
(SubclassOptionalData & ~FastMathFlags::AllowReassoc) |
(B * FastMathFlags::AllowReassoc);
} }
void setHasNoNaNs(bool B) { void setHasNoNaNs(bool B) {
@ -265,6 +277,12 @@ private:
(B * FastMathFlags::AllowContract); (B * FastMathFlags::AllowContract);
} }
void setHasApproxFunc(bool B) {
SubclassOptionalData =
(SubclassOptionalData & ~FastMathFlags::ApproxFunc) |
(B * FastMathFlags::ApproxFunc);
}
/// Convenience function for setting multiple fast-math flags. /// Convenience function for setting multiple fast-math flags.
/// FMF is a mask of the bits to set. /// FMF is a mask of the bits to set.
void setFastMathFlags(FastMathFlags FMF) { void setFastMathFlags(FastMathFlags FMF) {
@ -278,42 +296,53 @@ private:
} }
public: public:
/// Test whether this operation is permitted to be /// Test if this operation allows all non-strict floating-point transforms.
/// algebraically transformed, aka the 'A' fast-math property. bool isFast() const {
bool hasUnsafeAlgebra() const { return ((SubclassOptionalData & FastMathFlags::AllowReassoc) != 0 &&
return (SubclassOptionalData & FastMathFlags::UnsafeAlgebra) != 0; (SubclassOptionalData & FastMathFlags::NoNaNs) != 0 &&
(SubclassOptionalData & FastMathFlags::NoInfs) != 0 &&
(SubclassOptionalData & FastMathFlags::NoSignedZeros) != 0 &&
(SubclassOptionalData & FastMathFlags::AllowReciprocal) != 0 &&
(SubclassOptionalData & FastMathFlags::AllowContract) != 0 &&
(SubclassOptionalData & FastMathFlags::ApproxFunc) != 0);
} }
/// Test whether this operation's arguments and results are to be /// Test if this operation may be simplified with reassociative transforms.
/// treated as non-NaN, aka the 'N' fast-math property. bool hasAllowReassoc() const {
return (SubclassOptionalData & FastMathFlags::AllowReassoc) != 0;
}
/// Test if this operation's arguments and results are assumed not-NaN.
bool hasNoNaNs() const { bool hasNoNaNs() const {
return (SubclassOptionalData & FastMathFlags::NoNaNs) != 0; return (SubclassOptionalData & FastMathFlags::NoNaNs) != 0;
} }
/// Test whether this operation's arguments and results are to be /// Test if this operation's arguments and results are assumed not-infinite.
/// treated as NoN-Inf, aka the 'I' fast-math property.
bool hasNoInfs() const { bool hasNoInfs() const {
return (SubclassOptionalData & FastMathFlags::NoInfs) != 0; return (SubclassOptionalData & FastMathFlags::NoInfs) != 0;
} }
/// Test whether this operation can treat the sign of zero /// Test if this operation can ignore the sign of zero.
/// as insignificant, aka the 'S' fast-math property.
bool hasNoSignedZeros() const { bool hasNoSignedZeros() const {
return (SubclassOptionalData & FastMathFlags::NoSignedZeros) != 0; return (SubclassOptionalData & FastMathFlags::NoSignedZeros) != 0;
} }
/// Test whether this operation is permitted to use /// Test if this operation can use reciprocal multiply instead of division.
/// reciprocal instead of division, aka the 'R' fast-math property.
bool hasAllowReciprocal() const { bool hasAllowReciprocal() const {
return (SubclassOptionalData & FastMathFlags::AllowReciprocal) != 0; return (SubclassOptionalData & FastMathFlags::AllowReciprocal) != 0;
} }
/// Test whether this operation is permitted to /// Test if this operation can be floating-point contracted (FMA).
/// be floating-point contracted.
bool hasAllowContract() const { bool hasAllowContract() const {
return (SubclassOptionalData & FastMathFlags::AllowContract) != 0; return (SubclassOptionalData & FastMathFlags::AllowContract) != 0;
} }
/// Test if this operation allows approximations of math library functions or
/// intrinsics.
bool hasApproxFunc() const {
return (SubclassOptionalData & FastMathFlags::ApproxFunc) != 0;
}
/// Convenience function for getting all the fast-math flags /// Convenience function for getting all the fast-math flags
FastMathFlags getFastMathFlags() const { FastMathFlags getFastMathFlags() const {
return FastMathFlags(SubclassOptionalData); return FastMathFlags(SubclassOptionalData);

6
include/llvm/Transforms/Utils/LoopUtils.h
View File

@ -331,15 +331,13 @@ public:
/// not have the "fast-math" property. Such operation requires a relaxed FP /// not have the "fast-math" property. Such operation requires a relaxed FP
/// mode. /// mode.
bool hasUnsafeAlgebra() { bool hasUnsafeAlgebra() {
return InductionBinOp && return InductionBinOp && !cast<FPMathOperator>(InductionBinOp)->isFast();
!cast<FPMathOperator>(InductionBinOp)->hasUnsafeAlgebra();
} }
/// Returns induction operator that does not have "fast-math" property /// Returns induction operator that does not have "fast-math" property
/// and requires FP unsafe mode. /// and requires FP unsafe mode.
Instruction *getUnsafeAlgebraInst() { Instruction *getUnsafeAlgebraInst() {
if (!InductionBinOp || if (!InductionBinOp || cast<FPMathOperator>(InductionBinOp)->isFast())
cast<FPMathOperator>(InductionBinOp)->hasUnsafeAlgebra())
return nullptr; return nullptr;
return InductionBinOp; return InductionBinOp;
} }

2
lib/AsmParser/LLLexer.cpp
View File

@ -552,6 +552,8 @@ lltok::Kind LLLexer::LexIdentifier() {
KEYWORD(nsz); KEYWORD(nsz);
KEYWORD(arcp); KEYWORD(arcp);
KEYWORD(contract); KEYWORD(contract);
KEYWORD(reassoc);
KEYWORD(afn);
KEYWORD(fast); KEYWORD(fast);
KEYWORD(nuw); KEYWORD(nuw);
KEYWORD(nsw); KEYWORD(nsw);

4
lib/AsmParser/LLParser.h
View File

@ -193,7 +193,7 @@ namespace llvm {
FastMathFlags FMF; FastMathFlags FMF;
while (true) while (true)
switch (Lex.getKind()) { switch (Lex.getKind()) {
case lltok::kw_fast: FMF.setUnsafeAlgebra(); Lex.Lex(); continue; case lltok::kw_fast: FMF.setFast(); Lex.Lex(); continue;
case lltok::kw_nnan: FMF.setNoNaNs(); Lex.Lex(); continue; case lltok::kw_nnan: FMF.setNoNaNs(); Lex.Lex(); continue;
case lltok::kw_ninf: FMF.setNoInfs(); Lex.Lex(); continue; case lltok::kw_ninf: FMF.setNoInfs(); Lex.Lex(); continue;
case lltok::kw_nsz: FMF.setNoSignedZeros(); Lex.Lex(); continue; case lltok::kw_nsz: FMF.setNoSignedZeros(); Lex.Lex(); continue;
@ -202,6 +202,8 @@ namespace llvm {
FMF.setAllowContract(true); FMF.setAllowContract(true);
Lex.Lex(); Lex.Lex();
continue; continue;
case lltok::kw_reassoc: FMF.setAllowReassoc(); Lex.Lex(); continue;
case lltok::kw_afn: FMF.setApproxFunc(); Lex.Lex(); continue;
default: return FMF; default: return FMF;
} }
return FMF; return FMF;

2
lib/AsmParser/LLToken.h
View File

@ -102,6 +102,8 @@ enum Kind {
kw_nsz, kw_nsz,
kw_arcp, kw_arcp,
kw_contract, kw_contract,
kw_reassoc,
kw_afn,
kw_fast, kw_fast,
kw_nuw, kw_nuw,
kw_nsw, kw_nsw,

6
lib/Bitcode/Reader/BitcodeReader.cpp
View File

@ -1046,8 +1046,8 @@ static Comdat::SelectionKind getDecodedComdatSelectionKind(unsigned Val) {
static FastMathFlags getDecodedFastMathFlags(unsigned Val) { static FastMathFlags getDecodedFastMathFlags(unsigned Val) {
FastMathFlags FMF; FastMathFlags FMF;
if (0 != (Val & FastMathFlags::UnsafeAlgebra)) if (0 != (Val & FastMathFlags::AllowReassoc))
FMF.setUnsafeAlgebra(); FMF.setAllowReassoc();
if (0 != (Val & FastMathFlags::NoNaNs)) if (0 != (Val & FastMathFlags::NoNaNs))
FMF.setNoNaNs(); FMF.setNoNaNs();
if (0 != (Val & FastMathFlags::NoInfs)) if (0 != (Val & FastMathFlags::NoInfs))
@ -1058,6 +1058,8 @@ static FastMathFlags getDecodedFastMathFlags(unsigned Val) {
FMF.setAllowReciprocal(); FMF.setAllowReciprocal();
if (0 != (Val & FastMathFlags::AllowContract)) if (0 != (Val & FastMathFlags::AllowContract))
FMF.setAllowContract(true); FMF.setAllowContract(true);
if (0 != (Val & FastMathFlags::ApproxFunc))
FMF.setApproxFunc();
return FMF; return FMF;
} }

6
lib/Bitcode/Writer/BitcodeWriter.cpp
View File

@ -1321,8 +1321,8 @@ static uint64_t getOptimizationFlags(const Value *V) {
if (PEO->isExact()) if (PEO->isExact())
Flags |= 1 << bitc::PEO_EXACT; Flags |= 1 << bitc::PEO_EXACT;
} else if (const auto *FPMO = dyn_cast<FPMathOperator>(V)) { } else if (const auto *FPMO = dyn_cast<FPMathOperator>(V)) {
if (FPMO->hasUnsafeAlgebra()) if (FPMO->hasAllowReassoc())
Flags |= FastMathFlags::UnsafeAlgebra; Flags |= FastMathFlags::AllowReassoc;
if (FPMO->hasNoNaNs()) if (FPMO->hasNoNaNs())
Flags |= FastMathFlags::NoNaNs; Flags |= FastMathFlags::NoNaNs;
if (FPMO->hasNoInfs()) if (FPMO->hasNoInfs())
@ -1333,6 +1333,8 @@ static uint64_t getOptimizationFlags(const Value *V) {
Flags |= FastMathFlags::AllowReciprocal; Flags |= FastMathFlags::AllowReciprocal;
if (FPMO->hasAllowContract()) if (FPMO->hasAllowContract())
Flags |= FastMathFlags::AllowContract; Flags |= FastMathFlags::AllowContract;
if (FPMO->hasApproxFunc())
Flags |= FastMathFlags::ApproxFunc;
} }
return Flags; return Flags;

2
lib/CodeGen/ExpandReductions.cpp
View File

@ -95,7 +95,7 @@ bool expandReductions(Function &F, const TargetTransformInfo *TTI) {
// and it can't be handled by generating this shuffle sequence. // and it can't be handled by generating this shuffle sequence.
// TODO: Implement scalarization of ordered reductions here for targets // TODO: Implement scalarization of ordered reductions here for targets
// without native support. // without native support.
if (!II->getFastMathFlags().unsafeAlgebra()) if (!II->getFastMathFlags().isFast())
continue; continue;
Vec = II->getArgOperand(1); Vec = II->getArgOperand(1);
break; break;

10
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
View File

@ -2585,7 +2585,7 @@ static bool isVectorReductionOp(const User *I) {
case Instruction::FAdd: case Instruction::FAdd:
case Instruction::FMul: case Instruction::FMul:
if (const FPMathOperator *FPOp = dyn_cast<const FPMathOperator>(Inst)) if (const FPMathOperator *FPOp = dyn_cast<const FPMathOperator>(Inst))
if (FPOp->getFastMathFlags().unsafeAlgebra()) if (FPOp->getFastMathFlags().isFast())
break; break;
LLVM_FALLTHROUGH; LLVM_FALLTHROUGH;
default: default:
@ -2631,7 +2631,7 @@ static bool isVectorReductionOp(const User *I) {
if (Inst->getOpcode() == OpCode || isa<PHINode>(U)) { if (Inst->getOpcode() == OpCode || isa<PHINode>(U)) {
if (const FPMathOperator *FPOp = dyn_cast<const FPMathOperator>(Inst)) if (const FPMathOperator *FPOp = dyn_cast<const FPMathOperator>(Inst))
if (!isa<PHINode>(FPOp) && !FPOp->getFastMathFlags().unsafeAlgebra()) if (!isa<PHINode>(FPOp) && !FPOp->getFastMathFlags().isFast())
return false; return false;
UsersToVisit.push_back(U); UsersToVisit.push_back(U);
} else if (const ShuffleVectorInst *ShufInst = } else if (const ShuffleVectorInst *ShufInst =
@ -2725,7 +2725,7 @@ void SelectionDAGBuilder::visitBinary(const User &I, unsigned OpCode) {
Flags.setNoInfs(FMF.noInfs()); Flags.setNoInfs(FMF.noInfs());
Flags.setNoNaNs(FMF.noNaNs()); Flags.setNoNaNs(FMF.noNaNs());
Flags.setNoSignedZeros(FMF.noSignedZeros()); Flags.setNoSignedZeros(FMF.noSignedZeros());
Flags.setUnsafeAlgebra(FMF.unsafeAlgebra()); Flags.setUnsafeAlgebra(FMF.isFast());
SDValue BinNodeValue = DAG.getNode(OpCode, getCurSDLoc(), Op1.getValueType(), SDValue BinNodeValue = DAG.getNode(OpCode, getCurSDLoc(), Op1.getValueType(),
Op1, Op2, Flags); Op1, Op2, Flags);
@ -7959,13 +7959,13 @@ void SelectionDAGBuilder::visitVectorReduce(const CallInst &I,
switch (Intrinsic) { switch (Intrinsic) {
case Intrinsic::experimental_vector_reduce_fadd: case Intrinsic::experimental_vector_reduce_fadd:
if (FMF.unsafeAlgebra()) if (FMF.isFast())
Res = DAG.getNode(ISD::VECREDUCE_FADD, dl, VT, Op2); Res = DAG.getNode(ISD::VECREDUCE_FADD, dl, VT, Op2);
else else
Res = DAG.getNode(ISD::VECREDUCE_STRICT_FADD, dl, VT, Op1, Op2); Res = DAG.getNode(ISD::VECREDUCE_STRICT_FADD, dl, VT, Op1, Op2);
break; break;
case Intrinsic::experimental_vector_reduce_fmul: case Intrinsic::experimental_vector_reduce_fmul:
if (FMF.unsafeAlgebra()) if (FMF.isFast())
Res = DAG.getNode(ISD::VECREDUCE_FMUL, dl, VT, Op2); Res = DAG.getNode(ISD::VECREDUCE_FMUL, dl, VT, Op2);
else else
Res = DAG.getNode(ISD::VECREDUCE_STRICT_FMUL, dl, VT, Op1, Op2); Res = DAG.getNode(ISD::VECREDUCE_STRICT_FMUL, dl, VT, Op1, Op2);

8
lib/IR/AsmWriter.cpp
View File

@ -1108,10 +1108,12 @@ static void writeAtomicRMWOperation(raw_ostream &Out,
static void WriteOptimizationInfo(raw_ostream &Out, const User *U) { static void WriteOptimizationInfo(raw_ostream &Out, const User *U) {
if (const FPMathOperator *FPO = dyn_cast<const FPMathOperator>(U)) { if (const FPMathOperator *FPO = dyn_cast<const FPMathOperator>(U)) {
// Unsafe algebra implies all the others, no need to write them all out // 'Fast' is an abbreviation for all fast-math-flags.
if (FPO->hasUnsafeAlgebra()) if (FPO->isFast())
Out << " fast"; Out << " fast";
else { else {
if (FPO->hasAllowReassoc())
Out << " reassoc";
if (FPO->hasNoNaNs()) if (FPO->hasNoNaNs())
Out << " nnan"; Out << " nnan";
if (FPO->hasNoInfs()) if (FPO->hasNoInfs())
@ -1122,6 +1124,8 @@ static void WriteOptimizationInfo(raw_ostream &Out, const User *U) {
Out << " arcp"; Out << " arcp";
if (FPO->hasAllowContract()) if (FPO->hasAllowContract())
Out << " contract"; Out << " contract";
if (FPO->hasApproxFunc())
Out << " afn";
} }
} }

30
lib/IR/Instruction.cpp
View File

@ -146,9 +146,14 @@ bool Instruction::isExact() const {
return cast<PossiblyExactOperator>(this)->isExact(); return cast<PossiblyExactOperator>(this)->isExact();
} }
void Instruction::setHasUnsafeAlgebra(bool B) { void Instruction::setFast(bool B) {
assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op"); assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
cast<FPMathOperator>(this)->setHasUnsafeAlgebra(B); cast<FPMathOperator>(this)->setFast(B);
}
void Instruction::setHasAllowReassoc(bool B) {
assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
cast<FPMathOperator>(this)->setHasAllowReassoc(B);
} }
void Instruction::setHasNoNaNs(bool B) { void Instruction::setHasNoNaNs(bool B) {
@ -171,6 +176,11 @@ void Instruction::setHasAllowReciprocal(bool B) {
cast<FPMathOperator>(this)->setHasAllowReciprocal(B); cast<FPMathOperator>(this)->setHasAllowReciprocal(B);
} }
void Instruction::setHasApproxFunc(bool B) {
assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
cast<FPMathOperator>(this)->setHasApproxFunc(B);
}
void Instruction::setFastMathFlags(FastMathFlags FMF) { void Instruction::setFastMathFlags(FastMathFlags FMF) {
assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op"); assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
cast<FPMathOperator>(this)->setFastMathFlags(FMF); cast<FPMathOperator>(this)->setFastMathFlags(FMF);
@ -181,9 +191,14 @@ void Instruction::copyFastMathFlags(FastMathFlags FMF) {
cast<FPMathOperator>(this)->copyFastMathFlags(FMF); cast<FPMathOperator>(this)->copyFastMathFlags(FMF);
} }
bool Instruction::hasUnsafeAlgebra() const { bool Instruction::isFast() const {
assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op"); assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op");
return cast<FPMathOperator>(this)->hasUnsafeAlgebra(); return cast<FPMathOperator>(this)->isFast();
}
bool Instruction::hasAllowReassoc() const {
assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op");
return cast<FPMathOperator>(this)->hasAllowReassoc();
} }
bool Instruction::hasNoNaNs() const { bool Instruction::hasNoNaNs() const {
@ -211,6 +226,11 @@ bool Instruction::hasAllowContract() const {
return cast<FPMathOperator>(this)->hasAllowContract(); return cast<FPMathOperator>(this)->hasAllowContract();
} }
bool Instruction::hasApproxFunc() const {
assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op");
return cast<FPMathOperator>(this)->hasApproxFunc();
}
FastMathFlags Instruction::getFastMathFlags() const { FastMathFlags Instruction::getFastMathFlags() const {
assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op"); assert(isa<FPMathOperator>(this) && "getting fast-math flag on invalid op");
return cast<FPMathOperator>(this)->getFastMathFlags(); return cast<FPMathOperator>(this)->getFastMathFlags();
@ -579,7 +599,7 @@ bool Instruction::isAssociative() const {
switch (Opcode) { switch (Opcode) {
case FMul: case FMul:
case FAdd: case FAdd:
return cast<FPMathOperator>(this)->hasUnsafeAlgebra(); return cast<FPMathOperator>(this)->isFast();
default: default:
return false; return false;
} }

2
lib/Target/AMDGPU/AMDGPUCodeGenPrepare.cpp
View File

@ -400,7 +400,7 @@ bool AMDGPUCodeGenPrepare::visitFDiv(BinaryOperator &FDiv) {
return false; return false;
FastMathFlags FMF = FPOp->getFastMathFlags(); FastMathFlags FMF = FPOp->getFastMathFlags();
bool UnsafeDiv = HasUnsafeFPMath || FMF.unsafeAlgebra() || bool UnsafeDiv = HasUnsafeFPMath || FMF.isFast() ||
FMF.allowReciprocal(); FMF.allowReciprocal();
// With UnsafeDiv node will be optimized to just rcp and mul. // With UnsafeDiv node will be optimized to just rcp and mul.

2
lib/Target/AMDGPU/AMDGPULibCalls.cpp
View File

@ -487,7 +487,7 @@ bool AMDGPULibCalls::parseFunctionName(const StringRef& FMangledName,
bool AMDGPULibCalls::isUnsafeMath(const CallInst *CI) const { bool AMDGPULibCalls::isUnsafeMath(const CallInst *CI) const {
if (auto Op = dyn_cast<FPMathOperator>(CI)) if (auto Op = dyn_cast<FPMathOperator>(CI))
if (Op->hasUnsafeAlgebra()) if (Op->isFast())
return true; return true;
const Function *F = CI->getParent()->getParent(); const Function *F = CI->getParent()->getParent();
Attribute Attr = F->getFnAttribute("unsafe-fp-math"); Attribute Attr = F->getFnAttribute("unsafe-fp-math");

8
lib/Transforms/InstCombine/InstCombineAddSub.cpp
View File

@ -482,7 +482,7 @@ Value *FAddCombine::performFactorization(Instruction *I) {
return nullptr; return nullptr;
FastMathFlags Flags; FastMathFlags Flags;
Flags.setUnsafeAlgebra(); Flags.setFast();
if (I0) Flags &= I->getFastMathFlags(); if (I0) Flags &= I->getFastMathFlags();
if (I1) Flags &= I->getFastMathFlags(); if (I1) Flags &= I->getFastMathFlags();
@ -511,7 +511,7 @@ Value *FAddCombine::performFactorization(Instruction *I) {
} }
Value *FAddCombine::simplify(Instruction *I) { Value *FAddCombine::simplify(Instruction *I) {
assert(I->hasUnsafeAlgebra() && "Should be in unsafe mode"); assert(I->isFast() && "Expected 'fast' instruction");
// Currently we are not able to handle vector type. // Currently we are not able to handle vector type.
if (I->getType()->isVectorTy()) if (I->getType()->isVectorTy())
@ -1386,7 +1386,7 @@ Instruction *InstCombiner::visitFAdd(BinaryOperator &I) {
if (Value *V = SimplifySelectsFeedingBinaryOp(I, LHS, RHS)) if (Value *V = SimplifySelectsFeedingBinaryOp(I, LHS, RHS))
return replaceInstUsesWith(I, V); return replaceInstUsesWith(I, V);
if (I.hasUnsafeAlgebra()) { if (I.isFast()) {
if (Value *V = FAddCombine(Builder).simplify(&I)) if (Value *V = FAddCombine(Builder).simplify(&I))
return replaceInstUsesWith(I, V); return replaceInstUsesWith(I, V);
} }
@ -1736,7 +1736,7 @@ Instruction *InstCombiner::visitFSub(BinaryOperator &I) {
if (Value *V = SimplifySelectsFeedingBinaryOp(I, Op0, Op1)) if (Value *V = SimplifySelectsFeedingBinaryOp(I, Op0, Op1))
return replaceInstUsesWith(I, V); return replaceInstUsesWith(I, V);
if (I.hasUnsafeAlgebra()) { if (I.isFast()) {
if (Value *V = FAddCombine(Builder).simplify(&I)) if (Value *V = FAddCombine(Builder).simplify(&I))
return replaceInstUsesWith(I, V); return replaceInstUsesWith(I, V);
} }

2
lib/Transforms/InstCombine/InstCombineCalls.cpp
View File

@ -2017,7 +2017,7 @@ Instruction *InstCombiner::visitCallInst(CallInst &CI) {
} }
case Intrinsic::fmuladd: { case Intrinsic::fmuladd: {
// Canonicalize fast fmuladd to the separate fmul + fadd. // Canonicalize fast fmuladd to the separate fmul + fadd.
if (II->hasUnsafeAlgebra()) { if (II->isFast()) {
BuilderTy::FastMathFlagGuard Guard(Builder); BuilderTy::FastMathFlagGuard Guard(Builder);
Builder.setFastMathFlags(II->getFastMathFlags()); Builder.setFastMathFlags(II->getFastMathFlags());
Value *Mul = Builder.CreateFMul(II->getArgOperand(0), Value *Mul = Builder.CreateFMul(II->getArgOperand(0),

11
lib/Transforms/InstCombine/InstCombineMulDivRem.cpp
View File

@ -487,7 +487,7 @@ static void detectLog2OfHalf(Value *&Op, Value *&Y, IntrinsicInst *&Log2) {
IntrinsicInst *II = dyn_cast<IntrinsicInst>(Op); IntrinsicInst *II = dyn_cast<IntrinsicInst>(Op);
if (!II) if (!II)
return; return;
if (II->getIntrinsicID() != Intrinsic::log2 || !II->hasUnsafeAlgebra()) if (II->getIntrinsicID() != Intrinsic::log2 || !II->isFast())
return; return;
Log2 = II; Log2 = II;
@ -498,7 +498,8 @@ static void detectLog2OfHalf(Value *&Op, Value *&Y, IntrinsicInst *&Log2) {
Instruction *I = dyn_cast<Instruction>(OpLog2Of); Instruction *I = dyn_cast<Instruction>(OpLog2Of);
if (!I) if (!I)
return; return;
if (I->getOpcode() != Instruction::FMul || !I->hasUnsafeAlgebra())
if (I->getOpcode() != Instruction::FMul || !I->isFast())
return; return;
if (match(I->getOperand(0), m_SpecificFP(0.5))) if (match(I->getOperand(0), m_SpecificFP(0.5)))
@ -601,7 +602,7 @@ Value *InstCombiner::foldFMulConst(Instruction *FMulOrDiv, Constant *C,
} }
if (R) { if (R) {
R->setHasUnsafeAlgebra(true); R->setFast(true);
InsertNewInstWith(R, *InsertBefore); InsertNewInstWith(R, *InsertBefore);
} }
@ -622,7 +623,7 @@ Instruction *InstCombiner::visitFMul(BinaryOperator &I) {
SQ.getWithInstruction(&I))) SQ.getWithInstruction(&I)))
return replaceInstUsesWith(I, V); return replaceInstUsesWith(I, V);
bool AllowReassociate = I.hasUnsafeAlgebra(); bool AllowReassociate = I.isFast();
// Simplify mul instructions with a constant RHS. // Simplify mul instructions with a constant RHS.
if (isa<Constant>(Op1)) { if (isa<Constant>(Op1)) {
@ -1341,7 +1342,7 @@ Instruction *InstCombiner::visitFDiv(BinaryOperator &I) {
if (Instruction *R = FoldOpIntoSelect(I, SI)) if (Instruction *R = FoldOpIntoSelect(I, SI))
return R; return R;
bool AllowReassociate = I.hasUnsafeAlgebra(); bool AllowReassociate = I.isFast();
bool AllowReciprocal = I.hasAllowReciprocal(); bool AllowReciprocal = I.hasAllowReciprocal();
if (Constant *Op1C = dyn_cast<Constant>(Op1)) { if (Constant *Op1C = dyn_cast<Constant>(Op1)) {

12
lib/Transforms/Scalar/Reassociate.cpp
View File

@ -145,8 +145,7 @@ XorOpnd::XorOpnd(Value *V) {
static BinaryOperator *isReassociableOp(Value *V, unsigned Opcode) { static BinaryOperator *isReassociableOp(Value *V, unsigned Opcode) {
if (V->hasOneUse() && isa<Instruction>(V) && if (V->hasOneUse() && isa<Instruction>(V) &&
cast<Instruction>(V)->getOpcode() == Opcode && cast<Instruction>(V)->getOpcode() == Opcode &&
(!isa<FPMathOperator>(V) || (!isa<FPMathOperator>(V) || cast<Instruction>(V)->isFast()))
cast<Instruction>(V)->hasUnsafeAlgebra()))
return cast<BinaryOperator>(V); return cast<BinaryOperator>(V);
return nullptr; return nullptr;
} }
@ -156,8 +155,7 @@ static BinaryOperator *isReassociableOp(Value *V, unsigned Opcode1,
if (V->hasOneUse() && isa<Instruction>(V) && if (V->hasOneUse() && isa<Instruction>(V) &&
(cast<Instruction>(V)->getOpcode() == Opcode1 || (cast<Instruction>(V)->getOpcode() == Opcode1 ||
cast<Instruction>(V)->getOpcode() == Opcode2) && cast<Instruction>(V)->getOpcode() == Opcode2) &&
(!isa<FPMathOperator>(V) || (!isa<FPMathOperator>(V) || cast<Instruction>(V)->isFast()))
cast<Instruction>(V)->hasUnsafeAlgebra()))
return cast<BinaryOperator>(V); return cast<BinaryOperator>(V);
return nullptr; return nullptr;
} }
@ -565,7 +563,7 @@ static bool LinearizeExprTree(BinaryOperator *I,
assert((!isa<Instruction>(Op) || assert((!isa<Instruction>(Op) ||
cast<Instruction>(Op)->getOpcode() != Opcode cast<Instruction>(Op)->getOpcode() != Opcode
|| (isa<FPMathOperator>(Op) && || (isa<FPMathOperator>(Op) &&
!cast<Instruction>(Op)->hasUnsafeAlgebra())) && !cast<Instruction>(Op)->isFast())) &&
"Should have been handled above!"); "Should have been handled above!");
assert(Op->hasOneUse() && "Has uses outside the expression tree!"); assert(Op->hasOneUse() && "Has uses outside the expression tree!");
@ -2017,8 +2015,8 @@ void ReassociatePass::OptimizeInst(Instruction *I) {
if (I->isCommutative()) if (I->isCommutative())
canonicalizeOperands(I); canonicalizeOperands(I);
// Don't optimize floating point instructions that don't have unsafe algebra. // Don't optimize floating-point instructions unless they are 'fast'.
if (I->getType()->isFPOrFPVectorTy() && !I->hasUnsafeAlgebra()) if (I->getType()->isFPOrFPVectorTy() && !I->isFast())
return; return;
// Do not reassociate boolean (i1) expressions. We want to preserve the // Do not reassociate boolean (i1) expressions. We want to preserve the

12
lib/Transforms/Utils/LoopUtils.cpp
View File

@ -432,7 +432,7 @@ RecurrenceDescriptor::isRecurrenceInstr(Instruction *I, RecurrenceKind Kind,
InstDesc &Prev, bool HasFunNoNaNAttr) { InstDesc &Prev, bool HasFunNoNaNAttr) {
bool FP = I->getType()->isFloatingPointTy(); bool FP = I->getType()->isFloatingPointTy();
Instruction *UAI = Prev.getUnsafeAlgebraInst(); Instruction *UAI = Prev.getUnsafeAlgebraInst();
if (!UAI && FP && !I->hasUnsafeAlgebra()) if (!UAI && FP && !I->isFast())
UAI = I; // Found an unsafe (unvectorizable) algebra instruction. UAI = I; // Found an unsafe (unvectorizable) algebra instruction.
switch (I->getOpcode()) { switch (I->getOpcode()) {
@ -660,11 +660,11 @@ Value *RecurrenceDescriptor::createMinMaxOp(IRBuilder<> &Builder,
break; break;
} }
// We only match FP sequences with unsafe algebra, so we can unconditionally // We only match FP sequences that are 'fast', so we can unconditionally
// set it on any generated instructions. // set it on any generated instructions.
IRBuilder<>::FastMathFlagGuard FMFG(Builder); IRBuilder<>::FastMathFlagGuard FMFG(Builder);
FastMathFlags FMF; FastMathFlags FMF;
FMF.setUnsafeAlgebra(); FMF.setFast();
Builder.setFastMathFlags(FMF); Builder.setFastMathFlags(FMF);
Value *Cmp; Value *Cmp;
@ -768,7 +768,7 @@ Value *InductionDescriptor::transform(IRBuilder<> &B, Value *Index,
// Floating point operations had to be 'fast' to enable the induction. // Floating point operations had to be 'fast' to enable the induction.
FastMathFlags Flags; FastMathFlags Flags;
Flags.setUnsafeAlgebra(); Flags.setFast();
Value *MulExp = B.CreateFMul(StepValue, Index); Value *MulExp = B.CreateFMul(StepValue, Index);
if (isa<Instruction>(MulExp)) if (isa<Instruction>(MulExp))
@ -1338,7 +1338,7 @@ Optional<unsigned> llvm::getLoopEstimatedTripCount(Loop *L) {
static Value *addFastMathFlag(Value *V) { static Value *addFastMathFlag(Value *V) {
if (isa<FPMathOperator>(V)) { if (isa<FPMathOperator>(V)) {
FastMathFlags Flags; FastMathFlags Flags;
Flags.setUnsafeAlgebra(); Flags.setFast();
cast<Instruction>(V)->setFastMathFlags(Flags); cast<Instruction>(V)->setFastMathFlags(Flags);
} }
return V; return V;
@ -1401,7 +1401,7 @@ Value *llvm::createSimpleTargetReduction(
RD::MinMaxRecurrenceKind MinMaxKind = RD::MRK_Invalid; RD::MinMaxRecurrenceKind MinMaxKind = RD::MRK_Invalid;
// TODO: Support creating ordered reductions. // TODO: Support creating ordered reductions.
FastMathFlags FMFUnsafe; FastMathFlags FMFUnsafe;
FMFUnsafe.setUnsafeAlgebra(); FMFUnsafe.setFast();
switch (Opcode) { switch (Opcode) {
case Instruction::Add: case Instruction::Add:

35
lib/Transforms/Utils/SimplifyLibCalls.cpp
View File

@ -1111,7 +1111,7 @@ Value *LibCallSimplifier::optimizePow(CallInst *CI, IRBuilder<> &B) {
// Example: x = 1000, y = 0.001. // Example: x = 1000, y = 0.001.
// pow(exp(x), y) = pow(inf, 0.001) = inf, whereas exp(x*y) = exp(1). // pow(exp(x), y) = pow(inf, 0.001) = inf, whereas exp(x*y) = exp(1).
auto *OpC = dyn_cast<CallInst>(Op1); auto *OpC = dyn_cast<CallInst>(Op1);
if (OpC && OpC->hasUnsafeAlgebra() && CI->hasUnsafeAlgebra()) { if (OpC && OpC->isFast() && CI->isFast()) {
LibFunc Func; LibFunc Func;
Function *OpCCallee = OpC->getCalledFunction(); Function *OpCCallee = OpC->getCalledFunction();
if (OpCCallee && TLI->getLibFunc(OpCCallee->getName(), Func) && if (OpCCallee && TLI->getLibFunc(OpCCallee->getName(), Func) &&
@ -1136,7 +1136,7 @@ Value *LibCallSimplifier::optimizePow(CallInst *CI, IRBuilder<> &B) {
LibFunc_sqrtl)) { LibFunc_sqrtl)) {
// If -ffast-math: // If -ffast-math:
// pow(x, -0.5) -> 1.0 / sqrt(x) // pow(x, -0.5) -> 1.0 / sqrt(x)
if (CI->hasUnsafeAlgebra()) { if (CI->isFast()) {
IRBuilder<>::FastMathFlagGuard Guard(B); IRBuilder<>::FastMathFlagGuard Guard(B);
B.setFastMathFlags(CI->getFastMathFlags()); B.setFastMathFlags(CI->getFastMathFlags());
@ -1157,7 +1157,7 @@ Value *LibCallSimplifier::optimizePow(CallInst *CI, IRBuilder<> &B) {
LibFunc_sqrtl)) { LibFunc_sqrtl)) {
// In -ffast-math, pow(x, 0.5) -> sqrt(x). // In -ffast-math, pow(x, 0.5) -> sqrt(x).
if (CI->hasUnsafeAlgebra()) { if (CI->isFast()) {
IRBuilder<>::FastMathFlagGuard Guard(B); IRBuilder<>::FastMathFlagGuard Guard(B);
B.setFastMathFlags(CI->getFastMathFlags()); B.setFastMathFlags(CI->getFastMathFlags());
@ -1196,7 +1196,7 @@ Value *LibCallSimplifier::optimizePow(CallInst *CI, IRBuilder<> &B) {
return B.CreateFDiv(ConstantFP::get(CI->getType(), 1.0), Op1, "powrecip"); return B.CreateFDiv(ConstantFP::get(CI->getType(), 1.0), Op1, "powrecip");
// In -ffast-math, generate repeated fmul instead of generating pow(x, n). // In -ffast-math, generate repeated fmul instead of generating pow(x, n).
if (CI->hasUnsafeAlgebra()) { if (CI->isFast()) {
APFloat V = abs(Op2C->getValueAPF()); APFloat V = abs(Op2C->getValueAPF());
// We limit to a max of 7 fmul(s). Thus max exponent is 32. // We limit to a max of 7 fmul(s). Thus max exponent is 32.
// This transformation applies to integer exponents only. // This transformation applies to integer exponents only.
@ -1284,9 +1284,9 @@ Value *LibCallSimplifier::optimizeFMinFMax(CallInst *CI, IRBuilder<> &B) {
IRBuilder<>::FastMathFlagGuard Guard(B); IRBuilder<>::FastMathFlagGuard Guard(B);
FastMathFlags FMF; FastMathFlags FMF;
if (CI->hasUnsafeAlgebra()) { if (CI->isFast()) {
// Unsafe algebra sets all fast-math-flags to true. // If the call is 'fast', then anything we create here will also be 'fast'.
FMF.setUnsafeAlgebra(); FMF.setFast();
} else { } else {
// At a minimum, no-nans-fp-math must be true. // At a minimum, no-nans-fp-math must be true.
if (!CI->hasNoNaNs()) if (!CI->hasNoNaNs())
@ -1317,13 +1317,13 @@ Value *LibCallSimplifier::optimizeLog(CallInst *CI, IRBuilder<> &B) {
if (UnsafeFPShrink && hasFloatVersion(Name)) if (UnsafeFPShrink && hasFloatVersion(Name))
Ret = optimizeUnaryDoubleFP(CI, B, true); Ret = optimizeUnaryDoubleFP(CI, B, true);
if (!CI->hasUnsafeAlgebra()) if (!CI->isFast())
return Ret; return Ret;
Value *Op1 = CI->getArgOperand(0); Value *Op1 = CI->getArgOperand(0);
auto *OpC = dyn_cast<CallInst>(Op1); auto *OpC = dyn_cast<CallInst>(Op1);
// The earlier call must also be unsafe in order to do these transforms. // The earlier call must also be 'fast' in order to do these transforms.
if (!OpC || !OpC->hasUnsafeAlgebra()) if (!OpC || !OpC->isFast())
return Ret; return Ret;
// log(pow(x,y)) -> y*log(x) // log(pow(x,y)) -> y*log(x)
@ -1333,7 +1333,7 @@ Value *LibCallSimplifier::optimizeLog(CallInst *CI, IRBuilder<> &B) {
IRBuilder<>::FastMathFlagGuard Guard(B); IRBuilder<>::FastMathFlagGuard Guard(B);
FastMathFlags FMF; FastMathFlags FMF;
FMF.setUnsafeAlgebra(); FMF.setFast();
B.setFastMathFlags(FMF); B.setFastMathFlags(FMF);
LibFunc Func; LibFunc Func;
@ -1365,11 +1365,11 @@ Value *LibCallSimplifier::optimizeSqrt(CallInst *CI, IRBuilder<> &B) {
Callee->getIntrinsicID() == Intrinsic::sqrt)) Callee->getIntrinsicID() == Intrinsic::sqrt))
Ret = optimizeUnaryDoubleFP(CI, B, true); Ret = optimizeUnaryDoubleFP(CI, B, true);
if (!CI->hasUnsafeAlgebra()) if (!CI->isFast())
return Ret; return Ret;
Instruction *I = dyn_cast<Instruction>(CI->getArgOperand(0)); Instruction *I = dyn_cast<Instruction>(CI->getArgOperand(0));
if (!I || I->getOpcode() != Instruction::FMul || !I->hasUnsafeAlgebra()) if (!I || I->getOpcode() != Instruction::FMul || !I->isFast())
return Ret; return Ret;
// We're looking for a repeated factor in a multiplication tree, // We're looking for a repeated factor in a multiplication tree,
@ -1391,8 +1391,7 @@ Value *LibCallSimplifier::optimizeSqrt(CallInst *CI, IRBuilder<> &B) {
Value *OtherMul0, *OtherMul1; Value *OtherMul0, *OtherMul1;
if (match(Op0, m_FMul(m_Value(OtherMul0), m_Value(OtherMul1)))) { if (match(Op0, m_FMul(m_Value(OtherMul0), m_Value(OtherMul1)))) {
// Pattern: sqrt((x * y) * z) // Pattern: sqrt((x * y) * z)
if (OtherMul0 == OtherMul1 && if (OtherMul0 == OtherMul1 && cast<Instruction>(Op0)->isFast()) {
cast<Instruction>(Op0)->hasUnsafeAlgebra()) {
// Matched: sqrt((x * x) * z) // Matched: sqrt((x * x) * z)
RepeatOp = OtherMul0; RepeatOp = OtherMul0;
OtherOp = Op1; OtherOp = Op1;
@ -1437,8 +1436,8 @@ Value *LibCallSimplifier::optimizeTan(CallInst *CI, IRBuilder<> &B) {
if (!OpC) if (!OpC)
return Ret; return Ret;
// Both calls must allow unsafe optimizations in order to remove them. // Both calls must be 'fast' in order to remove them.
if (!CI->hasUnsafeAlgebra() || !OpC->hasUnsafeAlgebra()) if (!CI->isFast() || !OpC->isFast())
return Ret; return Ret;
// tan(atan(x)) -> x // tan(atan(x)) -> x
@ -2170,7 +2169,7 @@ Value *LibCallSimplifier::optimizeCall(CallInst *CI) {
// used by the intrinsic optimizations. // used by the intrinsic optimizations.
if (EnableUnsafeFPShrink.getNumOccurrences() > 0) if (EnableUnsafeFPShrink.getNumOccurrences() > 0)
UnsafeFPShrink = EnableUnsafeFPShrink; UnsafeFPShrink = EnableUnsafeFPShrink;
else if (isa<FPMathOperator>(CI) && CI->hasUnsafeAlgebra()) else if (isa<FPMathOperator>(CI) && CI->isFast())
UnsafeFPShrink = true; UnsafeFPShrink = true;
// First, check for intrinsics. // First, check for intrinsics.

6
lib/Transforms/Vectorize/LoopVectorize.cpp
View File

@ -385,7 +385,7 @@ static unsigned getReciprocalPredBlockProb() { return 2; }
static Value *addFastMathFlag(Value *V) { static Value *addFastMathFlag(Value *V) {
if (isa<FPMathOperator>(V)) { if (isa<FPMathOperator>(V)) {
FastMathFlags Flags; FastMathFlags Flags;
Flags.setUnsafeAlgebra(); Flags.setFast();
cast<Instruction>(V)->setFastMathFlags(Flags); cast<Instruction>(V)->setFastMathFlags(Flags);
} }
return V; return V;
@ -2720,7 +2720,7 @@ Value *InnerLoopVectorizer::getStepVector(Value *Val, int StartIdx, Value *Step,
// Floating point operations had to be 'fast' to enable the induction. // Floating point operations had to be 'fast' to enable the induction.
FastMathFlags Flags; FastMathFlags Flags;
Flags.setUnsafeAlgebra(); Flags.setFast();
Value *MulOp = Builder.CreateFMul(Cv, Step); Value *MulOp = Builder.CreateFMul(Cv, Step);
if (isa<Instruction>(MulOp)) if (isa<Instruction>(MulOp))
@ -5396,7 +5396,7 @@ bool LoopVectorizationLegality::canVectorizeInstrs() {
// operations, shuffles, or casts, as they don't change precision or // operations, shuffles, or casts, as they don't change precision or
// semantics. // semantics.
} else if (I.getType()->isFloatingPointTy() && (CI || I.isBinaryOp()) && } else if (I.getType()->isFloatingPointTy() && (CI || I.isBinaryOp()) &&
!I.hasUnsafeAlgebra()) { !I.isFast()) {
DEBUG(dbgs() << "LV: Found FP op with unsafe algebra.\n"); DEBUG(dbgs() << "LV: Found FP op with unsafe algebra.\n");
Hints->setPotentiallyUnsafe(); Hints->setPotentiallyUnsafe();
} }

4
lib/Transforms/Vectorize/SLPVectorizer.cpp
View File

@ -4880,7 +4880,7 @@ class HorizontalReduction {
case RK_Min: case RK_Min:
case RK_Max: case RK_Max:
return Opcode == Instruction::ICmp || return Opcode == Instruction::ICmp ||
cast<Instruction>(I->getOperand(0))->hasUnsafeAlgebra(); cast<Instruction>(I->getOperand(0))->isFast();
case RK_UMin: case RK_UMin:
case RK_UMax: case RK_UMax:
assert(Opcode == Instruction::ICmp && assert(Opcode == Instruction::ICmp &&
@ -5232,7 +5232,7 @@ public:
Value *VectorizedTree = nullptr; Value *VectorizedTree = nullptr;
IRBuilder<> Builder(ReductionRoot); IRBuilder<> Builder(ReductionRoot);
FastMathFlags Unsafe; FastMathFlags Unsafe;
Unsafe.setUnsafeAlgebra(); Unsafe.setFast();
Builder.setFastMathFlags(Unsafe); Builder.setFastMathFlags(Unsafe);
unsigned i = 0; unsigned i = 0;

32
test/Assembler/fast-math-flags.ll
View File

@ -7,6 +7,8 @@
@vec = external global <3 x float> @vec = external global <3 x float>
@arr = external global [3 x float] @arr = external global [3 x float]
declare float @foo(float)
define float @none(float %x, float %y) { define float @none(float %x, float %y) {
entry: entry:
; CHECK: %vec = load <3 x float>, <3 x float>* @vec ; CHECK: %vec = load <3 x float>, <3 x float>* @vec
@ -86,6 +88,28 @@ entry:
ret float %c ret float %c
} }
; CHECK: @reassoc(
define float @reassoc(float %x, float %y) {
; CHECK: %a = fsub reassoc float %x, %y
%a = fsub reassoc float %x, %y
; CHECK: %b = fmul reassoc float %x, %y
%b = fmul reassoc float %x, %y
; CHECK: %c = call reassoc float @foo(float %b)
%c = call reassoc float @foo(float %b)
ret float %c
}
; CHECK: @afn(
define float @afn(float %x, float %y) {
; CHECK: %a = fdiv afn float %x, %y
%a = fdiv afn float %x, %y
; CHECK: %b = frem afn float %x, %y
%b = frem afn float %x, %y
; CHECK: %c = call afn float @foo(float %b)
%c = call afn float @foo(float %b)
ret float %c
}
; CHECK: no_nan_inf ; CHECK: no_nan_inf
define float @no_nan_inf(float %x, float %y) { define float @no_nan_inf(float %x, float %y) {
entry: entry:
@ -130,10 +154,10 @@ entry:
; CHECK: %arr = load [3 x float], [3 x float]* @arr ; CHECK: %arr = load [3 x float], [3 x float]* @arr
%arr = load [3 x float], [3 x float]* @arr %arr = load [3 x float], [3 x float]* @arr
; CHECK: %a = fadd nnan ninf float %x, %y ; CHECK: %a = fadd nnan ninf afn float %x, %y
%a = fadd ninf nnan float %x, %y %a = fadd ninf nnan afn float %x, %y
; CHECK: %a_vec = fadd nnan <3 x float> %vec, %vec ; CHECK: %a_vec = fadd reassoc nnan <3 x float> %vec, %vec
%a_vec = fadd nnan <3 x float> %vec, %vec %a_vec = fadd reassoc nnan <3 x float> %vec, %vec
; CHECK: %b = fsub fast float %x, %y ; CHECK: %b = fsub fast float %x, %y
%b = fsub nnan nsz fast float %x, %y %b = fsub nnan nsz fast float %x, %y
; CHECK: %b_vec = fsub nnan <3 x float> %vec, %vec ; CHECK: %b_vec = fsub nnan <3 x float> %vec, %vec

4
test/Bitcode/compatibility-3.6.ll
View File

@ -612,7 +612,9 @@ define void @fastmathflags(float %op1, float %op2) {
%f.arcp = fadd arcp float %op1, %op2 %f.arcp = fadd arcp float %op1, %op2
; CHECK: %f.arcp = fadd arcp float %op1, %op2 ; CHECK: %f.arcp = fadd arcp float %op1, %op2
%f.fast = fadd fast float %op1, %op2 %f.fast = fadd fast float %op1, %op2
; CHECK: %f.fast = fadd fast float %op1, %op2 ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'contract' and 'afn' bits set, so this is not fully 'fast'.
; CHECK: %f.fast = fadd reassoc nnan ninf nsz arcp float %op1, %op2
ret void ret void
} }

4
test/Bitcode/compatibility-3.7.ll
View File

@ -656,7 +656,9 @@ define void @fastmathflags(float %op1, float %op2) {
%f.arcp = fadd arcp float %op1, %op2 %f.arcp = fadd arcp float %op1, %op2
; CHECK: %f.arcp = fadd arcp float %op1, %op2 ; CHECK: %f.arcp = fadd arcp float %op1, %op2
%f.fast = fadd fast float %op1, %op2 %f.fast = fadd fast float %op1, %op2
; CHECK: %f.fast = fadd fast float %op1, %op2 ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'contract' and 'afn' bits set, so this is not fully 'fast'.
; CHECK: %f.fast = fadd reassoc nnan ninf nsz arcp float %op1, %op2
ret void ret void
} }

8
test/Bitcode/compatibility-3.8.ll
View File

@ -687,7 +687,9 @@ define void @fastmathflags(float %op1, float %op2) {
%f.arcp = fadd arcp float %op1, %op2 %f.arcp = fadd arcp float %op1, %op2
; CHECK: %f.arcp = fadd arcp float %op1, %op2 ; CHECK: %f.arcp = fadd arcp float %op1, %op2
%f.fast = fadd fast float %op1, %op2 %f.fast = fadd fast float %op1, %op2
; CHECK: %f.fast = fadd fast float %op1, %op2 ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'contract' and 'afn' bits set, so this is not fully 'fast'.
; CHECK: %f.fast = fadd reassoc nnan ninf nsz arcp float %op1, %op2
ret void ret void
} }
@ -700,7 +702,9 @@ declare <4 x double> @fmf3()
; CHECK-LABEL: fastMathFlagsForCalls( ; CHECK-LABEL: fastMathFlagsForCalls(
define void @fastMathFlagsForCalls(float %f, double %d1, <4 x double> %d2) { define void @fastMathFlagsForCalls(float %f, double %d1, <4 x double> %d2) {
%call.fast = call fast float @fmf1() %call.fast = call fast float @fmf1()
; CHECK: %call.fast = call fast float @fmf1() ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'contract' and 'aml' bits set, so this is not fully 'fast'.
; CHECK: %call.fast = call reassoc nnan ninf nsz arcp float @fmf1()
; Throw in some other attributes to make sure those stay in the right places. ; Throw in some other attributes to make sure those stay in the right places.

8
test/Bitcode/compatibility-3.9.ll
View File

@ -758,7 +758,9 @@ define void @fastmathflags(float %op1, float %op2) {
%f.arcp = fadd arcp float %op1, %op2 %f.arcp = fadd arcp float %op1, %op2
; CHECK: %f.arcp = fadd arcp float %op1, %op2 ; CHECK: %f.arcp = fadd arcp float %op1, %op2
%f.fast = fadd fast float %op1, %op2 %f.fast = fadd fast float %op1, %op2
; CHECK: %f.fast = fadd fast float %op1, %op2 ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'contract' and 'afn' bits set, so this is not fully 'fast'.
; CHECK: %f.fast = fadd reassoc nnan ninf nsz arcp float %op1, %op2
ret void ret void
} }
@ -771,7 +773,9 @@ declare <4 x double> @fmf3()
; CHECK-LABEL: fastMathFlagsForCalls( ; CHECK-LABEL: fastMathFlagsForCalls(
define void @fastMathFlagsForCalls(float %f, double %d1, <4 x double> %d2) { define void @fastMathFlagsForCalls(float %f, double %d1, <4 x double> %d2) {
%call.fast = call fast float @fmf1() %call.fast = call fast float @fmf1()
; CHECK: %call.fast = call fast float @fmf1() ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'contract' and 'afn' bits set, so this is not fully 'fast'.
; CHECK: %call.fast = call reassoc nnan ninf nsz arcp float @fmf1()
; Throw in some other attributes to make sure those stay in the right places. ; Throw in some other attributes to make sure those stay in the right places.

8
test/Bitcode/compatibility-4.0.ll
View File

@ -757,8 +757,10 @@ define void @fastmathflags(float %op1, float %op2) {
; CHECK: %f.nsz = fadd nsz float %op1, %op2 ; CHECK: %f.nsz = fadd nsz float %op1, %op2
%f.arcp = fadd arcp float %op1, %op2 %f.arcp = fadd arcp float %op1, %op2
; CHECK: %f.arcp = fadd arcp float %op1, %op2 ; CHECK: %f.arcp = fadd arcp float %op1, %op2
; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'contract' and 'afn' bits set, so this is not fully 'fast'.
%f.fast = fadd fast float %op1, %op2 %f.fast = fadd fast float %op1, %op2
; CHECK: %f.fast = fadd fast float %op1, %op2 ; CHECK: %f.fast = fadd reassoc nnan ninf nsz arcp float %op1, %op2
ret void ret void
} }
@ -771,7 +773,9 @@ declare <4 x double> @fmf3()
; CHECK-LABEL: fastMathFlagsForCalls( ; CHECK-LABEL: fastMathFlagsForCalls(
define void @fastMathFlagsForCalls(float %f, double %d1, <4 x double> %d2) { define void @fastMathFlagsForCalls(float %f, double %d1, <4 x double> %d2) {
%call.fast = call fast float @fmf1() %call.fast = call fast float @fmf1()
; CHECK: %call.fast = call fast float @fmf1() ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'contract' and 'afn' bits set, so this is not fully 'fast'.
; CHECK: %call.fast = call reassoc nnan ninf nsz arcp float @fmf1()
; Throw in some other attributes to make sure those stay in the right places. ; Throw in some other attributes to make sure those stay in the right places.

8
test/Bitcode/compatibility-5.0.ll
View File

@ -765,7 +765,9 @@ define void @fastmathflags(float %op1, float %op2) {
%f.contract = fadd contract float %op1, %op2 %f.contract = fadd contract float %op1, %op2
; CHECK: %f.contract = fadd contract float %op1, %op2 ; CHECK: %f.contract = fadd contract float %op1, %op2
%f.fast = fadd fast float %op1, %op2 %f.fast = fadd fast float %op1, %op2
; CHECK: %f.fast = fadd fast float %op1, %op2 ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'afn' bit set, so this is not fully 'fast'.
; CHECK: %f.fast = fadd reassoc nnan ninf nsz arcp contract float %op1, %op2
ret void ret void
} }
@ -778,7 +780,9 @@ declare <4 x double> @fmf3()
; CHECK-LABEL: fastMathFlagsForCalls( ; CHECK-LABEL: fastMathFlagsForCalls(
define void @fastMathFlagsForCalls(float %f, double %d1, <4 x double> %d2) { define void @fastMathFlagsForCalls(float %f, double %d1, <4 x double> %d2) {
%call.fast = call fast float @fmf1() %call.fast = call fast float @fmf1()
; CHECK: %call.fast = call fast float @fmf1() ; 'fast' used to be its own bit, but this changed in Oct 2017.
; The binary test file does not have the newer 'afn' bit set, so this is not fully 'fast'.
; CHECK: %call.fast = call reassoc nnan ninf nsz arcp contract float @fmf1()
; Throw in some other attributes to make sure those stay in the right places. ; Throw in some other attributes to make sure those stay in the right places.

4
test/Bitcode/compatibility.ll
View File

@ -775,6 +775,10 @@ define void @fastmathflags(float %op1, float %op2) {
; CHECK: %f.arcp = fadd arcp float %op1, %op2 ; CHECK: %f.arcp = fadd arcp float %op1, %op2
%f.contract = fadd contract float %op1, %op2 %f.contract = fadd contract float %op1, %op2
; CHECK: %f.contract = fadd contract float %op1, %op2 ; CHECK: %f.contract = fadd contract float %op1, %op2
%f.afn = fadd afn float %op1, %op2
; CHECK: %f.afn = fadd afn float %op1, %op2
%f.reassoc = fadd reassoc float %op1, %op2
; CHECK: %f.reassoc = fadd reassoc float %op1, %op2
%f.fast = fadd fast float %op1, %op2 %f.fast = fadd fast float %op1, %op2
; CHECK: %f.fast = fadd fast float %op1, %op2 ; CHECK: %f.fast = fadd fast float %op1, %op2
ret void ret void

49
unittests/IR/IRBuilderTest.cpp
View File

@ -144,17 +144,40 @@ TEST_F(IRBuilderTest, FastMathFlags) {
FastMathFlags FMF; FastMathFlags FMF;
Builder.setFastMathFlags(FMF); Builder.setFastMathFlags(FMF);
// By default, no flags are set.
F = Builder.CreateFAdd(F, F); F = Builder.CreateFAdd(F, F);
EXPECT_FALSE(Builder.getFastMathFlags().any()); EXPECT_FALSE(Builder.getFastMathFlags().any());
ASSERT_TRUE(isa<Instruction>(F));
FAdd = cast<Instruction>(F);
EXPECT_FALSE(FAdd->hasNoNaNs());
EXPECT_FALSE(FAdd->hasNoInfs());
EXPECT_FALSE(FAdd->hasNoSignedZeros());
EXPECT_FALSE(FAdd->hasAllowReciprocal());
EXPECT_FALSE(FAdd->hasAllowContract());
EXPECT_FALSE(FAdd->hasAllowReassoc());
EXPECT_FALSE(FAdd->hasApproxFunc());
FMF.setUnsafeAlgebra(); // Set all flags in the instruction.
FAdd->setFast(true);
EXPECT_TRUE(FAdd->hasNoNaNs());
EXPECT_TRUE(FAdd->hasNoInfs());
EXPECT_TRUE(FAdd->hasNoSignedZeros());
EXPECT_TRUE(FAdd->hasAllowReciprocal());
EXPECT_TRUE(FAdd->hasAllowContract());
EXPECT_TRUE(FAdd->hasAllowReassoc());
EXPECT_TRUE(FAdd->hasApproxFunc());
// All flags are set in the builder.
FMF.setFast();
Builder.setFastMathFlags(FMF); Builder.setFastMathFlags(FMF);
F = Builder.CreateFAdd(F, F); F = Builder.CreateFAdd(F, F);
EXPECT_TRUE(Builder.getFastMathFlags().any()); EXPECT_TRUE(Builder.getFastMathFlags().any());
EXPECT_TRUE(Builder.getFastMathFlags().all());
ASSERT_TRUE(isa<Instruction>(F)); ASSERT_TRUE(isa<Instruction>(F));
FAdd = cast<Instruction>(F); FAdd = cast<Instruction>(F);
EXPECT_TRUE(FAdd->hasNoNaNs()); EXPECT_TRUE(FAdd->hasNoNaNs());
EXPECT_TRUE(FAdd->isFast());
// Now, try it with CreateBinOp // Now, try it with CreateBinOp
F = Builder.CreateBinOp(Instruction::FAdd, F, F); F = Builder.CreateBinOp(Instruction::FAdd, F, F);
@ -162,14 +185,15 @@ TEST_F(IRBuilderTest, FastMathFlags) {
ASSERT_TRUE(isa<Instruction>(F)); ASSERT_TRUE(isa<Instruction>(F));
FAdd = cast<Instruction>(F); FAdd = cast<Instruction>(F);
EXPECT_TRUE(FAdd->hasNoNaNs()); EXPECT_TRUE(FAdd->hasNoNaNs());
EXPECT_TRUE(FAdd->isFast());
F = Builder.CreateFDiv(F, F); F = Builder.CreateFDiv(F, F);
EXPECT_TRUE(Builder.getFastMathFlags().any()); EXPECT_TRUE(Builder.getFastMathFlags().all());
EXPECT_TRUE(Builder.getFastMathFlags().UnsafeAlgebra);
ASSERT_TRUE(isa<Instruction>(F)); ASSERT_TRUE(isa<Instruction>(F));
FDiv = cast<Instruction>(F); FDiv = cast<Instruction>(F);
EXPECT_TRUE(FDiv->hasAllowReciprocal()); EXPECT_TRUE(FDiv->hasAllowReciprocal());
// Clear all FMF in the builder.
Builder.clearFastMathFlags(); Builder.clearFastMathFlags();
F = Builder.CreateFDiv(F, F); F = Builder.CreateFDiv(F, F);
@ -177,6 +201,7 @@ TEST_F(IRBuilderTest, FastMathFlags) {
FDiv = cast<Instruction>(F); FDiv = cast<Instruction>(F);
EXPECT_FALSE(FDiv->hasAllowReciprocal()); EXPECT_FALSE(FDiv->hasAllowReciprocal());
// Try individual flags.
FMF.clear(); FMF.clear();
FMF.setAllowReciprocal(); FMF.setAllowReciprocal();
Builder.setFastMathFlags(FMF); Builder.setFastMathFlags(FMF);
@ -225,7 +250,25 @@ TEST_F(IRBuilderTest, FastMathFlags) {
FAdd = cast<Instruction>(FC); FAdd = cast<Instruction>(FC);
EXPECT_TRUE(FAdd->hasAllowContract()); EXPECT_TRUE(FAdd->hasAllowContract());
FMF.setApproxFunc();
Builder.clearFastMathFlags(); Builder.clearFastMathFlags();
Builder.setFastMathFlags(FMF);
// Now 'aml' and 'contract' are set.
F = Builder.CreateFMul(F, F);
FAdd = cast<Instruction>(F);
EXPECT_TRUE(FAdd->hasApproxFunc());
EXPECT_TRUE(FAdd->hasAllowContract());
EXPECT_FALSE(FAdd->hasAllowReassoc());
FMF.setAllowReassoc();
Builder.clearFastMathFlags();
Builder.setFastMathFlags(FMF);
// Now 'aml' and 'contract' and 'reassoc' are set.
F = Builder.CreateFMul(F, F);
FAdd = cast<Instruction>(F);
EXPECT_TRUE(FAdd->hasApproxFunc());
EXPECT_TRUE(FAdd->hasAllowContract());
EXPECT_TRUE(FAdd->hasAllowReassoc());
// Test a call with FMF. // Test a call with FMF.
auto CalleeTy = FunctionType::get(Type::getFloatTy(Ctx), auto CalleeTy = FunctionType::get(Type::getFloatTy(Ctx),