We need to make sure that the value types are the same. Otherwise
we both may not have the necessary dereferenceability implication,
nor can we directly form the desired select pattern.
Without opaque pointers this is enforced implicitly through the
pointer comparison.
This is SCC pass, moving it to the end of SCC PM saves one
Function PM. This needs the analysis to take into account
memory access width since it is now places after the
load/store optimizer (D105651).
Differential Revision: https://reviews.llvm.org/D105652
These had
```
.clampScalar(0, s1, 64)
.widenScalarToNextPow2(0, 8)
```
If you have s2 or s4, then `widenScalarToNextPow2` does nothing.
This changes the `widenScalarToNextPow2` rule to use s8 as the minimum type
instead, allowing us to correctly widen s2 and s4.
This does not impact s1, since it's marked as legal already.
Differential Revision: https://reviews.llvm.org/D106413
fixed fields with highly-aligned flexible fields.
The code was not considering the possibility that aligning
the current offset to the alignment of a queue might push
us past the end of the gap. Subtracting the offsets to
figure out the maximum field size for the gap then overflowed,
making us think that we had nearly unbounded space to fill.
Fixes PR 51131.
A function with less memory instructions but wider access
is the same as a function with more but narrower accesses
in terms of memory boundness. In fact the pass would give
different answers before and after vectorization without
this change.
Differential Revision: https://reviews.llvm.org/D105651
Revert "[profile] Change linkage type of a compiler-rt func"
This reverts commits f984ac2715f71c38a7872fa2c2ad535b3d4fa285 and
467c7191249b76abff33853b1692a77f327c2422 because it broke some builds.
Generating these tests unfortunately means a lot of junk, but it's hard
to write/update these tests by hand.
Added tests focus on atomic orderings for cmpxchg.
Actually writing out these tests showed some potentially dubious
results; we should probably consider using casp for 128-bit atomic
load/store/rmw.
Manifesting AbstractAttributes may add new BBs in the IR. This patch provides an interface to register those BBs in the Attributor so that those BBs and containing instructions are not deleted as dead.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D106383
The existing rule about the operand type is strange. Instead, just say
the operand is a TargetConstant with the right width. (Legalization
ignores TargetConstants, so it doesn't matter if that width is legal.)
Highlights:
1. I had to substantially rewrite the AArch64 isel patterns to expect a
TargetConstant. Nothing too exotic, but maybe a little hairy. Maybe
worth considering a target-specific node with some dagcombines instead
of this complicated nest of isel patterns.
2. Our behavior on RV32 for vectors of i64 has changed slightly. In
particular, we correctly preserve the width of the arithmetic through
legalization. This changes the DAG a bit. Maybe room for
improvement here.
3. I explicitly defined the behavior around overflow. This is necessary
to make the DAGCombine transforms legal, and I don't think it causes any
practical issues.
Differential Revision: https://reviews.llvm.org/D105673
There is no need for a non-const argument interface and the const argument modification covers existing and upcoming use cases.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D106418
The patch exposes the libomptarget runtime function that gets the hardware thread id through the kmpc API. This is to be used in SPMDization for checking the thread id to execute regions by a single thread in a block.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D106323
Replace the experimental clang builtins and LLVM intrinsics for these
instructions with normal instruction selection patterns. The wasm_simd128.h
intrinsics header was already using portable code for the corresponding
intrinsics, so now it produces the correct instructions.
Differential Revision: https://reviews.llvm.org/D106400
ML64.EXE applies implicit RIP-relative addressing only to memory references that include a named-variable reference.
Reviewed By: mstorsjo
Differential Revision: https://reviews.llvm.org/D105372
In weird cases, the inliner will inline internal recursive functions,
sometimes causing them to have no more uses, in which case the
inliner will mark the function to be deleted. The function is
actually deleted after the call to
updateCGAndAnalysisManagerForCGSCCPass(). In
updateCGAndAnalysisManagerForCGSCCPass(), UR.UpdatedC may be set to
the SCC containing the function to be deleted. Then the inliner calls
CG.removeDeadFunction() which can cause that SCC to be deleted, even
though it's still stored in UR.UpdatedC.
We could potentially check in the wrappers/pass managers if UR.UpdatedC
is in UR.InvalidatedSCCs before doing anything with it, but it's safer
to do this as close to possible to the call to CG.removeDeadFunction()
to avoid issues with allocating a new SCC in the same address as
the deleted one.
It's hard to find a small test case since we need to have recursive
internal functions be reachable from non-internal functions, yet they
need to become non-recursive and not referenced by other functions when
inlined.
Similar to https://reviews.llvm.org/D106306.
Fixes PR50788.
Reviewed By: asbirlea
Differential Revision: https://reviews.llvm.org/D106405
Add support for all built-in text macros supported by ML64:
@Date, @Time, @FileName, @FileCur, and @CurSeg.
Reviewed By: thakis
Differential Revision: https://reviews.llvm.org/D104965
Support @Version and @Line as built-in symbols. For now, resolves @Version to 1427 (the same as for the VS 2019 release of ML.EXE).
Reviewed By: thakis
Differential Revision: https://reviews.llvm.org/D104964
This patch is in a series of patches to provide
builtins for compatibility with the XL compiler.
This patch adds builtins related to floating point
operations
Reviewed By: #powerpc, nemanjai, amyk, NeHuang
Differential Revision: https://reviews.llvm.org/D103986
Make it easier to initialize small maps inline. Note that DenseMap already has an initializer_list constructor.
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D106363
The killed flag is not always set. E.g. when a variable is used in a
loop, it is never marked as killed, although it is unused in following
basic blocks. Also, we try to deprecate kill flags and not use them.
Check if the register is live in the endif block. If not, consider it
killed in the then and else blocks.
The vgpr-liverange tests have two new tests with loops
(pre-committed, so the diff is visible).
I also needed to change the subtarget to gfx10.1, otherwise calls
are not working.
Differential Revision: https://reviews.llvm.org/D106291
This patch allows iterating typed enum via the ADT/Sequence utility.
It also changes the original design to better separate concerns:
- `StrongInt` only deals with safe `intmax_t` operations,
- `SafeIntIterator` presents the iterator and reverse iterator
interface but only deals with safe `StrongInt` internally.
- `iota_range` only deals with `SafeIntIterator` internally.
This design ensures that operations are always valid. In particular,
"Out of bounds" assertions fire when:
- the `value_type` is not representable as an `intmax_t`
- iterator operations make internal computation underflow/overflow
- the internal representation cannot be converted back to `value_type`
Differential Revision: https://reviews.llvm.org/D106279
Rename getBufferOffsetForMMO to updateBufferMMO and pass in the MMO to
be updated, in preparation for the bug fix in D106284.
Call updateBufferMMO consistently for all buffer intrinsics, even the
ones that use setBufferOffsets to decompose a combined offset
expression.
Add a getIdxEn helper function.
Differential Revision: https://reviews.llvm.org/D106354
Make getLatchCmpInst non-static and use it in LoopFlatten as a more
robust way of identifying the compare.
Differential Revision: https://reviews.llvm.org/D106256
If a reduction Phi has a single user which `AND`s the Phi with a type mask,
`lookThroughAnd` will return the user of the Phi and the narrower type represented
by the mask. Currently this is only used for arithmetic reductions, whereas loops
containing logical reductions will create a reduction intrinsic using the widened
type, for example:
for.body:
%phi = phi i32 [ %and, %for.body ], [ 255, %entry ]
%mask = and i32 %phi, 255
%gep = getelementptr inbounds i8, i8* %ptr, i32 %iv
%load = load i8, i8* %gep
%ext = zext i8 %load to i32
%and = and i32 %mask, %ext
...
^ this will generate an and reduction intrinsic such as the following:
call i32 @llvm.vector.reduce.and.v8i32(<8 x i32>...)
The same example for an add instruction would create an intrinsic of type i8:
call i8 @llvm.vector.reduce.add.v8i8(<8 x i8>...)
This patch changes AddReductionVar to call lookThroughAnd for other integer
reductions, allowing loops similar to the example above with reductions such
as and, or & xor to vectorize.
Reviewed By: david-arm, dmgreen
Differential Revision: https://reviews.llvm.org/D105632
In mandatory tail calling conventions we might have to deallocate stack
space used by our arguments before return. This happens after popping
CSRs, so the pop cannot be turned into the return itself in this case.
The else branch here was already a nop, so removing it as a tidy-up.
We have SelectionDAG patterns for 8 & 16-bit atomic operations, but they
assume the value types will have been legalized to 32-bits. So this adds
the ability to widen them to both AArch64 & generic GISel
infrastructure.
This patch adds the mova instruction to insert/extract an SVE vector
register to/from a ZA tile vector.
The preferred MOV aliases are also implemented.
Depends on D105572.
The reference can be found here:
https://developer.arm.com/documentation/ddi0602/2021-06
Reviewed By: david-arm, CarolineConcatto
Differential Revision: https://reviews.llvm.org/D105574
This reverts commit d4abdefc998a1ee19d5edc79ec233774cbf64f6a ("[ORC-RT] Rename
macho_tlv.x86-64.s to macho_tlv.x86-64.S (uppercase suffix)", and
a7733e9556b5a6334c910f88bcd037e84e17e3fc ("Re-apply "[ORC][ORC-RT] Add initial
native-TLV support to MachOPlatform."), while I investigate failures on
ccache builders (e.g. https://lab.llvm.org/buildbot/#/builders/109/builds/18981)
Reapplies fe1fa43f16beac1506a2e73a9f7b3c81179744eb, which was reverted in
6d8c63946cc259c0af02584b7cc690dde11dea35, with fixes:
1. Remove .subsections_via_symbols directive from macho_tlv.x86-64.s (it's
not needed here anyway).
2. Return error from pthread_key_create to the MachOPlatform to silence unused
variable warning.
If a CMOV is in a loop and is converted to branches, CMOV conversion wouldn't
add newly created basic blocks to loop info. Since the candidates is collected
based on loops, instructions in these basic blocks will be ignored.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D104623
