Asaf Badouh
547a7d4edb
[X86][AVX512] small fix in ptestm intrinsics
...
move ptestm{q|d} intrinsics from patterns form (in td file) to the intrinsics table
Differential Revision: http://reviews.llvm.org/D16633
llvm-svn: 259029
2016-01-28 08:33:22 +00:00
Igor Breger
04cebad2fa
AVX512: Add store mask patterns.
...
Differential Revision: http://reviews.llvm.org/D16596
llvm-svn: 258914
2016-01-27 08:43:25 +00:00
Asaf Badouh
ec3729528a
[X86][IFMA] adding intrinsics and encoding for multiply and add of unsigned 52bit integer
...
VPMADD52LUQ - Packed Multiply of Unsigned 52-bit Integers and Add the Low 52-bit Products to Qword Accumulators
VPMADD52HUQ - Packed Multiply of Unsigned 52-bit Unsigned Integers and Add High 52-bit Products to 64-bit Accumulators
Differential Revision: http://reviews.llvm.org/D16407
llvm-svn: 258680
2016-01-25 11:14:24 +00:00
Igor Breger
f91b2666bb
AVX512: VMOVDQU8/16/32/64 (load) intrinsic implementation.
...
Differential Revision: http://reviews.llvm.org/D16137
llvm-svn: 258657
2016-01-24 08:04:33 +00:00
Igor Breger
73167c5d63
AVX512: Masked move intrinsic implementation.
...
Implemented intrinsic for the follow instructions (reg move) : VMOVDQU8/16, VMOVDQA32/64, VMOVAPS/PD.
Differential Revision: http://reviews.llvm.org/D16316
llvm-svn: 258398
2016-01-21 14:18:11 +00:00
Igor Breger
866bd3ac74
AVX512: Store (MOVNTPD, MOVNTPS, MOVNTDQ) using non-temporal hint intrinsic implementation.
...
Differential Revision: http://reviews.llvm.org/D16350
llvm-svn: 258309
2016-01-20 13:11:47 +00:00
Michael Zuckerman
553ef84e85
[AVX512] Adding VPERMT2B and VPERMI2B instruction .
...
Differential Revision: http://reviews.llvm.org/D16297
llvm-svn: 258161
2016-01-19 18:47:02 +00:00
Michael Zuckerman
71a84dc5a5
[AVX512] Adding VPERMB instruction
...
Differential Revision: http://reviews.llvm.org/D16294
llvm-svn: 258144
2016-01-19 17:07:43 +00:00
Asaf Badouh
19e99238a0
[X86][AVX512]fix dag & add intrinsics for fixupimm
...
cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics
Differential Revision: http://reviews.llvm.org/D16313
llvm-svn: 258124
2016-01-19 14:21:39 +00:00
Igor Breger
7327a3bf3b
AVX512: Masked store intrinsic implementation.
...
Implemented intrinsic for the follow instructions (store) : VMOVDQU8/16/32/64, VMOVDQA32/64, VMOVAPS/PD, VMOVUPS/PD.
Differential Revision: http://reviews.llvm.org/D16271
llvm-svn: 258047
2016-01-18 13:52:57 +00:00
Igor Breger
74d74d20c2
AVX512 : Change v8i1 bitconvert GR8 pattern, remove unnecessary movzbl instruction.
...
code example , previous implementation.
movzbl %dil, %eax
kmovw %eax, %k0
new code
kmovw %edi, %k0
Differential Revision: http://reviews.llvm.org/D16287
llvm-svn: 258045
2016-01-18 12:02:45 +00:00
Michael Zuckerman
3e4a1e477a
[AVX512] adding PRORQ , PRORD , PRORLVQ and PRORLVD Intrinsics
...
Differential Revision: http://reviews.llvm.org/D16052
llvm-svn: 257594
2016-01-13 12:39:33 +00:00
Michael Zuckerman
412db37229
[AVX512] adding PROLQ and PROLD Intrinsics
...
Differential Revision: http://reviews.llvm.org/D16048
llvm-svn: 257523
2016-01-12 21:19:17 +00:00
Igor Breger
46e273fe48
AVX512: VPMOVAPS/PD and VPMOVUPS/PD (load) intrinsic implementation.
...
Differential Revision: http://reviews.llvm.org/D16042
llvm-svn: 257463
2016-01-12 10:02:32 +00:00
Craig Topper
d62ff1659a
[AVX-512] Remove another extra space from the Intel syntax asm strings.
...
llvm-svn: 257304
2016-01-11 01:03:40 +00:00
Craig Topper
d9fb66564c
[AVX-512] Remove more superfluous spaces from asm strings.
...
llvm-svn: 257301
2016-01-11 00:44:58 +00:00
Craig Topper
e6ae356ee7
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used.
...
llvm-svn: 257300
2016-01-11 00:44:56 +00:00
Craig Topper
59a087ed05
[AVX-512] Make spacing between comma and {sae} operand consistent in asm strings.
...
llvm-svn: 257299
2016-01-11 00:44:52 +00:00
Craig Topper
9845f978d1
[AVX-512] Remove superfluous spaces from some asm strings.
...
llvm-svn: 257150
2016-01-08 06:09:20 +00:00
Craig Topper
a899645fbc
[AVX512] Add hasSideEffects=0 to kunpck instructions since they lack a pattern in their instructions.
...
llvm-svn: 256816
2016-01-05 07:44:08 +00:00
Asaf Badouh
6fcb80c7ac
[X86][AVX512] add fp scalar broadcast intrinsics
...
Differential Revision: http://reviews.llvm.org/D15790
llvm-svn: 256489
2015-12-28 08:09:25 +00:00
Craig Topper
228bc66bfc
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0.
...
llvm-svn: 256486
2015-12-28 06:32:47 +00:00
Craig Topper
a315cd0b1d
[AVX512] Fix some places that used FR64 instead of FR64X.
...
llvm-svn: 256484
2015-12-28 06:11:45 +00:00
Craig Topper
cf3121d888
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler.
...
I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there.
llvm-svn: 256483
2015-12-28 06:11:42 +00:00
Craig Topper
e4e0592ca3
[AVX512] Remove separate instruction and patterns for lowering ctlz_zero_undef. Change the operation for CTLZ_ZERO_UNDEF to Expand so SelectionDAG will convert them to CTLZ before lowering.
...
llvm-svn: 256477
2015-12-27 21:33:50 +00:00
Craig Topper
ce5014e9fe
[AVX512] Remove alternate data type versions of VALIGND, VALIGNQ, VMOVSHDUP and VMOVSLDUP. They don't have any tests and I don't think they can be selected. If they are truly needed they should be implemented with patterns against the normal instructions and not separate instructions.
...
llvm-svn: 256475
2015-12-27 19:45:21 +00:00
Igor Breger
a848a96908
AVX512: Change VPMOVB2M DAG lowering , use CVT2MASK node instead TRUNCATE.
...
Fix TRUNCATE lowering vector to vector i1, use LSB and not MSB.
Implement VPMOVB/W/D/Q2M intrinsic.
Differential Revision: http://reviews.llvm.org/D15675
llvm-svn: 256470
2015-12-27 13:56:16 +00:00
Asaf Badouh
f94cbd0492
[X86][AVX512] change broadcast to use maskable pattern
...
Differential Revision: http://reviews.llvm.org/D15786
llvm-svn: 256469
2015-12-27 12:14:34 +00:00
Craig Topper
c79efd26f5
[AVX-512] Remove alernate integer forms for VPERMILPS and VPERMILPD. There no tests for them and I don't see any way to select them anyway. If they are really needed they should be implemented as patterns and not full fledged instructions.
...
llvm-svn: 256462
2015-12-27 06:55:08 +00:00
Igor Breger
305115c35b
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts.
...
Differential Revision: http://reviews.llvm.org/D15713
llvm-svn: 256324
2015-12-23 08:06:50 +00:00
Asaf Badouh
d891bbfe44
[X86][AVX512] Add rcp14 and rsqrt14 intrinsics
...
Differential Revision: http://reviews.llvm.org/D15414
llvm-svn: 256237
2015-12-22 11:40:04 +00:00
Elena Demikhovsky
61fc55d5da
Type legalizer for masked gather and scatter intrinsics.
...
Full type legalizer that works with all vectors length - from 2 to 16, (i32, i64, float, double).
This intrinsic, for example
void @llvm.masked.scatter.v2f32(<2 x float>%data , <2 x float*>%ptrs , i32 align , <2 x i1>%mask )
requires type widening for data and type promotion for mask.
Differential Revision: http://reviews.llvm.org/D13633
llvm-svn: 255629
2015-12-15 08:40:41 +00:00
Matt Arsenault
fad94dae85
Start replacing vector_extract/vector_insert with extractelt/insertelt
...
These are redundant pairs of nodes defined for
INSERT_VECTOR_ELEMENT/EXTRACT_VECTOR_ELEMENT.
insertelement/extractelement are slightly closer to the corresponding
C++ node name, and has stricter type checking so prefer it.
Update targets to only use these nodes where it is trivial to do so.
AArch64, ARM, and Mips all have various type errors on simple replacement,
so they will need work to fix.
Example from AArch64:
def : Pat<(sext_inreg (vector_extract (v16i8 V128:$Rn), VectorIndexB:$idx), i8),
(i32 (SMOVvi8to32 V128:$Rn, VectorIndexB:$idx))>;
Which is trying to do sext_inreg i8, i8.
llvm-svn: 255359
2015-12-11 19:20:16 +00:00
Elena Demikhovsky
4e301fd23f
AVX-512: Fixed masked load / store instruction selection for KNL.
...
Patterns were missing for KNL target for <8 x i32>, <8 x float> masked load/store.
This intrinsic comes with all legal types:
<8 x float> @llvm.masked.load.v8f32(<8 x float>* %addr, i32 align, <8 x i1> %mask, <8 x float> %passThru),
but still requires lowering, because VMASKMOVPS, VMASKMOVDQU32 work with 512-bit vectors only.
All data operands should be widened to 512-bit vector.
The mask operand should be widened to v16i1 with zeroes.
Differential Revision: http://reviews.llvm.org/D15265
llvm-svn: 254909
2015-12-07 13:39:24 +00:00
Igor Breger
2e5da39635
AVX-512: implement kunpck intrinsics.
...
Differential Revision: http://reviews.llvm.org/D14821
llvm-svn: 254908
2015-12-07 13:25:18 +00:00
Asaf Badouh
903869d4c1
[X86][AVX512] add vmovss/sd missing encoding
...
Differential Revision: http://reviews.llvm.org/D14701
llvm-svn: 254875
2015-12-06 13:26:56 +00:00
Asaf Badouh
d6d08d5567
[X86][AVX512] add comi with Sae
...
add builtin_ia32_vcomisd and builtin_ia32_vcomisd
Differential Revision: http://reviews.llvm.org/D14331
llvm-svn: 254493
2015-12-02 08:17:51 +00:00
Elena Demikhovsky
a67f2cd5e4
AVX-512: fixed asm string of vsqrtss
...
(vvsqrtss was generated before)
llvm-svn: 254411
2015-12-01 12:43:46 +00:00
Craig Topper
32bf88a844
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns.
...
For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output.
llvm-svn: 254277
2015-11-30 00:13:24 +00:00
Igor Breger
31205fdf6a
AVX512:Implemented encoding for the vmovq.s instruction.
...
Differential Revision: http://reviews.llvm.org/D14810
llvm-svn: 254248
2015-11-29 07:41:26 +00:00
Craig Topper
c5257f522e
[X86] Pair a NoVLX with HasAVX512 to match the others and remove a unique predicate check in the isel tables. NFC
...
llvm-svn: 254191
2015-11-27 05:44:02 +00:00
Craig Topper
c259757ddf
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC
...
llvm-svn: 254177
2015-11-26 20:21:29 +00:00
Craig Topper
f284e70884
[X86] Merge X86VPermt2Fp and X86VPermt2Int back together by weakening them just enough. The SDTCisSameSizeAs introduced in r254138 helps here.
...
llvm-svn: 254176
2015-11-26 20:02:01 +00:00
Elena Demikhovsky
f792042843
AVX-512: Fixed a bug in VPERMT2* intrinsic.
...
It was wrong order of operands (from intrinsic to DAG node).
I added more strict type specification for instruction selection.
Differential Revision: http://reviews.llvm.org/D14942
llvm-svn: 254059
2015-11-25 08:17:56 +00:00
Cong Hou
c0bb26286b
[X86] Fix several issues related to X86's psadbw instruction.
...
This patch fixes the following issues:
1. Fix the return type of X86psadbw: it should not be the same type of inputs.
For vNi8 inputs the output should be vMi64, where M = N/8.
2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly.
3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly.
4. Adjust the return type when building a DAG node of X86ISD::PSADBW type.
5. Update related tests.
Differential revision: http://reviews.llvm.org/D14897
llvm-svn: 254010
2015-11-24 19:51:26 +00:00
Elena Demikhovsky
678dc46339
AVX-512: Optimized INSERT_SUBVECTOR for i1 vector types
...
ISERT_SUBVECTOR for i1 vectors may be done with shifts, when we insert into the lower part, or into the upper part, on into all-zero vector.
CONCAT_VECTORS uses ISERT_SUBVECTOR.
Differential Revision: http://reviews.llvm.org/D14815
llvm-svn: 253819
2015-11-22 13:57:38 +00:00
Igor Breger
0a68600909
AVX512: Implemented encoding, intrinsics and DAG lowering for VMOVDDUP instructions.
...
Differential Revision: http://reviews.llvm.org/D14702
llvm-svn: 253548
2015-11-19 08:26:56 +00:00
Igor Breger
fb07c48ef1
AVX512: Implemented encoding for the vmovss.s and vmovsd.s instructions.
...
Differential Revision: http://reviews.llvm.org/D14771
llvm-svn: 253547
2015-11-19 07:58:33 +00:00
Igor Breger
8a3c708d1f
AVX512: Implemented encoding for the follow instructions.
...
vmovapd.s, vmovaps.s, vmovdqa32.s, vmovdqa64.s, vmovdqu16.s, vmovdqu32.s, vmovdqu64.s, vmovdqu8.s, vmovupd.s, vmovups.s
Differential Revision: http://reviews.llvm.org/D14768
llvm-svn: 253546
2015-11-19 07:43:43 +00:00
Asaf Badouh
e49f73285d
[X86][AVX512CD] add mask broadcast intrinsics
...
Differential Revision: http://reviews.llvm.org/D14573
llvm-svn: 253450
2015-11-18 09:42:45 +00:00