mirror of
https://github.com/RPCS3/llvm-mirror.git
synced 2024-11-26 12:43:36 +01:00
a69222f795
Unfortunately the current call lowering code is built on top of the legacy MVT/DAG based code. However, GlobalISel was not using it the same way. In short, the DAG passes legalized types to the assignment function, and GlobalISel was passing the original raw type if it was simple. I do believe the DAG lowering is conceptually broken since it requires picking a type up front before knowing how/where the value will be passed. This ends up being a problem for AArch64, which wants to pass i1/i8/i16 values as a different size if passed on the stack or in registers. The argument type decision is split across 3 different places which is hard to follow. SelectionDAG builder uses getRegisterTypeForCallingConv to pick a legal type, tablegen gives the illusion of controlling the type, and the target may have additional hacks in the C++ part of the call lowering. AArch64 hacks around this by not using the standard AnalyzeFormalArguments and special casing i1/i8/i16 by looking at the underlying type of the original IR argument. I believe people have generally assumed the calling convention code is processing the original types, and I've discovered a number of dead paths in several targets. x86 actually relies on the opposite behavior from AArch64, and relies on x86_32 and x86_64 sharing calling convention code where the 64-bit cases implicitly do not work on x86_32 due to using the pre-legalized types. AMDGPU targets without legal i16/f16 have always used a broken ABI that promotes to i32/f32. GlobalISel accidentally fixed this to be the ABI we should have, but this fixes it so we're using the worse ABI that is compatible with the DAG. Ideally we would fix the DAG to match the old GlobalISel behavior, but I don't wish to fight that battle. A new native GlobalISel call lowering framework should let the target process the incoming types directly. CCValAssigns select a "ValVT" and "LocVT" but the meanings of these aren't entirely clear. Different targets don't use them consistently, even within their own call lowering code. My current belief is the intent was "ValVT" is supposed to be the legalized value type to use in the end, and and LocVT was supposed to be the ABI passed type (which is also legalized). With the default CCState::Analyze functions always passing the same type for these arguments, these only differ when the TableGen part of the lowering decide to promote the type from one legal type to another. AArch64's i1/i8/i16 hack ends up inverting the meanings of these values, so I had to add an additional hack to let the target interpret how large the argument memory is. Since targets don't consistently interpret ValVT and LocVT, this doesn't produce quite equivalent code to the initial DAG lowerings. I've opted to consistently interpret LocVT as the in-memory size for stack passed values, and ValVT as the register type to assign from that memory. We therefore produce extending loads directly out of the IRTranslator, whereas the DAG would emit regular loads of smaller values. This will also produce loads/stores that are wider than the argument value if the allocated stack slot is larger (and there will be undef padding bytes). If we had the optimizations to reduce load/stores based on truncated values, this wouldn't produce a different end result. Since ValVT/LocVT are more consistently interpreted, we now will emit more G_BITCASTS as requested by the CCAssignFn. For example AArch64 was directly assigning types to some physical vector registers which according to the tablegen spec should have been casted to a vector with a different element type. This also moves the responsibility for inserting G_ASSERT_SEXT/G_ASSERT_ZEXT from the target ValueHandlers into the generic code, which is closer to how SelectionDAGBuilder works. I had to xfail an x86 test since I don't see a quick way to fix it right now (I filed bug 50035 for this). It's broken independently of this change, and only triggers since now we end up with more ands which hit the improperly handled selection pattern. I also observed that FP arguments that need promotion (e.g. f16 passed as f32) are broken, and use regular G_TRUNC and G_ANYEXT. TLDR; the current call lowering infrastructure is bad and nobody has ever understood how it chooses types.
1500 lines
65 KiB
LLVM
1500 lines
65 KiB
LLVM
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
|
|
; Denormal mode shouldn't matter for f16, check with and without flushing.
|
|
; RUN: llc -global-isel -march=amdgcn -mcpu=tahiti -denormal-fp-math=ieee -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX6,GFX6-IEEE %s
|
|
; RUN: llc -global-isel -march=amdgcn -mcpu=tahiti -denormal-fp-math=preserve-sign -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX6,GFX6-FLUSH %s
|
|
|
|
; RUN: llc -global-isel -march=amdgcn -mcpu=fiji -denormal-fp-math=ieee -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX89,GFX8 %s
|
|
; RUN: llc -global-isel -march=amdgcn -mcpu=fiji -denormal-fp-math=preserve-sign -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX89,GFX8 %s
|
|
|
|
; RUN: llc -global-isel -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX89,GFX9 %s
|
|
; RUN: llc -global-isel -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX89,GFX9 %s
|
|
|
|
; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=ieee -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX10 %s
|
|
; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=preserve-sign -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX10 %s
|
|
|
|
define half @v_fdiv_f16(half %a, half %b) {
|
|
; GFX6-IEEE-LABEL: v_fdiv_f16:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v0
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, vcc, v0, v1, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v2, v1, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_fdiv_f16:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v0
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, vcc, v0, v1, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v2, v1, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_fdiv_f16:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX89-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX89-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX89-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX89-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_f16:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX10-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv half %a, %b
|
|
ret half %fdiv
|
|
}
|
|
|
|
define half @v_fdiv_f16_afn(half %a, half %b) {
|
|
; GFX6-LABEL: v_fdiv_f16_afn:
|
|
; GFX6: ; %bb.0:
|
|
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX6-NEXT: v_mul_f32_e32 v0, v0, v1
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_fdiv_f16_afn:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_rcp_f16_e32 v1, v1
|
|
; GFX89-NEXT: v_mul_f16_e32 v0, v0, v1
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_f16_afn:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v1, v1
|
|
; GFX10-NEXT: v_mul_f16_e32 v0, v0, v1
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv afn half %a, %b
|
|
ret half %fdiv
|
|
}
|
|
|
|
define half @v_fdiv_f16_ulp25(half %a, half %b) {
|
|
; GFX6-IEEE-LABEL: v_fdiv_f16_ulp25:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v0
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, vcc, v0, v1, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v2, v1, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_fdiv_f16_ulp25:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v0
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, vcc, v0, v1, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v2, v1, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_fdiv_f16_ulp25:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX89-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX89-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX89-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX89-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_f16_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX10-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv half %a, %b, !fpmath !0
|
|
ret half %fdiv
|
|
}
|
|
|
|
define half @v_rcp_f16(half %x) {
|
|
; GFX6-IEEE-LABEL: v_rcp_f16:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, 1.0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v0, v0, v1
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, vcc, v1, v0, v1
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v2, v0, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_rcp_f16:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, 1.0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v0, v0, v1
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, vcc, v1, v0, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v2, v0, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_rcp_f16:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v1, v0
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v2, 1.0
|
|
; GFX89-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX89-NEXT: v_mul_f32_e32 v1, v2, v1
|
|
; GFX89-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX89-NEXT: v_div_fixup_f16 v0, v1, v0, 1.0
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_rcp_f16:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v1, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v2, 1.0
|
|
; GFX10-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX10-NEXT: v_mul_f32_e32 v1, v2, v1
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v1, v0, 1.0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv half 1.0, %x
|
|
ret half %fdiv
|
|
}
|
|
|
|
define half @v_rcp_f16_arcp(half %x) {
|
|
; GFX6-IEEE-LABEL: v_rcp_f16_arcp:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, 1.0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v0, v0, v1
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, vcc, v1, v0, v1
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v2, v0, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_rcp_f16_arcp:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, 1.0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v0, v0, v1
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, vcc, v1, v0, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v2, v0, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_rcp_f16_arcp:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v1, v0
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v2, 1.0
|
|
; GFX89-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX89-NEXT: v_mul_f32_e32 v1, v2, v1
|
|
; GFX89-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX89-NEXT: v_div_fixup_f16 v0, v1, v0, 1.0
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_rcp_f16_arcp:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v1, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v2, 1.0
|
|
; GFX10-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX10-NEXT: v_mul_f32_e32 v1, v2, v1
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v1, v0, 1.0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv arcp half 1.0, %x
|
|
ret half %fdiv
|
|
}
|
|
|
|
define half @v_rcp_f16_arcp_afn(half %x) {
|
|
; GFX6-LABEL: v_rcp_f16_arcp_afn:
|
|
; GFX6: ; %bb.0:
|
|
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v1, 1.0
|
|
; GFX6-NEXT: v_rcp_f32_e32 v0, v0
|
|
; GFX6-NEXT: v_mul_f32_e32 v0, v1, v0
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_rcp_f16_arcp_afn:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_rcp_f16_e32 v0, v0
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_rcp_f16_arcp_afn:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v0, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv arcp afn half 1.0, %x
|
|
ret half %fdiv
|
|
}
|
|
|
|
define half @v_rcp_f16_ulp25(half %x) {
|
|
; GFX6-IEEE-LABEL: v_rcp_f16_ulp25:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, 1.0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v0, v0, v1
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, vcc, v1, v0, v1
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v2, v0, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_rcp_f16_ulp25:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, 1.0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v0, v0, v1
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, vcc, v1, v0, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v2, v0, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_rcp_f16_ulp25:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_rcp_f16_e32 v0, v0
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_rcp_f16_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v0, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv half 1.0, %x, !fpmath !0
|
|
ret half %fdiv
|
|
}
|
|
|
|
define half @v_fdiv_f16_afn_ulp25(half %a, half %b) {
|
|
; GFX6-LABEL: v_fdiv_f16_afn_ulp25:
|
|
; GFX6: ; %bb.0:
|
|
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX6-NEXT: v_mul_f32_e32 v0, v0, v1
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_fdiv_f16_afn_ulp25:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_rcp_f16_e32 v1, v1
|
|
; GFX89-NEXT: v_mul_f16_e32 v0, v0, v1
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_f16_afn_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v1, v1
|
|
; GFX10-NEXT: v_mul_f16_e32 v0, v0, v1
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv afn half %a, %b, !fpmath !0
|
|
ret half %fdiv
|
|
}
|
|
|
|
define half @v_fdiv_f16_arcp_ulp25(half %a, half %b) {
|
|
; GFX6-IEEE-LABEL: v_fdiv_f16_arcp_ulp25:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v0
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, vcc, v0, v1, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v2, v1, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_fdiv_f16_arcp_ulp25:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v0
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, vcc, v0, v1, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v5, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v5, v4, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v6, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v5, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v5
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v2, v1, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX89-LABEL: v_fdiv_f16_arcp_ulp25:
|
|
; GFX89: ; %bb.0:
|
|
; GFX89-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX89-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX89-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX89-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX89-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX89-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX89-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_f16_arcp_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX10-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv arcp half %a, %b, !fpmath !0
|
|
ret half %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_fdiv_v2f16(<2 x half> %a, <2 x half> %b) {
|
|
; GFX6-IEEE-LABEL: v_fdiv_v2f16:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, s[4:5], v2, v2, v0
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v6, vcc, v0, v2, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v4, v5, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v7, v5, v5
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v7, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v8, -v4, v7, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, v8, v5, v7
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, -v4, v7, v6
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v4, v4, v5, v7
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v4, v2, v0
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v3, v3, v1
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v1, v3, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v1, v2, v3, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_fdiv_v2f16:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, s[4:5], v2, v2, v0
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v5, v4
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v6, vcc, v0, v2, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v4, v5, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v7, v5, v5
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v7, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v8, -v4, v7, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, v8, v5, v7
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, -v4, v7, v6
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v4, v4, v5, v7
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v4, v2, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v3, v3, v1
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v4, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v1, v3, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v4, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v1, v2, v3, v1
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_fdiv_v2f16:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_lshrrev_b32_e32 v4, 16, v1
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v5, v4
|
|
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 16, v0
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX8-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v7, v6
|
|
; GFX8-NEXT: v_rcp_f32_e32 v5, v5
|
|
; GFX8-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX8-NEXT: v_mul_f32_e32 v3, v7, v5
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX8-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX8-NEXT: v_mov_b32_e32 v2, 16
|
|
; GFX8-NEXT: v_div_fixup_f16 v1, v3, v4, v6
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_fdiv_v2f16:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_lshrrev_b32_e32 v4, 16, v1
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v5, v4
|
|
; GFX9-NEXT: v_lshrrev_b32_e32 v6, 16, v0
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX9-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v7, v6
|
|
; GFX9-NEXT: v_rcp_f32_e32 v5, v5
|
|
; GFX9-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX9-NEXT: v_mul_f32_e32 v3, v7, v5
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX9-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX9-NEXT: v_mov_b32_e32 v2, 0xffff
|
|
; GFX9-NEXT: v_div_fixup_f16 v1, v3, v4, v6
|
|
; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v1
|
|
; GFX9-NEXT: v_and_or_b32 v0, v0, v2, v1
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_v2f16:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_lshrrev_b32_e32 v2, 16, v1
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v4, v1
|
|
; GFX10-NEXT: v_lshrrev_b32_e32 v5, 16, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v7, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v3, v2
|
|
; GFX10-NEXT: v_rcp_f32_e32 v4, v4
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v6, v5
|
|
; GFX10-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_mul_f32_e32 v4, v7, v4
|
|
; GFX10-NEXT: v_mul_f32_e32 v3, v6, v3
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v4, v4
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v4, v1, v0
|
|
; GFX10-NEXT: v_div_fixup_f16 v2, v3, v2, v5
|
|
; GFX10-NEXT: v_lshlrev_b32_e32 v1, 16, v2
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v0, v1
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv <2 x half> %a, %b
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_fdiv_v2f16_afn(<2 x half> %a, <2 x half> %b) {
|
|
; GFX6-LABEL: v_fdiv_v2f16_afn:
|
|
; GFX6: ; %bb.0:
|
|
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX6-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX6-NEXT: v_mul_f32_e32 v0, v0, v2
|
|
; GFX6-NEXT: v_mul_f32_e32 v1, v1, v3
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_fdiv_v2f16_afn:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX8-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX8-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX8-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX8-NEXT: v_mov_b32_e32 v1, 16
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v2, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_fdiv_v2f16_afn:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX9-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX9-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX9-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX9-NEXT: v_mov_b32_e32 v1, 0xffff
|
|
; GFX9-NEXT: v_and_or_b32 v0, v2, v1, v0
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_v2f16_afn:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX10-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX10-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX10-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v2, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv afn <2 x half> %a, %b
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_fdiv_v2f16_ulp25(<2 x half> %a, <2 x half> %b) {
|
|
; GFX6-IEEE-LABEL: v_fdiv_v2f16_ulp25:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, s[4:5], v2, v2, v0
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v6, vcc, v0, v2, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v4, v5, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v7, v5, v5
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v7, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v8, -v4, v7, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, v8, v5, v7
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, -v4, v7, v6
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v4, v4, v5, v7
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v4, v2, v0
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v3, v3, v1
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v1, v3, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v1, v2, v3, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_fdiv_v2f16_ulp25:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, s[4:5], v2, v2, v0
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v5, v4
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v6, vcc, v0, v2, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v4, v5, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v7, v5, v5
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v7, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v8, -v4, v7, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, v8, v5, v7
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, -v4, v7, v6
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v4, v4, v5, v7
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v4, v2, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v3, v3, v1
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v4, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v1, v3, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v4, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v1, v2, v3, v1
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_fdiv_v2f16_ulp25:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_lshrrev_b32_e32 v4, 16, v1
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v5, v4
|
|
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 16, v0
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX8-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v7, v6
|
|
; GFX8-NEXT: v_rcp_f32_e32 v5, v5
|
|
; GFX8-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX8-NEXT: v_mul_f32_e32 v3, v7, v5
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX8-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX8-NEXT: v_mov_b32_e32 v2, 16
|
|
; GFX8-NEXT: v_div_fixup_f16 v1, v3, v4, v6
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_fdiv_v2f16_ulp25:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_lshrrev_b32_e32 v4, 16, v1
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v5, v4
|
|
; GFX9-NEXT: v_lshrrev_b32_e32 v6, 16, v0
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX9-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v7, v6
|
|
; GFX9-NEXT: v_rcp_f32_e32 v5, v5
|
|
; GFX9-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX9-NEXT: v_mul_f32_e32 v3, v7, v5
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX9-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX9-NEXT: v_mov_b32_e32 v2, 0xffff
|
|
; GFX9-NEXT: v_div_fixup_f16 v1, v3, v4, v6
|
|
; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v1
|
|
; GFX9-NEXT: v_and_or_b32 v0, v0, v2, v1
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_v2f16_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_lshrrev_b32_e32 v2, 16, v1
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v4, v1
|
|
; GFX10-NEXT: v_lshrrev_b32_e32 v5, 16, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v7, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v3, v2
|
|
; GFX10-NEXT: v_rcp_f32_e32 v4, v4
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v6, v5
|
|
; GFX10-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_mul_f32_e32 v4, v7, v4
|
|
; GFX10-NEXT: v_mul_f32_e32 v3, v6, v3
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v4, v4
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v4, v1, v0
|
|
; GFX10-NEXT: v_div_fixup_f16 v2, v3, v2, v5
|
|
; GFX10-NEXT: v_lshlrev_b32_e32 v1, 16, v2
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v0, v1
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv <2 x half> %a, %b, !fpmath !0
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_rcp_v2f16(<2 x half> %x) {
|
|
; GFX6-IEEE-LABEL: v_rcp_v2f16:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v2, 1.0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v3, s[4:5], v0, v0, v2
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v2, v0, v2
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v3, v0, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v3, s[4:5], v1, v1, v2
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v2, v1, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v1, v3, v1, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_rcp_v2f16:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: s_movk_i32 s6, 0x3c00
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v2, s6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v3, s[4:5], v0, v0, v2
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v2, v0, v2
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v4, s6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v3, v0, v2
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v4
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v4, v1, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v6, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v3, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v6
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v1, v2, v1, v4
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_rcp_v2f16:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_lshrrev_b32_e32 v2, 16, v0
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v1, v0
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v3, v2
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v4, 1.0
|
|
; GFX8-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX8-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX8-NEXT: v_mul_f32_e32 v1, v4, v1
|
|
; GFX8-NEXT: v_mul_f32_e32 v3, v4, v3
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX8-NEXT: v_div_fixup_f16 v0, v1, v0, 1.0
|
|
; GFX8-NEXT: v_div_fixup_f16 v1, v3, v2, 1.0
|
|
; GFX8-NEXT: v_mov_b32_e32 v2, 16
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_rcp_v2f16:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_lshrrev_b32_e32 v2, 16, v0
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v1, v0
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v3, v2
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v4, 1.0
|
|
; GFX9-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX9-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX9-NEXT: v_mul_f32_e32 v1, v4, v1
|
|
; GFX9-NEXT: v_mul_f32_e32 v3, v4, v3
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX9-NEXT: v_div_fixup_f16 v0, v1, v0, 1.0
|
|
; GFX9-NEXT: v_div_fixup_f16 v1, v3, v2, 1.0
|
|
; GFX9-NEXT: v_mov_b32_e32 v2, 0xffff
|
|
; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v1
|
|
; GFX9-NEXT: v_and_or_b32 v0, v0, v2, v1
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_rcp_v2f16:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_lshrrev_b32_e32 v1, 16, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v4, 1.0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX10-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_mul_f32_e32 v3, v4, v3
|
|
; GFX10-NEXT: v_mul_f32_e32 v2, v4, v2
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v3, v0, 1.0
|
|
; GFX10-NEXT: v_div_fixup_f16 v1, v2, v1, 1.0
|
|
; GFX10-NEXT: v_lshlrev_b32_e32 v1, 16, v1
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v0, v1
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv <2 x half> <half 1.0, half 1.0>, %x
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_rcp_v2f16_arcp(<2 x half> %x) {
|
|
; GFX6-IEEE-LABEL: v_rcp_v2f16_arcp:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v2, 1.0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v3, s[4:5], v0, v0, v2
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v2, v0, v2
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v3, v0, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v3, s[4:5], v1, v1, v2
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v2, v1, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v1, v3, v1, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_rcp_v2f16_arcp:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: s_movk_i32 s6, 0x3c00
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v2, s6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v3, s[4:5], v0, v0, v2
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v2, v0, v2
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v4, s6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v3, v0, v2
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v4
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v4, v1, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v6, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v3, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v6
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v1, v2, v1, v4
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_rcp_v2f16_arcp:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_lshrrev_b32_e32 v2, 16, v0
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v1, v0
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v3, v2
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v4, 1.0
|
|
; GFX8-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX8-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX8-NEXT: v_mul_f32_e32 v1, v4, v1
|
|
; GFX8-NEXT: v_mul_f32_e32 v3, v4, v3
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX8-NEXT: v_div_fixup_f16 v0, v1, v0, 1.0
|
|
; GFX8-NEXT: v_div_fixup_f16 v1, v3, v2, 1.0
|
|
; GFX8-NEXT: v_mov_b32_e32 v2, 16
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_rcp_v2f16_arcp:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_lshrrev_b32_e32 v2, 16, v0
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v1, v0
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v3, v2
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v4, 1.0
|
|
; GFX9-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX9-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX9-NEXT: v_mul_f32_e32 v1, v4, v1
|
|
; GFX9-NEXT: v_mul_f32_e32 v3, v4, v3
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX9-NEXT: v_div_fixup_f16 v0, v1, v0, 1.0
|
|
; GFX9-NEXT: v_div_fixup_f16 v1, v3, v2, 1.0
|
|
; GFX9-NEXT: v_mov_b32_e32 v2, 0xffff
|
|
; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v1
|
|
; GFX9-NEXT: v_and_or_b32 v0, v0, v2, v1
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_rcp_v2f16_arcp:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_lshrrev_b32_e32 v1, 16, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v4, 1.0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX10-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_mul_f32_e32 v3, v4, v3
|
|
; GFX10-NEXT: v_mul_f32_e32 v2, v4, v2
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v3, v0, 1.0
|
|
; GFX10-NEXT: v_div_fixup_f16 v1, v2, v1, 1.0
|
|
; GFX10-NEXT: v_lshlrev_b32_e32 v1, 16, v1
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v0, v1
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv arcp <2 x half> <half 1.0, half 1.0>, %x
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_rcp_v2f16_arcp_afn(<2 x half> %x) {
|
|
; GFX6-LABEL: v_rcp_v2f16_arcp_afn:
|
|
; GFX6: ; %bb.0:
|
|
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v2, 1.0
|
|
; GFX6-NEXT: v_rcp_f32_e32 v0, v0
|
|
; GFX6-NEXT: v_rcp_f32_e32 v1, v1
|
|
; GFX6-NEXT: v_mul_f32_e32 v0, v2, v0
|
|
; GFX6-NEXT: v_mul_f32_e32 v1, v2, v1
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_rcp_v2f16_arcp_afn:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_rcp_f16_sdwa v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX8-NEXT: v_rcp_f16_e32 v0, v0
|
|
; GFX8-NEXT: v_mov_b32_e32 v2, 16
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_rcp_v2f16_arcp_afn:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_rcp_f16_e32 v1, v0
|
|
; GFX9-NEXT: v_rcp_f16_sdwa v0, v0 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX9-NEXT: v_mov_b32_e32 v2, 0xffff
|
|
; GFX9-NEXT: v_and_or_b32 v0, v1, v2, v0
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_rcp_v2f16_arcp_afn:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v1, v0
|
|
; GFX10-NEXT: v_rcp_f16_sdwa v0, v0 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v1, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv arcp afn <2 x half> <half 1.0, half 1.0>, %x
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_rcp_v2f16_ulp25(<2 x half> %x) {
|
|
; GFX6-IEEE-LABEL: v_rcp_v2f16_ulp25:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v2, 1.0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v3, s[4:5], v0, v0, v2
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v2, v0, v2
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v3, v0, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v3, s[4:5], v1, v1, v2
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v2, v1, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v1, v3, v1, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_rcp_v2f16_ulp25:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: s_movk_i32 s6, 0x3c00
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v2, s6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v3, s[4:5], v0, v0, v2
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v4, v3
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v2, v0, v2
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v3, v4, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v3, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, -v3, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v3, v3, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v4, s6
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v3, v0, v2
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v1, v1, v4
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v3, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v4, v1, v4
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v3, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v3, v6, v3, v3
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v3, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v3, v6
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v1, v2, v1, v4
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_rcp_v2f16_ulp25:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_rcp_f16_sdwa v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX8-NEXT: v_rcp_f16_e32 v0, v0
|
|
; GFX8-NEXT: v_mov_b32_e32 v2, 16
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_rcp_v2f16_ulp25:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_rcp_f16_e32 v1, v0
|
|
; GFX9-NEXT: v_rcp_f16_sdwa v0, v0 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX9-NEXT: v_mov_b32_e32 v2, 0xffff
|
|
; GFX9-NEXT: v_and_or_b32 v0, v1, v2, v0
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_rcp_v2f16_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v1, v0
|
|
; GFX10-NEXT: v_rcp_f16_sdwa v0, v0 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v1, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv <2 x half> <half 1.0, half 1.0>, %x, !fpmath !0
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_fdiv_v2f16_afn_ulp25(<2 x half> %a, <2 x half> %b) {
|
|
; GFX6-LABEL: v_fdiv_v2f16_afn_ulp25:
|
|
; GFX6: ; %bb.0:
|
|
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX6-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX6-NEXT: v_mul_f32_e32 v0, v0, v2
|
|
; GFX6-NEXT: v_mul_f32_e32 v1, v1, v3
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_fdiv_v2f16_afn_ulp25:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX8-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX8-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX8-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX8-NEXT: v_mov_b32_e32 v1, 16
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v2, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_fdiv_v2f16_afn_ulp25:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX9-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX9-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX9-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX9-NEXT: v_mov_b32_e32 v1, 0xffff
|
|
; GFX9-NEXT: v_and_or_b32 v0, v2, v1, v0
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_v2f16_afn_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX10-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX10-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX10-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v2, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv afn <2 x half> %a, %b, !fpmath !0
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_fdiv_v2f16_arcp_ulp25(<2 x half> %a, <2 x half> %b) {
|
|
; GFX6-IEEE-LABEL: v_fdiv_v2f16_arcp_ulp25:
|
|
; GFX6-IEEE: ; %bb.0:
|
|
; GFX6-IEEE-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v4, s[4:5], v2, v2, v0
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v5, v4
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v6, vcc, v0, v2, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v4, v5, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v5, v7, v5, v5
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v7, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v8, -v4, v7, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, v8, v5, v7
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, -v4, v7, v6
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v4, v4, v5, v7
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v0, v4, v2, v0
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v2, s[4:5], v3, v3, v1
|
|
; GFX6-IEEE-NEXT: v_rcp_f32_e32 v4, v2
|
|
; GFX6-IEEE-NEXT: v_div_scale_f32 v5, vcc, v1, v3, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, -v2, v4, 1.0
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-IEEE-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-IEEE-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-IEEE-NEXT: v_div_fmas_f32 v2, v2, v4, v6
|
|
; GFX6-IEEE-NEXT: v_div_fixup_f32 v1, v2, v3, v1
|
|
; GFX6-IEEE-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-IEEE-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX6-FLUSH-LABEL: v_fdiv_v2f16_arcp_ulp25:
|
|
; GFX6-FLUSH: ; %bb.0:
|
|
; GFX6-FLUSH-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v4, s[4:5], v2, v2, v0
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v5, v4
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v6, vcc, v0, v2, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v4, v5, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v5, v7, v5, v5
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v7, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v8, -v4, v7, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, v8, v5, v7
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, -v4, v7, v6
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v4, v4, v5, v7
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v0, v4, v2, v0
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v2, s[4:5], v3, v3, v1
|
|
; GFX6-FLUSH-NEXT: v_rcp_f32_e32 v4, v2
|
|
; GFX6-FLUSH-NEXT: v_div_scale_f32 v5, vcc, v1, v3, v1
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 3
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, -v2, v4, 1.0
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v4, v6, v4, v4
|
|
; GFX6-FLUSH-NEXT: v_mul_f32_e32 v6, v5, v4
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v7, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v6, v7, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_fma_f32 v2, -v2, v6, v5
|
|
; GFX6-FLUSH-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_MODE, 4, 2), 0
|
|
; GFX6-FLUSH-NEXT: v_div_fmas_f32 v2, v2, v4, v6
|
|
; GFX6-FLUSH-NEXT: v_div_fixup_f32 v1, v2, v3, v1
|
|
; GFX6-FLUSH-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-FLUSH-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_fdiv_v2f16_arcp_ulp25:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_lshrrev_b32_e32 v4, 16, v1
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v5, v4
|
|
; GFX8-NEXT: v_lshrrev_b32_e32 v6, 16, v0
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX8-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX8-NEXT: v_cvt_f32_f16_e32 v7, v6
|
|
; GFX8-NEXT: v_rcp_f32_e32 v5, v5
|
|
; GFX8-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX8-NEXT: v_mul_f32_e32 v3, v7, v5
|
|
; GFX8-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX8-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX8-NEXT: v_mov_b32_e32 v2, 16
|
|
; GFX8-NEXT: v_div_fixup_f16 v1, v3, v4, v6
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v1, v2, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_fdiv_v2f16_arcp_ulp25:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_lshrrev_b32_e32 v4, 16, v1
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v2, v1
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v5, v4
|
|
; GFX9-NEXT: v_lshrrev_b32_e32 v6, 16, v0
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v3, v0
|
|
; GFX9-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX9-NEXT: v_cvt_f32_f16_e32 v7, v6
|
|
; GFX9-NEXT: v_rcp_f32_e32 v5, v5
|
|
; GFX9-NEXT: v_mul_f32_e32 v2, v3, v2
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v2, v2
|
|
; GFX9-NEXT: v_mul_f32_e32 v3, v7, v5
|
|
; GFX9-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX9-NEXT: v_div_fixup_f16 v0, v2, v1, v0
|
|
; GFX9-NEXT: v_mov_b32_e32 v2, 0xffff
|
|
; GFX9-NEXT: v_div_fixup_f16 v1, v3, v4, v6
|
|
; GFX9-NEXT: v_lshlrev_b32_e32 v1, 16, v1
|
|
; GFX9-NEXT: v_and_or_b32 v0, v0, v2, v1
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_v2f16_arcp_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_lshrrev_b32_e32 v2, 16, v1
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v4, v1
|
|
; GFX10-NEXT: v_lshrrev_b32_e32 v5, 16, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v7, v0
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v3, v2
|
|
; GFX10-NEXT: v_rcp_f32_e32 v4, v4
|
|
; GFX10-NEXT: v_cvt_f32_f16_e32 v6, v5
|
|
; GFX10-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_mul_f32_e32 v4, v7, v4
|
|
; GFX10-NEXT: v_mul_f32_e32 v3, v6, v3
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v4, v4
|
|
; GFX10-NEXT: v_cvt_f16_f32_e32 v3, v3
|
|
; GFX10-NEXT: v_div_fixup_f16 v0, v4, v1, v0
|
|
; GFX10-NEXT: v_div_fixup_f16 v2, v3, v2, v5
|
|
; GFX10-NEXT: v_lshlrev_b32_e32 v1, 16, v2
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v0, v1
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv arcp <2 x half> %a, %b, !fpmath !0
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
define <2 x half> @v_fdiv_v2f16_arcp_afn_ulp25(<2 x half> %a, <2 x half> %b) {
|
|
; GFX6-LABEL: v_fdiv_v2f16_arcp_afn_ulp25:
|
|
; GFX6: ; %bb.0:
|
|
; GFX6-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v2, v2
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v3, v3
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f32_f16_e32 v1, v1
|
|
; GFX6-NEXT: v_rcp_f32_e32 v2, v2
|
|
; GFX6-NEXT: v_rcp_f32_e32 v3, v3
|
|
; GFX6-NEXT: v_mul_f32_e32 v0, v0, v2
|
|
; GFX6-NEXT: v_mul_f32_e32 v1, v1, v3
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v0, v0
|
|
; GFX6-NEXT: v_cvt_f16_f32_e32 v1, v1
|
|
; GFX6-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX8-LABEL: v_fdiv_v2f16_arcp_afn_ulp25:
|
|
; GFX8: ; %bb.0:
|
|
; GFX8-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX8-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX8-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX8-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX8-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX8-NEXT: v_mov_b32_e32 v1, 16
|
|
; GFX8-NEXT: v_lshlrev_b32_sdwa v0, v1, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_0
|
|
; GFX8-NEXT: v_or_b32_sdwa v0, v2, v0 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
|
|
; GFX8-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX9-LABEL: v_fdiv_v2f16_arcp_afn_ulp25:
|
|
; GFX9: ; %bb.0:
|
|
; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX9-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX9-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX9-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX9-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX9-NEXT: v_mov_b32_e32 v1, 0xffff
|
|
; GFX9-NEXT: v_and_or_b32 v0, v2, v1, v0
|
|
; GFX9-NEXT: s_setpc_b64 s[30:31]
|
|
;
|
|
; GFX10-LABEL: v_fdiv_v2f16_arcp_afn_ulp25:
|
|
; GFX10: ; %bb.0:
|
|
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
|
|
; GFX10-NEXT: s_waitcnt_vscnt null, 0x0
|
|
; GFX10-NEXT: v_rcp_f16_e32 v2, v1
|
|
; GFX10-NEXT: v_rcp_f16_sdwa v1, v1 dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
|
|
; GFX10-NEXT: v_mul_f16_e32 v2, v0, v2
|
|
; GFX10-NEXT: v_mul_f16_sdwa v0, v0, v1 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
|
|
; GFX10-NEXT: v_and_or_b32 v0, 0xffff, v2, v0
|
|
; GFX10-NEXT: s_setpc_b64 s[30:31]
|
|
%fdiv = fdiv afn arcp <2 x half> %a, %b, !fpmath !0
|
|
ret <2 x half> %fdiv
|
|
}
|
|
|
|
!0 = !{float 2.500000e+00}
|