1da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; 2da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; Copyright (c) 2015 The WebM project authors. All Rights Reserved. 3da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; 4da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; Use of this source code is governed by a BSD-style license 5da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; that can be found in the LICENSE file in the root of the source 6da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; tree. An additional intellectual property rights grant can be found 7da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; in the file PATENTS. All contributing project authors may 8da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; be found in the AUTHORS file in the root of the source tree. 9da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian; 10da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 11da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian%include "third_party/x86inc/x86inc.asm" 12da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 13da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh VenkatasubramanianSECTION_RODATA 14da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 15da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanianpw_11585x2: times 8 dw 23170 16da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanianpd_8192: times 4 dd 8192 17da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 18da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian%macro TRANSFORM_COEFFS 2 19da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanianpw_%1_%2: dw %1, %2, %1, %2, %1, %2, %1, %2 20da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanianpw_%2_m%1: dw %2, -%1, %2, -%1, %2, -%1, %2, -%1 21da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian%endmacro 22da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 23da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh VenkatasubramanianTRANSFORM_COEFFS 11585, 11585 24da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh VenkatasubramanianTRANSFORM_COEFFS 15137, 6270 25da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh VenkatasubramanianTRANSFORM_COEFFS 16069, 3196 26da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh VenkatasubramanianTRANSFORM_COEFFS 9102, 13623 27da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 28da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh VenkatasubramanianSECTION .text 29da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 30da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian%if ARCH_X86_64 31da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh VenkatasubramanianINIT_XMM ssse3 32da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramaniancglobal fdct8x8, 3, 5, 13, input, output, stride 33da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 34da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m8, [pd_8192] 35da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m12, [pw_11585x2] 36da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 37da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian lea r3, [2 * strideq] 38da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian lea r4, [4 * strideq] 39da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m0, [inputq] 40da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m1, [inputq + r3] 41da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian lea inputq, [inputq + r4] 42da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m2, [inputq] 43da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m3, [inputq + r3] 44da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian lea inputq, [inputq + r4] 45da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m4, [inputq] 46da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m5, [inputq + r3] 47da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian lea inputq, [inputq + r4] 48da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m6, [inputq] 49da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian mova m7, [inputq + r3] 50da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 51da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian ; left shift by 2 to increase forward transformation precision 52da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian psllw m0, 2 53da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian psllw m1, 2 54da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian psllw m2, 2 55da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian psllw m3, 2 56da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian psllw m4, 2 57da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian psllw m5, 2 58da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian psllw m6, 2 59da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian psllw m7, 2 60da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 61da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian ; column transform 620a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 1 630a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m10, m0, m7 640a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m0, m7 650a39d0a697ff3603e8c100300fda363658e10b23James Zern 660a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m9, m1, m6 670a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m1, m6 680a39d0a697ff3603e8c100300fda363658e10b23James Zern 690a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m7, m2, m5 700a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m2, m5 710a39d0a697ff3603e8c100300fda363658e10b23James Zern 720a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m6, m3, m4 730a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m3, m4 740a39d0a697ff3603e8c100300fda363658e10b23James Zern 750a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 2 760a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m5, m9, m7 770a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m9, m7 780a39d0a697ff3603e8c100300fda363658e10b23James Zern 790a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m4, m10, m6 800a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m10, m6 810a39d0a697ff3603e8c100300fda363658e10b23James Zern 820a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m7, m1, m2 830a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m1, m2 840a39d0a697ff3603e8c100300fda363658e10b23James Zern 850a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 3 860a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m6, m4, m5 870a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m4, m5 880a39d0a697ff3603e8c100300fda363658e10b23James Zern 890a39d0a697ff3603e8c100300fda363658e10b23James Zern pmulhrsw m1, m12 900a39d0a697ff3603e8c100300fda363658e10b23James Zern pmulhrsw m7, m12 910a39d0a697ff3603e8c100300fda363658e10b23James Zern 920a39d0a697ff3603e8c100300fda363658e10b23James Zern ; sin(pi / 8), cos(pi / 8) 930a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m2, m10, m9 940a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m10, m9 950a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m5, m2, [pw_15137_6270] 960a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m2, [pw_6270_m15137] 970a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m9, m10, [pw_15137_6270] 980a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m10, [pw_6270_m15137] 990a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m5, m8 1000a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m2, m8 1010a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m9, m8 1020a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m10, m8 1030a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m5, 14 1040a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m2, 14 1050a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m9, 14 1060a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m10, 14 1070a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m5, m9 1080a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m2, m10 1090a39d0a697ff3603e8c100300fda363658e10b23James Zern 1100a39d0a697ff3603e8c100300fda363658e10b23James Zern pmulhrsw m6, m12 1110a39d0a697ff3603e8c100300fda363658e10b23James Zern pmulhrsw m4, m12 1120a39d0a697ff3603e8c100300fda363658e10b23James Zern 1130a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m9, m3, m1 1140a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m3, m1 1150a39d0a697ff3603e8c100300fda363658e10b23James Zern 1160a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m10, m0, m7 1170a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m0, m7 1180a39d0a697ff3603e8c100300fda363658e10b23James Zern 1190a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 4 1200a39d0a697ff3603e8c100300fda363658e10b23James Zern ; sin(pi / 16), cos(pi / 16) 1210a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m1, m10, m9 1220a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m10, m9 1230a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m7, m1, [pw_16069_3196] 1240a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m1, [pw_3196_m16069] 1250a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m9, m10, [pw_16069_3196] 1260a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m10, [pw_3196_m16069] 1270a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m7, m8 1280a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m1, m8 1290a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m9, m8 1300a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m10, m8 1310a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m7, 14 1320a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m1, 14 1330a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m9, 14 1340a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m10, 14 1350a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m7, m9 1360a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m1, m10 1370a39d0a697ff3603e8c100300fda363658e10b23James Zern 1380a39d0a697ff3603e8c100300fda363658e10b23James Zern ; sin(3 * pi / 16), cos(3 * pi / 16) 1390a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m11, m0, m3 1400a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m0, m3 1410a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m9, m11, [pw_9102_13623] 1420a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m11, [pw_13623_m9102] 1430a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m3, m0, [pw_9102_13623] 1440a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m0, [pw_13623_m9102] 1450a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m9, m8 1460a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m11, m8 1470a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m3, m8 1480a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m0, m8 1490a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m9, 14 1500a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m11, 14 1510a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m3, 14 1520a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m0, 14 1530a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m9, m3 1540a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m11, m0 1550a39d0a697ff3603e8c100300fda363658e10b23James Zern 1560a39d0a697ff3603e8c100300fda363658e10b23James Zern ; transpose 1570a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 1 1580a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m0, m6, m7 1590a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m3, m5, m11 1600a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m6, m7 1610a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m5, m11 1620a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m7, m4, m9 1630a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m10, m2, m1 1640a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m4, m9 1650a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m2, m1 1660a39d0a697ff3603e8c100300fda363658e10b23James Zern 1670a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 2 1680a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckldq m9, m0, m3 1690a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckldq m1, m6, m5 1700a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhdq m0, m3 1710a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhdq m6, m5 1720a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckldq m3, m7, m10 1730a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckldq m5, m4, m2 1740a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhdq m7, m10 1750a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhdq m4, m2 1760a39d0a697ff3603e8c100300fda363658e10b23James Zern 1770a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 3 1780a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklqdq m10, m9, m3 1790a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhqdq m9, m3 1800a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklqdq m2, m0, m7 1810a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhqdq m0, m7 1820a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklqdq m3, m1, m5 1830a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhqdq m1, m5 1840a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklqdq m7, m6, m4 1850a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhqdq m6, m4 1860a39d0a697ff3603e8c100300fda363658e10b23James Zern 1870a39d0a697ff3603e8c100300fda363658e10b23James Zern ; row transform 1880a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 1 1890a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m5, m10, m6 1900a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m10, m6 1910a39d0a697ff3603e8c100300fda363658e10b23James Zern 1920a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m4, m9, m7 1930a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m9, m7 1940a39d0a697ff3603e8c100300fda363658e10b23James Zern 1950a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m6, m2, m1 1960a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m2, m1 1970a39d0a697ff3603e8c100300fda363658e10b23James Zern 1980a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m7, m0, m3 1990a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m0, m3 2000a39d0a697ff3603e8c100300fda363658e10b23James Zern 2010a39d0a697ff3603e8c100300fda363658e10b23James Zern ;stage 2 2020a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m1, m5, m7 2030a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m5, m7 2040a39d0a697ff3603e8c100300fda363658e10b23James Zern 2050a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m3, m4, m6 2060a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m4, m6 2070a39d0a697ff3603e8c100300fda363658e10b23James Zern 2080a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m7, m9, m2 2090a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m9, m2 2100a39d0a697ff3603e8c100300fda363658e10b23James Zern 2110a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 3 2120a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m6, m1, m3 2130a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m1, m3 2140a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m2, m6, [pw_11585_11585] 2150a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m6, [pw_11585_m11585] 2160a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m3, m1, [pw_11585_11585] 2170a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m1, [pw_11585_m11585] 2180a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m2, m8 2190a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m6, m8 2200a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m3, m8 2210a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m1, m8 2220a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m2, 14 2230a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m6, 14 2240a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m3, 14 2250a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m1, 14 2260a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m2, m3 2270a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m6, m1 2280a39d0a697ff3603e8c100300fda363658e10b23James Zern 2290a39d0a697ff3603e8c100300fda363658e10b23James Zern pmulhrsw m7, m12 2300a39d0a697ff3603e8c100300fda363658e10b23James Zern pmulhrsw m9, m12 2310a39d0a697ff3603e8c100300fda363658e10b23James Zern 2320a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m3, m5, m4 2330a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m5, m4 2340a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m1, m3, [pw_15137_6270] 2350a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m3, [pw_6270_m15137] 2360a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m4, m5, [pw_15137_6270] 2370a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m5, [pw_6270_m15137] 2380a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m1, m8 2390a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m3, m8 2400a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m4, m8 2410a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m5, m8 2420a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m1, 14 2430a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m3, 14 2440a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m4, 14 2450a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m5, 14 2460a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m1, m4 2470a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m3, m5 2480a39d0a697ff3603e8c100300fda363658e10b23James Zern 2490a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m4, m0, m9 2500a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m0, m9 2510a39d0a697ff3603e8c100300fda363658e10b23James Zern 2520a39d0a697ff3603e8c100300fda363658e10b23James Zern paddw m5, m10, m7 2530a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m10, m7 2540a39d0a697ff3603e8c100300fda363658e10b23James Zern 2550a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 4 2560a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m9, m5, m4 2570a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m5, m4 2580a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m7, m9, [pw_16069_3196] 2590a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m9, [pw_3196_m16069] 2600a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m4, m5, [pw_16069_3196] 2610a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m5, [pw_3196_m16069] 2620a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m7, m8 2630a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m9, m8 2640a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m4, m8 2650a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m5, m8 2660a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m7, 14 2670a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m9, 14 2680a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m4, 14 2690a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m5, 14 2700a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m7, m4 2710a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m9, m5 2720a39d0a697ff3603e8c100300fda363658e10b23James Zern 2730a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m4, m10, m0 2740a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m10, m0 2750a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m5, m4, [pw_9102_13623] 2760a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m4, [pw_13623_m9102] 2770a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m0, m10, [pw_9102_13623] 2780a39d0a697ff3603e8c100300fda363658e10b23James Zern pmaddwd m10, [pw_13623_m9102] 2790a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m5, m8 2800a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m4, m8 2810a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m0, m8 2820a39d0a697ff3603e8c100300fda363658e10b23James Zern paddd m10, m8 2830a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m5, 14 2840a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m4, 14 2850a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m0, 14 2860a39d0a697ff3603e8c100300fda363658e10b23James Zern psrad m10, 14 2870a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m5, m0 2880a39d0a697ff3603e8c100300fda363658e10b23James Zern packssdw m4, m10 2890a39d0a697ff3603e8c100300fda363658e10b23James Zern 2900a39d0a697ff3603e8c100300fda363658e10b23James Zern ; transpose 2910a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 1 2920a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m0, m2, m7 2930a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m10, m1, m4 2940a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m2, m7 2950a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m1, m4 2960a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m7, m6, m5 2970a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklwd m4, m3, m9 2980a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m6, m5 2990a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhwd m3, m9 3000a39d0a697ff3603e8c100300fda363658e10b23James Zern 3010a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 2 3020a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckldq m5, m0, m10 3030a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckldq m9, m2, m1 3040a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhdq m0, m10 3050a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhdq m2, m1 3060a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckldq m10, m7, m4 3070a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckldq m1, m6, m3 3080a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhdq m7, m4 3090a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhdq m6, m3 3100a39d0a697ff3603e8c100300fda363658e10b23James Zern 3110a39d0a697ff3603e8c100300fda363658e10b23James Zern ; stage 3 3120a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklqdq m4, m5, m10 3130a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhqdq m5, m10 3140a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklqdq m3, m0, m7 3150a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhqdq m0, m7 3160a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklqdq m10, m9, m1 3170a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhqdq m9, m1 3180a39d0a697ff3603e8c100300fda363658e10b23James Zern punpcklqdq m7, m2, m6 3190a39d0a697ff3603e8c100300fda363658e10b23James Zern punpckhqdq m2, m6 3200a39d0a697ff3603e8c100300fda363658e10b23James Zern 3210a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m1, m4, 15 3220a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m6, m5, 15 3230a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m8, m3, 15 3240a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m11, m0, 15 3250a39d0a697ff3603e8c100300fda363658e10b23James Zern 3260a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m4, m1 3270a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m5, m6 3280a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m3, m8 3290a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m0, m11 3300a39d0a697ff3603e8c100300fda363658e10b23James Zern 3310a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m4, 1 3320a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m5, 1 3330a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m3, 1 3340a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m0, 1 3350a39d0a697ff3603e8c100300fda363658e10b23James Zern 3360a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m1, m10, 15 3370a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m6, m9, 15 3380a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m8, m7, 15 3390a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m11, m2, 15 3400a39d0a697ff3603e8c100300fda363658e10b23James Zern 3410a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m10, m1 3420a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m9, m6 3430a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m7, m8 3440a39d0a697ff3603e8c100300fda363658e10b23James Zern psubw m2, m11 3450a39d0a697ff3603e8c100300fda363658e10b23James Zern 3460a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m10, 1 3470a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m9, 1 3480a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m7, 1 3490a39d0a697ff3603e8c100300fda363658e10b23James Zern psraw m2, 1 3500a39d0a697ff3603e8c100300fda363658e10b23James Zern 3510a39d0a697ff3603e8c100300fda363658e10b23James Zern mova [outputq + 0], m4 3520a39d0a697ff3603e8c100300fda363658e10b23James Zern mova [outputq + 16], m5 3530a39d0a697ff3603e8c100300fda363658e10b23James Zern mova [outputq + 32], m3 3540a39d0a697ff3603e8c100300fda363658e10b23James Zern mova [outputq + 48], m0 3550a39d0a697ff3603e8c100300fda363658e10b23James Zern mova [outputq + 64], m10 3560a39d0a697ff3603e8c100300fda363658e10b23James Zern mova [outputq + 80], m9 3570a39d0a697ff3603e8c100300fda363658e10b23James Zern mova [outputq + 96], m7 3580a39d0a697ff3603e8c100300fda363658e10b23James Zern mova [outputq + 112], m2 359da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian 360da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian RET 361da49e34c1fb5e99681f4ad99c21d9cfd83eddb96Vignesh Venkatasubramanian%endif 362