History log of /external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
2c3e0051c31c3f5b2328b447eadf1cf9c4427442 06-May-2015 Pirama Arumuga Nainar <pirama@google.com> Update aosp/master LLVM for rebase to r235153

Change-Id: I9bf53792f9fc30570e81a8d80d296c681d005ea7
(cherry picked from commit 0c7f116bb6950ef819323d855415b2f2b0aad987)
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4c5e43da7792f75567b693105cc53e3f1992ad98 08-Apr-2015 Pirama Arumuga Nainar <pirama@google.com> Update aosp/master llvm for rebase to r233350

Change-Id: I07d935f8793ee8ec6b7da003f6483046594bca49
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ebe69fe11e48d322045d5949c83283927a0d790b 23-Mar-2015 Stephen Hines <srhines@google.com> Update aosp/master LLVM for rebase to r230699.

Change-Id: I2b5be30509658cb8266be782de0ab24f9099f9b9
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
37ed9c199ca639565f6ce88105f9e39e898d82d0 01-Dec-2014 Stephen Hines <srhines@google.com> Update aosp/master LLVM for rebase to r222494.

Change-Id: Ic787f5e0124df789bd26f3f24680f45e678eef2d
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c6a4f5e819217e1e12c458aed8e7b122e23a3a58 21-Jul-2014 Stephen Hines <srhines@google.com> Update LLVM for rebase to r212749.

Includes a cherry-pick of:
r212948 - fixes a small issue with atomic calls

Change-Id: Ib97bd980b59f18142a69506400911a6009d9df18
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
dce4a407a24b04eebc6a376f8e62b41aaa7b071f 29-May-2014 Stephen Hines <srhines@google.com> Update LLVM for 3.5 rebase (r209712).

Change-Id: I149556c940fb7dc92d075273c87ff584f400941f
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
36b56886974eae4f9c5ebc96befd3e7bfe5de338 24-Apr-2014 Stephen Hines <srhines@google.com> Update to LLVM 3.5a.

Change-Id: Ifadecab779f128e62e430c2b4f6ddd84953ed617
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e39b15195a3607ee708be9d105b5fc591b4665dd 17-Dec-2013 Bill Wendling <isanbard@gmail.com> Merging r197449:
------------------------------------------------------------------------
r197449 | arnolds | 2013-12-16 17:11:01 -0800 (Mon, 16 Dec 2013) | 7 lines

LoopVectorizer: Don't if-convert constant expressions that can trap

A phi node operand or an instruction operand could be a constant expression that
can trap (division). Check that we don't vectorize such cases.

PR16729
radar://15653590
------------------------------------------------------------------------


git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@197453 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
3f297541c5440c4758b34214fdbbf9ae5414c0f1 01-Dec-2013 Bill Wendling <isanbard@gmail.com> Merging r195787:
------------------------------------------------------------------------
r195787 | arnolds | 2013-11-26 14:11:23 -0800 (Tue, 26 Nov 2013) | 8 lines

LoopVectorizer: Truncate i64 trip counts of i32 phis if necessary

In signed arithmetic we could end up with an i64 trip count for an i32 phi.
Because it is signed arithmetic we know that this is only defined if the i32
does not wrap. It is therefore safe to truncate the i64 trip count to a i32
value.

Fixes PR18049.
------------------------------------------------------------------------


git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@195991 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
07a3c481c656c9cc1e0ace3d599eef1fa81e3cc6 18-Nov-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Extend the induction variable to a larger type

In some case the loop exit count computation can overflow. Extend the type to
prevent most of those cases.

The problem is loops like:
int main ()
{
int a = 1;
char b = 0;
lbl:
a &= 4;
b--;
if (b) goto lbl;
return a;
}

The backedge count is 255. The induction variable type is i8. If we add one to
255 to get the exit count we overflow to zero.

To work around this issue we extend the type of the induction variable to i32 in
the case of i8 and i16.

PR17532

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195008 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4634338655449a6e76b6a948c91fbe5bc736d24b 16-Nov-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Use abi alignment for accesses with no alignment

When we vectorize a scalar access with no alignment specified, we have to set
the target's abi alignment of the scalar access on the vectorized access.
Using the same alignment of zero would be wrong because most targets will have a
bigger abi alignment for vector types.

This probably fixes PR17878.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194876 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4921d5b0a9b70d201f18804452aeb62f1c41fcc3 11-Nov-2013 Renato Golin <renato.golin@linaro.org> Move debug message in vectorizer

No functional change, just better reporting.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194388 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ff566d8f4492d7f32814656eaeca75635526d2db 02-Nov-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Remove quadratic behavior the local CSE.

Doing this with a hash map doesn't change behavior and avoids calling
isIdenticalTo O(n^2) times. This should probably eventually move into a utility
class shared with EarlyCSE and the limited CSE in the SLPVectorizer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193926 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
bc28e88a2861ab1183e138f19e92e5d862eaa8a6 02-Nov-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Move cse code into its own function

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193895 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f4775827d046aa12f6aaffd5bd4746744e8fdff8 01-Nov-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Perform redundancy elimination on induction variables

When the loop vectorizer was part of the SCC inliner pass manager gvn would
run after the loop vectorizer followed by instcombine. This way redundancy
(multiple uses) were removed and instcombine could perform scalarization on the
induction variables. Having moved the loop vectorizer to later we no longer run
any form of redundancy elimination before we perform instcombine. This caused
vectorized induction variables to survive that did not before.

On a recent iMac this helps linpack back from 6000Mflops to 7000Mflops.

This should also help lpbench and paq8p.

I ran a Release (without Asserts) build over the test-suite and did not see any
negative impact on compile time.

radar://15339680

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193891 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
7208b0763c9fca2c6364c642d28fd014a496cc37 01-Nov-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Look for consecutive acces in GEPs with trailing zero indices

If we have a pointer to a single-element struct we can still build wide loads
and stores to it (if there is no padding).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193860 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0097e155025767c11790912dcf780f82dffaffb1 01-Nov-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: If dependency checks fail try runtime checks

When a dependence check fails we can still try to vectorize loops with runtime
array bounds checks.

This helps linpack to vectorize a loop in dgefa. And we are back to 2x of the
scalar performance on a corei7-avx.

radar://15339680

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193853 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d272a1223314a69e4678816feeff2cfb3e740f8f 01-Nov-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Clear all member data structures in RuntimeCheck.reset()

Clear all data structures when resetting the RuntimeCheck data structure.

No test case. This was exposed by an upcomming change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193852 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
887f9c5ec15582aec34aa6c28955d01e4e9961e2 26-Oct-2013 Wan Xiaofei <xiaofei.wan@intel.com> Quick look-up for block in loop.

This patch implements quick look-up for block in loop by maintaining a hash set for blocks.
It improves the efficiency of loop analysis a lot, the biggest improvement could be 5-6%(458.sjeng).
Below are the compilation time for our benchmark in llc before & after the patch.

Benchmark llc - trunk llc - patched
401.bzip2 0.339081 100.00% 0.329657 102.86%
403.gcc 19.853966 100.00% 19.605466 101.27%
429.mcf 0.049823 100.00% 0.048451 102.83%
433.milc 0.514898 100.00% 0.510217 100.92%
444.namd 1.109328 100.00% 1.103481 100.53%
445.gobmk 4.988028 100.00% 4.929114 101.20%
456.hmmer 0.843871 100.00% 0.825865 102.18%
458.sjeng 0.754238 100.00% 0.714095 105.62%
464.h264ref 2.9668 100.00% 2.90612 102.09%
471.omnetpp 4.556533 100.00% 4.511886 100.99%
bitmnp01 0.038168 100.00% 0.0357 106.91%
idctrn01 0.037745 100.00% 0.037332 101.11%
libquake2 3.78689 100.00% 3.76209 100.66%
libquake_ 2.251525 100.00% 2.234104 100.78%
linpack 0.033159 100.00% 0.032788 101.13%
matrix01 0.045319 100.00% 0.043497 104.19%
nbench 0.333161 100.00% 0.329799 101.02%
tblook01 0.017863 100.00% 0.017666 101.12%
ttsprk01 0.054337 100.00% 0.053057 102.41%

Reviewer : Andrew Trick <atrick@apple.com>, Hal Finkel <hfinkel@anl.gov>
Approver : Andrew Trick <atrick@apple.com>
Test : Pass make check-all & llvm test-suite


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193460 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
006183a9364660daba786d352df720e079412d60 25-Oct-2013 Hal Finkel <hfinkel@anl.gov> LoopVectorizer: Don't attempt to vectorize extractelement instructions

The loop vectorizer does not currently understand how to vectorize
extractelement instructions. The existing check, which excluded all
vector-valued instructions, did not catch extractelement instructions because
it checked only the return value. As a result, vectorization would proceed,
producing illegal instructions like this:

%58 = extractelement <2 x i32> %15, i32 0
%59 = extractelement i32 %58, i32 0

where the second extractelement is illegal because its first operand is not a vector.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193434 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d6aa89eca5be76dece4b4cd44359e8577e587dff 24-Oct-2013 Renato Golin <renato.golin@linaro.org> Mark vector loops as already vectorized

Make sure we mark all loops (scalar and vector) when vectorizing,
so that we don't try to vectorize them anymore. Also, set unroll
to 1, since this is what we check for on early exit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193349 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
244d24597497c09ab68969c8bbbdf2576130262c 21-Oct-2013 Matt Arsenault <Matthew.Arsenault@amd.com> Use more type helper functions

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193109 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d45b3c4653dc4b18074b04662b6d0009880214e3 15-Oct-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Properly reflect PODness in comments.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192717 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
fe82a3e360fc850e7551f2f1f32b58e539182c68 12-Oct-2013 Tobias Grosser <tobias@grosser.es> LoopVectorize: Add missing INITIALIZE_PASS_DEPENDENCY macros

Contributed-by: Peter Zotov <whitequark@whitequark.org>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192536 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
de2aa608438c1675bb69c2b2087663b9dfcf752b 11-Oct-2013 Renato Golin <renato.golin@linaro.org> Better info when debugging vectorizer

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192460 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
1ee3c0008be6f7012aa69f7a73a2819300bdf23f 07-Oct-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: External uses must use the last value in a reduction cycle

Otherwise, we don't perform operations that would have been performed on
the scalar version.

Fixes PR17498.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192133 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
407847f130885dd9e26e908f033f697c0975aeae 03-Oct-2013 Matt Arsenault <Matthew.Arsenault@amd.com> Don't use runtime bounds check between address spaces.

Don't vectorize with a runtime check if it requires a
comparison between pointers with different address spaces.
The values can't be assumed to be directly comparable.
Previously it would create an illegal bitcast.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191862 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
7b7294c534f97f97860090401672a9c9831033db 02-Oct-2013 Matt Arsenault <Matthew.Arsenault@amd.com> Fix debug printing spacing.

Fix missing newlines, missing and extra spaces in printed messages.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191851 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5c86f1296947df003e22b3a08e241dc26b408e4b 02-Oct-2013 Matt Arsenault <Matthew.Arsenault@amd.com> Fix comment grammar and capitalization.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191850 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8819c84aed10777ba91d4e862229882b8da0b272 01-Oct-2013 Rafael Espindola <rafael.espindola@gmail.com> Remove several unused variables.

Patch by Alp Toker.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191757 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d4278821665aa97f5fc0d19a32ff1fb39a22d395 30-Sep-2013 Benjamin Kramer <benny.kra@googlemail.com> Convert manual insert point restores to the new RAII object.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191675 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
3f4f420ab7acb10221ba971543a7eed5489fb626 28-Sep-2013 Robert Wilhelm <robert.wilhelm@gmx.net> Even more spelling fixes for "instruction".


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191611 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d237e834a816399b7e1561dd4db2c501f5095712 27-Sep-2013 Justin Bogner <mail@justinbogner.com> Transforms: Use getFirstNonPHI to set the insertion point for PHIs

We were previously using getFirstInsertionPt to insert PHI
instructions when vectorizing, but getFirstInsertionPt also skips past
landingpads, causing this to generate invalid IR.

We can avoid this issue by using getFirstNonPHI instead.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191526 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4e7b015a4a724782bf37284c3c74f2540cf8f3e1 23-Sep-2013 Arnold Schwaighofer <aschwaighofer@apple.com> Revert "LoopVectorizer: Only allow vectorization of intrinsics."

Revert 191122 - with extra checks we are allowed to vectorize math library
function calls.

Standard library indentifiers are reserved names so functions with external
linkage must not overrided them. However, functions with internal linkage can.

Therefore, we can vectorize calls to math library functions with a check for
external linkage and matching signature. This matches what we do during
SelectionDAG building.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191206 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
2ecd8c90b037e58f4914acfc6a4ced5a01774a05 21-Sep-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Only allow vectorization of intrinsics. We can't know for sure that the functions 'abs' or 'round' are the functions from libm.

rdar://15012650



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191122 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d072d1b2a36f6c6e1b4b513dc69e91b61ad8c21a 18-Sep-2013 Robert Lytton <robert@xmos.com> Prevent LoopVectorizer and SLPVectorizer running if the target has no vector registers.

XCore target: Add XCoreTargetTransformInfo
This is where getNumberOfRegisters() resides, which in turn returns the
number of vector registers (=0).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190936 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5721d2f674008cd2cfa5696441e46daa035408b3 16-Sep-2013 Arnold Schwaighofer <aschwaighofer@apple.com> Don't vectorize if there are outside loop users of the induction variable.

We would have to compute the pre increment value, either by computing it on
every loop iteration or by splitting the edge out of the loop and inserting a
computation for it there.

For now, just give up vectorizing such loops.

Fixes PR17179.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190790 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8e5eb2b160687bbe3afbe522e4af7518c1bea73f 11-Sep-2013 Eli Friedman <eli.friedman@gmail.com> Don't assert on invalid loop vectorization hint.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190450 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8e12d95d15e4140311919a3b60461817baf68ca5 10-Sep-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: PHI nodes are always at the beginning of a block, no need to scan the whole block.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190422 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
435798e96a64738b55a01055dde1bc9a88a15191 28-Aug-2013 Hal Finkel <hfinkel@anl.gov> Disable unrolling in the loop vectorizer when disabled in the pass manager

When unrolling is disabled in the pass manager, the loop vectorizer should also
not unroll loops. This will allow the -fno-unroll-loops option in Clang to
behave as expected (even for vectorizable loops). The loop vectorizer's
-force-vector-unroll option will (continue to) override the pass-manager
setting (including -force-vector-unroll=0 to force use of the internal
auto-selection logic).

In order to test this, I added a flag to opt (-disable-loop-unrolling) to force
disable unrolling through opt (the analog of -fno-unroll-loops in Clang). Also,
this fixes a small bug in opt where the loop vectorizer was enabled only after
the pass manager populated the queue of passes (the global_alias.ll test needed
a slight update to the RUN line as a result of this fix).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189499 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
bd28f5c8562d537031cc50979bc1609cadbfaad2 27-Aug-2013 Nadav Rotem <nrotem@apple.com> Refactor 'vectorizeLoop' no functionality change.

This patch merges LoopVectorize of InnerLoopVectorizer and InnerLoopUnroller by adding checks for VF=1. This helps in erasing the Unroller code that is almost identical to the InnerLoopVectorizer code.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189391 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4bf6326d0871961a7402c797a0399a2bdaf53574 27-Aug-2013 Nadav Rotem <nrotem@apple.com> LoopVectorize: Implement partial loop unrolling when vectorization is not profitable.
This patch enables unrolling of loops when vectorization is legal but not profitable.
We add a new class InnerLoopUnroller, that extends InnerLoopVectorizer and replaces some of the vector-specific logic with scalars.

This patch does not introduce any runtime regressions and improves the following workloads:

SingleSource/Benchmarks/Shootout/matrix -22.64%
SingleSource/Benchmarks/Shootout-C++/matrix -13.06%
External/SPEC/CINT2006/464_h264ref/464_h264ref -3.99%
SingleSource/Benchmarks/Adobe-C++/simple_types_constant_folding -1.95%



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189281 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
978de6b56a4eb4d3dbc1b65f2b095a192b240f90 22-Aug-2013 Matt Arsenault <Matthew.Arsenault@amd.com> Teach LoopVectorize about address space sizes

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188980 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
66d1fa6f4b443ac9f8bcea5d1f71a73ada733a42 20-Aug-2013 Hal Finkel <hfinkel@anl.gov> Add a llvm.copysign intrinsic

This adds a llvm.copysign intrinsic; We already have Libfunc recognition for
copysign (which is turned into the FCOPYSIGN SDAG node). In order to
autovectorize calls to copysign in the loop vectorizer, we need a corresponding
intrinsic as well.

In addition to the expected changes to the language reference, the loop
vectorizer, BasicTTI, and the SDAG builder (the intrinsic is transformed into
an FCOPYSIGN node, just like the function call), this also adds FCOPYSIGN to a
few lists in LegalizeVector{Ops,Types} so that vector copysigns can be
expanded.

In TargetLoweringBase::initActions, I've made the default action for FCOPYSIGN
be Expand for vector types. This seems correct for all in-tree targets, and I
think is the right thing to do because, previously, there was no way to generate
vector-values FCOPYSIGN nodes (and most targets don't specify an action for
vector-typed FCOPYSIGN).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188728 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
41418d17cced656f91038b2482bc9d173b4974b0 08-Aug-2013 Hal Finkel <hfinkel@anl.gov> Add ISD::FROUND for libm round()

All libm floating-point rounding functions, except for round(), had their own
ISD nodes. Recent PowerPC cores have an instruction for round(), and so here I'm
adding ISD::FROUND so that round() can be custom lowered as well.

For the most part, this is straightforward. I've added an intrinsic
and a matching ISD node just like those for nearbyint() and friends. The
SelectionDAG pattern I've named frnd (because ISD::FP_ROUND has already claimed
fround).

This will be used by the PowerPC backend in a follow-up commit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187926 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
2d66d4cf42022239bfc9cd260a7b924400139cb5 07-Aug-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Allow vectorization of loops with lifetime markers

Patch by Marc Jessome!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187825 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
03fb46bed1a1489725e0da3dea4608d6ef6e6e4b 01-Aug-2013 Nadav Rotem <nrotem@apple.com> 80-col

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187535 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b987c405481336aefdea61ee1df8f5d8ffa143cc 22-Jul-2013 Nadav Rotem <nrotem@apple.com> Fix an obvious typo in the loop vectorizer where the cost model uses the wrong variable. The variable BlockCost is ignored.
We don't have tests for the effect of if-conversion loops because it requires a big test (that includes if-converted loops) and it is difficult to find and balance a loop to do the right thing.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186845 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ae4e1a94e3fb7d1d2d0eec6eb7d04b74326c8453 13-Jul-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Disallow reductions whose header phi is used outside the loop

If an outside loop user of the reduction value uses the header phi node we
cannot just reduce the vectorized phi value in the vector code epilog because
we would loose VF-1 reductions.

lp:
p = phi (0, lv)
lv = lv + 1
...
brcond , lp, outside

outside:
usr = add 0, p

(Say the loop iterates two times, the value of p coming out of the loop is one).

We cannot just transform this to:

vlp:
p = phi (<0,0>, lv)
lv = lv + <1,1>
..
brcond , lp, outside

outside:
p_reduced = p[0] + [1];
usr = add 0, p_reduced

(Because the original loop iterated two times the vectorized loop would iterate
one time, but p_reduced ends up being zero instead of one).

We would have to execute VF-1 iterations in the scalar remainder loop in such
cases. For now, just disable vectorization.

PR16522

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186256 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
9a26e1320b3510623c052324de18b1324f92fee2 13-Jul-2013 Andrew Trick <atrick@apple.com> LoopVectorize fix: LoopInfo must be valid when invoking utils like SCEVExpander.

In general, one should always complete CFG modifications first, update
CFG-based analyses, like Dominatores and LoopInfo, then generate
instruction sequences.

LoopVectorizer was creating a new loop, calling SCEVExpander to
generate checks, then updating LoopInfo. I just changed the order.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186241 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c0a11edba6ea46c782672ab3fb4e4ab3dc267a22 12-Jul-2013 Arnold Schwaighofer <aschwaighofer@apple.com> TargetTransformInfo: address calculation parameter for gather/scather

Address calculation for gather/scather in vectorized code can incur a
significant cost making vectorization unbeneficial. Add infrastructure to add
cost.
Tests and cost model for targets will be in follow-up commits.

radar://14351991

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186187 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
11eb51e23935e22e1cb7b346c45713e8c9169c84 11-Jul-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Vectorize all accesses in address space zero with unit stride

We can vectorize them because in the case where we wrap in the address space the
unvectorized code would have had to access a pointer value of zero which is
undefined behavior in address space zero according to the LLVM IR semantics.
(Thank you Duncan, for pointing this out to me).

Fixes PR16592.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186088 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c14380d1953acd5a735e1d8bcf18d0f96876da74 01-Jul-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Math functions only read rounding mode

Math functions are mark as readonly because they read the floating point
rounding mode. Because we don't vectorize loops that would contain function
calls that set the rounding mode it is safe to ignore this memory read.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185299 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
97b808bf70010c8784aa303870e8a0281cf624e6 29-Jun-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorizer: Pack MemAccessInfo pairs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185263 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
3298179fc15eebfeafdb65aab7512926135b7b39 29-Jun-2013 Benjamin Kramer <benny.kra@googlemail.com> Move helper classes into anonymous namespaces.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185262 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4025fa5b85cf7e3df47c40c2a03e05afe44841fa 28-Jun-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Refactor the code that checks if it is safe to predicate blocks.
In this code we keep track of pointers that we are allowed to read from, if they are accessed by non-predicated blocks.
We use this list to allow vectorization of conditional loads in predicated blocks because we know that these addresses don't segfault.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185214 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0ee20c9d801f8bec14460d6e3fb21d43dfa3918c 28-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Pull dyn_cast into setDebugLocFromInst

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185168 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b0f11e370141c98d24493760e72f9c0d6959150d 28-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Use static function instead of DebugLocSetter class

I used the class to safely reset the state of the builder's debug location. I
think I have caught all places where we need to set the debug location to a new
one. Therefore, we can replace the class by a function that just sets the debug
location.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185165 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
57a7da8b23050f8c78ab7d3a0f44f214183897a0 28-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Preserve debug location info

radar://14169017

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185122 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0bbbf7cbb063e7d966a764b8cda56f96ad326dc6 27-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Cache edge masks created during if-conversion

Otherwise, we end up with an exponential IR blowup.
Fixes PR16472.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185097 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0862d589ee51941f2c84fb110e6b42458767aea7 27-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Use vectorized loop invariant gep index anchored in loop

Use vectorized instruction instead of original instruction anchored in the
original loop.

Fixes PR16452 and t2075.c of PR16455.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185081 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
45ef457b8f92ca030681cd47545fad8b1d693180 27-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Don't store a reversed value in the vectorized value map

When we store values for reversed induction stores we must not store the
reversed value in the vectorized value map. Another instruction might use this
value.

This fixes 3 test cases of PR16455.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185051 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
bc7c58d2b1a04514ee19f59a58e0027798e59d56 24-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> Reapply 184685 after the SetVector iteration order fix.

This should hopefully have fixed the stage2/stage3 miscompare on the dragonegg
testers.

"LoopVectorize: Use the dependence test utility class

We now no longer need alias analysis - the cases that alias analysis would
handle are now handled as accesses with a large dependence distance.

We can now vectorize loops with simple constant dependence distances.

for (i = 8; i < 256; ++i) {
a[i] = a[i+4] * a[i+8];
}

for (i = 8; i < 256; ++i) {
a[i] = a[i-4] * a[i-8];
}

We would be able to vectorize about 200 more loops (in many cases the cost model
instructs us no to) in the test suite now. Results on x86-64 are a wash.

I have seen one degradation in ammp. Interestingly, the function in which we
now vectorize a loop is never executed so we probably see some instruction
cache effects. There is a 2% improvement in h264ref. There is one or the other
TSCV loop kernel that speeds up.

radar://13681598"

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184724 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
7e96b4dfce63f967f150c617f0a69ded9f1f7416 24-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Use SetVector for the access set

We are creating the runtime checks using this set so we need a deterministic
iteration order.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184723 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ec677e2a6470c5fc6753a857e7d64254ff9ca19f 24-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> Revert "LoopVectorize: Use the dependence test utility class"

This reverts commit cbfa1ca993363ca5c4dbf6c913abc957c584cbac.

We are seeing a stage2 and stage3 miscompare on some dragonegg bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184690 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
cbfa1ca993363ca5c4dbf6c913abc957c584cbac 24-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Use the dependence test utility class

We now no longer need alias analysis - the cases that alias analysis would
handle are now handled as accesses with a large dependence distance.

We can now vectorize loops with simple constant dependence distances.

for (i = 8; i < 256; ++i) {
a[i] = a[i+4] * a[i+8];
}

for (i = 8; i < 256; ++i) {
a[i] = a[i-4] * a[i-8];
}

We would be able to vectorize about 200 more loops (in many cases the cost model
instructs us no to) in the test suite now. Results on x86-64 are a wash.

I have seen one degradation in ammp. Interestingly, the function in which we
now vectorize a loop is never executed so we probably see some instruction
cache effects. There is a 2% improvement in h264ref. There is one or the other
TSCV loop kernel that speeds up.

radar://13681598

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184685 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5df684a6d1bc655c93df4d2f1efafe9093420f4f 24-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Add utility class for checking dependency among accesses

This class checks dependences by subtracting two Scalar Evolution access
functions allowing us to catch very simple linear dependences.

The checker assumes source order in determining whether vectorization is safe.
We currently don't reorder accesses.
Positive true dependencies need to be a multiple of VF otherwise we impede
store-load forwarding.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184684 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5737d3f433782e22ffca4433a1de2ab119464715 24-Jun-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Add utility class for building sets of dependent accesses

Sets of dependent accesses are built by unioning sets based on underlying
objects. This class will be used by the upcoming dependence checker.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184683 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a8a04380c597e1cdb8d635abd9e2669eab401545 17-Jun-2013 Pekka Jaaskelainen <pekka.jaaskelainen@tut.fi> Fix for a regression caused by the LoopVectorizer when
vectorizing loops with memory accesses to non-zero address spaces. It
simply dropped the AS info. Fixes PR16306.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184103 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
35b4cf868e2da677214d97a12d1f34ce6811a088 31-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Change API call to get the backedge taken count

Use ScalarEvolution's getBackedgeTakenCount API instead of getExitCount since
that is really what we want to know. Using the more specific getExitCount was
safe because we made sure that there is only one exiting block.

No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183047 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
47afc19625e97a3716d813f014a3a48468e591ca 31-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: PHIs with only outside users should prevent vectorization

We check that instructions in the loop don't have outside users (except if
they are reduction values). Unfortunately, we skipped this check for
if-convertable PHIs.

Fixes PR16184.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183035 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0cf4adbcef584cb56254e253b47315b4c3d073ec 29-May-2013 NAKAMURA Takumi <geek4civic@gmail.com> LoopVectorize.cpp: Fix abuse of StringRef on Twine. Twine captures the pointer of StringRef.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182820 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
51c292a3605a0cf7be24e4b7dc40c2b8a740006c 29-May-2013 NAKAMURA Takumi <geek4civic@gmail.com> Whitespace.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182819 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ee21b6f7b41e3fc19031f6d410b2ebe6a1a2f361 28-May-2013 Paul Redmond <paul.redmond@intel.com> Add support for llvm.vectorizer metadata

- llvm.loop.parallel metadata has been renamed to llvm.loop to be more generic
by making the root of additional loop metadata.
- Loop::isAnnotatedParallel now looks for llvm.loop and associated
llvm.mem.parallel_loop_access
- document llvm.loop and update llvm.mem.parallel_loop_access
- add support for llvm.vectorizer.width and llvm.vectorizer.unroll
- document llvm.vectorizer.* metadata
- add utility class LoopVectorizerHints for getting/setting loop metadata
- use llvm.vectorizer.width=1 to indicate already vectorized instead of
already_vectorized
- update existing tests that used llvm.loop.parallel and
llvm.vectorizer.already_vectorized

Reviewed by: Nadav Rotem


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182802 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
959ecb2eec9cffd762edc46fd496a7bb6c65a2e6 24-May-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: LoopSimplify can't canonicalize loops with an indirectbr in it, don't assert on those cases.

Fixes PR16139.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182656 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6e4a9c14f6e59b075b3a24825d2dc7dc9e163281 22-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Make Value pointers that could be RAUW'ed a VH

The Value pointers we store in the induction variable list can be RAUW'ed by a
call to SCEVExpander::expandCodeFor, use a TrackingVH instead. Do the same thing
in some other places where we store pointers that could potentially be RAUW'ed.

Fixes PR16073.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182485 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
688b5103eb4b70410d1ea54bcd8df4ef2f2ff8e8 18-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Handle single edge PHIs

We might encouter single edge PHIs - handle them with an identity select.

Fixes PR15990.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182199 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c53bee6eaeac821ca53c81efa1dfc9495a72d24b 17-May-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Simplify code. No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182100 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0aad08adfda3ff7c4f9c453ba821057b5436c4ef 16-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Move call of canHoistAllLoads to canVectorizeWithIfConvert

We only want to check this once, not for every conditional block in the loop.

No functionality change (except that we don't perform a check redudantly
anymore).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181942 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e3ab7cca3abbc2dc5056695582b7a46abcc245ef 15-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Fix comments

No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181862 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
1386692ef64d3151da8986589eadf0c58aba5c50 15-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Hoist conditional loads if possible

InstCombine can be uncooperative to vectorization and sink loads into
conditional blocks. This prevents vectorization.

Undo this optimization if there are unconditional memory accesses to the same
addresses in the loop.

radar://13815763

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181860 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
123f18bcb9baeb6dc177cb642126a3a4d9ca8b43 14-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Handle loops with multiple forward inductions

We used to give up if we saw two integer inductions. After this patch, we base
further induction variables on the chosen one like we do in the reverse
induction and pointer induction case.

Fixes PR15720.

radar://13851975

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181746 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
9b5d70f07630f99f1ec5589aeaba96c6d8ab0aee 12-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Use the widest induction variable type

Use the widest induction type encountered for the cannonical induction variable.

We used to turn the following loop into an empty loop because we used i8 as
induction variable type and truncated 1024 to 0 as trip count.

int a[1024];
void fail() {
int reverse_induction = 1023;
unsigned char forward_induction = 0;
while ((reverse_induction) >= 0) {
forward_induction++;
a[reverse_induction] = forward_induction;
--reverse_induction;
}
}

radar://13862901

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181667 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6bb539a643b385520c25f3b03199f68012b9c09b 12-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Use variable instead of repeated function call

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181666 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8112d227017f70b870217b4c96841d1dc5e95277 12-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Use IRBuilder interface in more places

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181665 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c121f5dc267e5d7048b8f27ddcfdc41f8c8e7073 09-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Don't assert on the absence of induction variables

A computable loop exit count does not imply the presence of an induction
variable. Scalar evolution can return a value for an infinite loop.

Fixes PR15926.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181495 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
280e1df8585e62f0c801de8e4b625a7e73178d85 07-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Improve reduction variable identification

The two nested loops were confusing and also conservative in identifying
reduction variables. This patch replaces them by a worklist based approach.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181369 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
eb95cec176ad4154ac4ed509baf1c9c51ca29be0 07-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: getConsecutiveVector must respect signed arithmetic

We were passing an i32 to ConstantInt::get where an i64 was needed and we must
also pass the sign if we pass negatives numbers. The start index passed to
getConsecutiveVector must also be signed.

Should fix PR15882.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181286 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
37d38b7668f3dd64e2263426c29253ace50865b3 06-May-2013 Nadav Rotem <nrotem@apple.com> Update the comment to mention that we use TTI.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181178 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
acc47c738d5888f7445275095a174c7f48296449 05-May-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Print values instead of pointers in debug output.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181157 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
87defd0924e08dd9c9db51e2fb208f289fa6adf7 05-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Add support for floating point min/max reductions

Add support for min/max reductions when "no-nans-float-math" is enabled. This
allows us to assume we have ordered floating point math and treat ordered and
unordered predicates equally.

radar://13723044

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181144 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f852472823fd2182a3ca54bdf4d30ad8a6a6cd57 05-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Cleanup of miminimum/maximum pattern match code

No need for setting the operands. The pointers are going to be bound by the
matcher.

radar://13723044

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181142 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c1738fdadd1c969f13bbf09fe9c36fff56ccd709 05-May-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: We don't need an identity element for min/max reductions

We can just use the initial element that feeds the reduction.

max(max(x, y), z) == max(max(x,y), max(x,z))

radar://13723044

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181141 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5c332dbd30d9398ed25b30c3080506f7b8e92290 05-May-2013 Dmitri Gribenko <gribozavr@gmail.com> Add ArrayRef constructor from None, and do the cleanups that this constructor enables

Patch by Robert Wilhelm.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181138 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4bcd5f888fa762613cf8096a79ba7b8a72665de2 03-May-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Add support for if-conversion of PHINodes with 3+ incoming values.
By supporting the vectorization of PHINodes with more than two incoming values we can increase the complexity of nested if statements.

We can now vectorize this loop:

int foo(int *A, int *B, int n) {
for (int i=0; i < n; i++) {
int x = 9;
if (A[i] > B[i]) {
if (A[i] > 19) {
x = 3;
} else if (B[i] < 4 ) {
x = 4;
} else {
x = 5;
}
}
A[i] = x;
}
}



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181037 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
7557e521e50180817a165f8c897220b0e2020b7b 26-Apr-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Calculate the number of pointers to disambiguate at runtime based on the numbers of reads and writes.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180593 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
975b1ddf60387139357c8cbbaeb613de5a39294f 25-Apr-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: No need to generate pointer disambiguation checks between readonly pointers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180570 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
84875bad9c32b1110c6b8abd4f1346cc0dca761b 24-Apr-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Change variable name Stride to ConsecutiveStride

This makes it easier to read the code.

No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180197 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a4b8b4ccc9ecd6863606c7cdde6ec1b38734708a 24-Apr-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Scalarize padded types

This patch disables memory-instruction vectorization for types that need padding
bytes, e.g., x86_fp80 has 10 bytes store size with 6 bytes padding in darwin on
x86_64. Because the load/store vectorization is performed by the bit casting to
a packed vector, which has incompatible memory layout due to the lack of padding
bytes, the present vectorizer produces inconsistent result for memory
instructions of those types.
This patch checks an equality of the AllocSize of a scalar type and allocated
size for each vector element, to ensure that there is no padding bytes and the
array can be read/written using vector operations.

Patch by Daisuke Takahashi!

Fixes PR15758.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180196 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b03ad17536097a0f172428c939e80ce7657e201b 24-Apr-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Bail out if we don't have datalayout we need it

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180195 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a7d9a6ee63bec70fecea79b85a30108ed3e8fabd 23-Apr-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Fix 15830. When scalarizing and unrolling stores make sure that the order in which the elements are scalarized is the same as the original order.
This fixes a miscompilation in FreeBSD's regex library.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180121 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
2e59a125fce4235beeb4e72cafbcbe62854ed77c 23-Apr-2013 Pekka Jaaskelainen <pekka.jaaskelainen@tut.fi> Call the potentially costly isAnnotatedParallel() only once.
Made the uniform write test's checks a bit stricter.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180119 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a8958769ea5583d58ae51931d8b1271c23fd20ea 23-Apr-2013 Pekka Jaaskelainen <pekka.jaaskelainen@tut.fi> Refuse to (even try to) vectorize loops which have uniform writes,
even if erroneously annotated with the parallel loop metadata.

Fixes Bug 15794:
"Loop Vectorizer: Crashes with the use of llvm.loop.parallel metadata"



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180081 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d717e202a2d50d47b96534dbf67b8aa6ea01b912 19-Apr-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Use matcher from PatternMatch.h for the min/max patterns

Also make some static function class functions to avoid having to mention the
class namespace for enums all the time.

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179886 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4b15d6ae49b407cfcf526a3f7bec7ab15fa803c0 18-Apr-2013 Dmitri Gribenko <gribozavr@gmail.com> Fix a -Wdocumentation warning


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179789 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a3fb330d05e85107d01ecf133355d0c6a88196fd 18-Apr-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Recognize min/max reductions

A min/max operation is represented by a select(cmp(lt/le/gt/ge, X, Y), X, Y)
sequence in LLVM. If we see such a sequence we can treat it just as any other
commutative binary instruction and reduce it.

This appears to help bzip2 by about 1.5% on an imac12,2.

radar://12960601

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179773 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
403fc143709bfe5cf014929d633c0b1c28f6e0c9 18-Apr-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Use a set to avoid longer cycles in the reduction chain too.

Fixes PR15748.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179757 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
08a0e8f8dbae6314dade60e1ef92e73e89c5ff23 12-Apr-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: integer division is not a reduction operation

Don't classify idiv/udiv as a reduction operation. Integer division is lossy.
For example : (1 / 2) * 4 != 4/2.

Example:

int a[] = { 2, 5, 2, 2}
int x = 80;

for()
x /= a[i];

Scalar:
x /= 2 // = 40
x /= 5 // = 8
x /= 2 // = 4
x /= 2 // = 2

Vectorized:

<80, 1> / <2,5> //= <40,0>
<40, 0> / <2,2> //= <20,0>

20*0 = 0

radar://13640654

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179381 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ac2cc0170f7d13e7c2e87f7728c99a6d07876f5c 05-Apr-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Pass OperandValueKind information to the cost model

Pass down the fact that an operand is going to be a vector of constants.

This should bring the performance of MultiSource/Benchmarks/PAQ8p/paq8p on x86
back. It had degraded to scalar performance due to my pervious shift cost change
that made all shifts expensive on x86.

radar://13576547

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178809 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d517da33b712b5d8d687ee5e5974056a4787ec4f 14-Mar-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorize: Invert case when we use a vector cmp value to query select cost

We generate a select with a vectorized condition argument when the condition is
NOT loop invariant. Not the other way around.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177098 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0d932717d8d22b0e747b15fddea2c718043e4d51 09-Mar-2013 Benjamin Kramer <benny.kra@googlemail.com> Remove a source of nondeterminism from the LoopVectorizer.

This made us emit runtime checks in a random order. Hopefully bootstrap
miscompares will go away now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176775 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
738295e4570f502360c11bc51843f5a8516a9526 09-Mar-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Ignore all dbg intrinisic

Ignore all DbgIntriniscInfo instructions instead of just DbgValueInst.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176769 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
56ee544a3af2b019329f06422c00e8a3645b895c 09-Mar-2013 Arnold Schwaighofer <aschwaighofer@apple.com> LoopVectorizer: Ignore dbg.value instructions

We want vectorization to happen at -g. Ignore calls to the dbg.value intrinsic
and don't transfer them to the vectorized code.

radar://13378964

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176768 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f22d9cfa6d145ce26930804cc3ac54340f49c38b 08-Mar-2013 Benjamin Kramer <benny.kra@googlemail.com> Insert the reduction start value into the first bypass block to preserve domination.

Fixes PR15344.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176701 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5290baacb8ca4fb75d798e873a441cad11cbfb2c 02-Mar-2013 Nadav Rotem <nrotem@apple.com> PR14448 - prevent the loop vectorizer from vectorizing the same loop twice.
The LoopVectorizer often runs multiple times on the same function due to inlining.
When this happens the loop vectorizer often vectorizes the same loops multiple times, increasing code size and adding unneeded branches.
With this patch, the vectorizer during vectorization puts metadata on scalar loops and marks them as 'already vectorized' so that it knows to ignore them when it sees them a second time.

PR14448.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176399 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5d79bb8770a5a655af0dccc87b952c3ea9bad45e 01-Mar-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Don't hang forever if a PHI only has skipped PHI uses.

Fixes PR15384.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176366 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
2fe71f8f7e678174ba217d3d16525092805ad45b 27-Feb-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Vectorize math builtin calls.

This properly asks TargetLibraryInfo if a call is available and if it is, it
can be translated into the corresponding LLVM builtin. We don't vectorize sqrt()
yet because I'm not sure about the semantics for negative numbers. The other
intrinsic should be exact equivalents to the libm functions.

Differential Revision: http://llvm-reviews.chandlerc.com/D465

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176188 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e18bce5317ff9f64b3c02418f28c6d383d88b294 21-Feb-2013 Renato Golin <renato.golin@linaro.org> Allow GlobalValues to vectorize with AliasAnalysis

Storing the load/store instructions with the values
and inspect them using Alias Analysis to make sure
they don't alias, since the GEP pointer operand doesn't
take the offset into account.

Trying hard to not add any extra cost to loads and stores
that don't overlap on global values, AA is *only* calculated
if all of the previous attempts failed.

Using biggest vector register size as the stride for the
vectorization access, as we're being conservative and
the cost model (which calculates the real vectorization
factor) is only run after the legalization phase.

We might re-think this relationship in the future, but
for now, I'd rather be safe than sorry.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175818 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c0a6e070fc8fccb86ed91d503a7efc64a2abaa14 13-Feb-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Simplify code for clarity.

No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175076 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5d0ce79e26f40141f35cc0002dc5cc6060382359 13-Feb-2013 Pekka Jaaskelainen <pekka.jaaskelainen@tut.fi> Metadata for annotating loops as parallel. The first consumer for this
metadata is the loop vectorizer.

See the documentation update for more info.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175060 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
9de31bd8298ae4c94c13bef9b8984745bfe41a60 08-Feb-2013 Jakob Stoklund Olesen <stoklund@2pi.dk> Typos.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174723 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
fb55a8fd7c38aa09d9c243d48a8a72d890f36a3d 08-Feb-2013 Arnold Schwaighofer <aschwaighofer@apple.com> ARM cost model: Address computation in vector mem ops not free

Adds a function to target transform info to query for the cost of address
computation. The cost model analysis pass now also queries this interface.
The code in LoopVectorize adds the cost of address computation as part of the
memory instruction cost calculation. Only there, we know whether the instruction
will be scalarized or not.
Increase the penality for inserting in to D registers on swift. This becomes
necessary because we now always assume that address computation has a cost and
three is a closer value to the architecture.

radar://13097204

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174713 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
bcf60de5eb05e255f0c1bcf7ce26125c57026b65 08-Feb-2013 Michael Kuperstein <michael.m.kuperstein@intel.com> Test Commit

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174709 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4fae15e664290a974d44145f4a178ec06e8538e9 07-Feb-2013 Nadav Rotem <nrotem@apple.com> fix 80-col violation and fix the docs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174671 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
96a6555b5706af59e408bb190c3685f0f15bc2a9 07-Feb-2013 Arnold Schwaighofer <aschwaighofer@apple.com> Loop Vectorizer: Refactor Memory Cost Computation

We don't want too many classes in a pass and the classes obscure the details. I
was going a little overboard with object modeling here. Replace classes by
generic code that handles both loads and stores.

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174646 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
7d80dad40741ecc2e45943f685bf539eeed2ba51 05-Feb-2013 Arnold Schwaighofer <aschwaighofer@apple.com> Loop Vectorizer: Refactor code to compute vectorized memory instruction cost

Introduce a helper class that computes the cost of memory access instructions.
No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174422 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
935645b7655a0b5189d40b3d65b3bcb14e30d859 05-Feb-2013 Arnold Schwaighofer <aschwaighofer@apple.com> Loop Vectorizer: Handle pointer stores/loads in getWidestType()

In the loop vectorizer cost model, we used to ignore stores/loads of a pointer
type when computing the widest type within a loop. This meant that if we had
only stores/loads of pointers in a loop we would return a widest type of 8bits
(instead of 32 or 64 bit) and therefore a vector factor that was too big.

Now, if we see a consecutive store/load of pointers we use the size of a pointer
(from data layout).

This problem occured in SingleSource/Benchmarks/Shootout-C++/hash.cpp (reduced
test case is the first test in vector_ptr_load_store.ll).

radar://13139343

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174377 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d8550495768d87f9b0d3219c40a5b15518ccacc8 29-Jan-2013 Pekka Jaaskelainen <pekka.jaaskelainen@tut.fi> LoopVectorize: convert TinyTripCountVectorThreshold constant
to a command line switch.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173837 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8c3a411cd6a193672438a80fa37fbff690833a0c 29-Jan-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Clean up ValueMap a bit and avoid double lookups.

No intended functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173809 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
237f09db096f88b8bdaf7ef51fad480c52af5e50 28-Jan-2013 Renato Golin <renato.golin@linaro.org> Vectorization Factor clarification

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173691 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c655c43ba54990535cf7f374ed78f4fd5bdee093 25-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorize: Refactor the code that vectorizes loads/stores to remove duplication.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173500 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5ac70a3804148749a15edf6113867a8bdae12974 25-Jan-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Simplify code. No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173475 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
1275708f2980918230eb0e9dec78890de08a94f4 25-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Refactor more code to use the IRBuilder.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173471 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f8c2f90bb3c70ab1830b44da990bda499f1e65d1 25-Jan-2013 Nadav Rotem <nrotem@apple.com> Refactor some code to use the IRBuilder.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173467 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f148c66ce4c22130ff1ae242582e024ea18492bb 23-Jan-2013 Nadav Rotem <nrotem@apple.com> Add support for reverse pointer induction variables. These are loops that contain pointers that count backwards.
For example, this is the hot loop in BZIP:

do {
m = *--p;
*p = ( ... );
} while (--n);



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173219 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
dc0a5fd226f85711018663526b4acd1c886367d3 21-Jan-2013 Nadav Rotem <nrotem@apple.com> Fix a comment. Induction vars dont need to start at zero.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173061 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5ff7a3f947c245df9ae95a381ef38184527e83e1 20-Jan-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Fix a C++11 incompatibility.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172990 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e6e7606118d6862747a84ecfb0d70f3c0cf601af 20-Jan-2013 Nadav Rotem <nrotem@apple.com> Fix a build error.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172971 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0bbbc52dc8f1d7d41ab06e0d84daf990c8bc7f93 20-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Implement a new heuristics for selecting the unroll factor.
We ignore the cpu frontend and focus on pipeline utilization. We do this because we
don't have a good way to estimate the loop body size at the IR level.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172964 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
1af132dcf3c9fd87ac8ad0c103e74ef3f8a0bae6 19-Jan-2013 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorizer: Emit memory checks into their own basic block.

This separates the check for "too few elements to run the vector loop" from the
"memory overlap" check, giving a lot nicer code and allowing to skip the memory
checks when we're not going to execute the vector code anyways. We still leave
the decision of whether to emit the memory checks as branches or setccs, but it
seems to be doing a good job. If ugly code pops up we may want to emit them as
separate blocks too. Small speedup on MultiSource/Benchmarks/MallocBench/espresso.

Most of this is legwork to allow multiple bypass blocks while updating PHIs,
dominators and loop info.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172902 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6a459e65c8bd90e8381361379255ebfb9552b5cf 15-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer cost model. Honor the user command line flag that selects the vectorization factor even if the target machine does not have any vector registers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172544 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b6db95f42b7c3b58f980e387d20ddb3e16bffd56 13-Jan-2013 Nadav Rotem <nrotem@apple.com> Fix PR14547. Handle induction variables of small sizes smaller than i32 (i8 and i16).



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172348 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a675c74208f77351ea7fa3eed4f542ae781ab566 11-Jan-2013 Nadav Rotem <nrotem@apple.com> ARM Cost Model: We need to detect the max bitwidth of types in the loop in order to select the max vectorization factor.
We don't have a detailed analysis on which values are vectorized and which stay scalars in the vectorized loop so we use
another method. We look at reduction variables, loads and stores, which are the only ways to get information in and out
of loop iterations. If the data types are extended and truncated then the cost model will catch the cost of the vector
zext/sext/trunc operations.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172178 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c560bf638b74eb48347a7b945b90aa89ffcc1620 10-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Fix a bug in the vectorization of BinaryOperators. The BinaryOperator can be folded to an Undef, and we don't want to set NSW flags to undef vals.

PR14878



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172079 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
14925e6b885f8bd8cf448627386d412831f4bf1b 09-Jan-2013 Nadav Rotem <nrotem@apple.com> ARM Cost model: Use the size of vector registers and widest vectorizable instruction to determine the max vectorization factor.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172010 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
83be7b0dd3ae9a3cb22d36ae4c1775972553b94b 09-Jan-2013 Nadav Rotem <nrotem@apple.com> Cost Model: Move the 'max unroll factor' variable to the TTI and add initial Cost Model support on ARM.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171928 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8327474e4b2c69d5eca4d62a01b89508705d0b40 08-Jan-2013 Nadav Rotem <nrotem@apple.com> Code cleanup: refactor the switch statements in the generation of reduction variables into an IR builder call.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171871 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
aae3d6fb53e1c472d9e25641a15a68c3c72c7129 08-Jan-2013 Nadav Rotem <nrotem@apple.com> Rename the enum members to match the LLVM coding style.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171868 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
111e5fe7e089e0ffe73873848315ea5358120dfa 08-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Add support for floating point reductions

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171812 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
9a6c6a373629fb5a3cc5afd08aafa51339df95df 07-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: When we vectorizer and widen loops we process many elements at once. This is a good thing, except for
small loops. On small loops post-loop that handles scalars (and runs slower) can take more time to execute than the
rest of the loop. This patch disables widening of loops with a small static trip count.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171798 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
1cbeaeb1944f6fd1ab0997201a47ea6c23e9a979 07-Jan-2013 Chandler Carruth <chandlerc@gmail.com> Simplify LoopVectorize to require target transform info and rely on it
being present. Make a member of one of the helper classes a reference as
part of this.

Reformatting goodness brought to you by clang-format.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171726 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f3252b12e02b1fcf01abf0a79b761c53de5985d0 07-Jan-2013 Chandler Carruth <chandlerc@gmail.com> Merge the unused header file for LoopVectorizer into the source file.
This makes the loop vectorizer match the pattern followed by roughly all
other passses. =]

Notably, this header file was braken in several regards: it contained
a using namespace directive, global #define's that aren't globaly
appropriate, and global constants defined directly in the header file.

As a side benefit, lots of the types in this file become internal, which
will cause the optimizer to chew on this pass more effectively.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171723 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
bb00800ff46e7a2a628d0a6741a7f0422c74c198 07-Jan-2013 Chandler Carruth <chandlerc@gmail.com> Fix the enumerator names for ShuffleKind to match tho coding standards,
and make its comments doxygen comments.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171688 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
be04929f7fd76a921540e9901f24563e51dc1219 07-Jan-2013 Chandler Carruth <chandlerc@gmail.com> Move TargetTransformInfo to live under the Analysis library. This no
longer would violate any dependency layering and it is in fact an
analysis. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171686 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
be73c7b903647221fbcaae302d31e90f53583040 05-Jan-2013 Chandler Carruth <chandlerc@gmail.com> Switch the loop vectorizer from VTTI to just use TTI directly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171620 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d5b92c389133c5d587e4094af553ec345ed40045 05-Jan-2013 Nadav Rotem <nrotem@apple.com> iLoopVectorize: Non commutative operators can be used as reduction variables as long as the reduction chain is used in the LHS.

PR14803.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171583 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5767d919569f7b4be51c0a13bff24f722625d9f3 04-Jan-2013 Paul Redmond <paul.redmond@intel.com> Do not vectorize loops with subtraction reductions

Since subtraction does not commute the loop vectorizer incorrectly vectorizes
reductions such as x = A[i] - x.

Disabling for now.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171537 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
608467d1e1b318220307143d3af81f31cd7a18f8 04-Jan-2013 Nadav Rotem <nrotem@apple.com> Fix a warning

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171525 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e503319874f57ab4a0354521b03a71cf8e07b866 04-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer:

1. Add code to estimate register pressure.
2. Add code to select the unroll factor based on register pressure.
3. Add bits to TargetTransformInfo to provide the number of registers.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171469 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e4159491a7d94f87f99fb99a15c76d5d7b26851c 03-Jan-2013 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Add support for loop-unrolling during vectorization for increasing the ILP. At the moment this feature is disabled by default and this commit should not cause any functional changes.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171436 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
00a6bcaeb4a74b930bc2b21f8021c3358d507ecf 03-Jan-2013 Nadav Rotem <nrotem@apple.com> Avoid vectorization when the function has the "noimplicitflot" attribute.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171429 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0b8c9a80f20772c3793201ab5b251d3520b9cea3 02-Jan-2013 Chandler Carruth <chandlerc@gmail.com> Move all of the header files which are involved in modelling the LLVM IR
into their new header subdirectory: include/llvm/IR. This matches the
directory structure of lib, and begins to correct a long standing point
of file layout clutter in LLVM.

There are still more header files to move here, but I wanted to handle
them in separate commits to make tracking what files make sense at each
layer easier.

The only really questionable files here are the target intrinsic
tablegen files. But that's a battle I'd rather not fight today.

I've updated both CMake and Makefile build systems (I think, and my
tests think, but I may have missed something).

I've also re-sorted the includes throughout the project. I'll be
committing updates to Clang, DragonEgg, and Polly momentarily.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171366 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6c3074958370bf25dc6e4e4b757f0c083e245dbe 01-Jan-2013 Benjamin Kramer <benny.kra@googlemail.com> Add IRBuilder::CreateVectorSplat and use it to simplify code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171349 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
831737d329a727f53a1fb0572f7b7a8127208881 30-Dec-2012 Bill Wendling <isanbard@gmail.com> Remove the Function::getFnAttributes method in favor of using the AttributeSet
directly.

This is in preparation for removing the use of the 'Attribute' class as a
collection of attributes. That will shift to the AttributeSet class instead.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171253 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
db2367512e87dbd7b93c3250ef30c9df5e40cb43 30-Dec-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Fix a bug in the code that updates the loop exiting block.
LCSSA PHIs may have undef values. The vectorizer updates values that are used by outside users such as PHIs.
The bug happened because undefs are not loop values. This patch handles these PHIs.

PR14725



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171251 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5dd839430c1dbce6cd35dc44f68718a1fc69bfba 27-Dec-2012 Nadav Rotem <nrotem@apple.com> If all of the write objects are identified then we can vectorize the loop even if the read objects are unidentified.

PR14719.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171124 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
13eb1e7817be11ea84be6571dce827a77bc9640b 26-Dec-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Optimize the vectorization of consecutive memory access when the iteration step is -1

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171114 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
1d59f5fa53cac23b6debc1d7214451c65b0399a7 26-Dec-2012 Hal Finkel <hfinkel@anl.gov> LoopVectorize: Enable vectorization of the fmuladd intrinsic

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171076 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
9e5329d77e590f757dbd8384f418e44df9dbf91a 24-Dec-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: When checking for vectorizable types, also check
the StoreInst operands.

PR14705.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171023 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
470ea9b72f87f2ce4bb09fc6f9829211a090652a 24-Dec-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Fix an endless loop in the code that looks for reductions.
The bug was in the code that detects PHIs in if-then-else block sequence.

PR14701.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171008 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a1acf55738a9d6c0376c7d36b27897817685e157 23-Dec-2012 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Fix accidentaly inverted condition.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171001 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
417872ed08128e8885b4c7fbaeb2d735c150ea57 23-Dec-2012 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: For scalars and void types there is no need to compute vector insert/extract costs.

Fixes an assert during the build of oggenc in the test suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171000 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d54fed27865dcbc69932e1e6c372bb5a932e662a 23-Dec-2012 Nadav Rotem <nrotem@apple.com> Loop Vectorizer: Update the cost model of scatter/gather operations and make
them more expensive.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170995 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
629fb82419d9bfff6ae475363bcce66192dfcc8e 22-Dec-2012 Bill Wendling <isanbard@gmail.com> Change 'AttrVal' to 'AttrKind' to better reflect that it's a kind of attribute instead of the value of the attribute.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170972 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
38b06020dbd804f01ee3802779a52c05cffdf87d 21-Dec-2012 Roman Divacky <rdivacky@freebsd.org> Remove duplicate includes.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170902 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ebf395d39ee81d1635783688dbc49ff86c24e7e6 21-Dec-2012 Nadav Rotem <nrotem@apple.com> Enable if-conversion.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170841 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
55306bdea5d2d53be39f3ac59fadf5220ee6b5d0 21-Dec-2012 Nadav Rotem <nrotem@apple.com> Fix a bug in the code that checks if we can vectorize loops while using dynamic
memory bound checks. Before the fix we were able to vectorize this loop from
the Livermore Loops benchmark:

for ( k=1 ; k<n ; k++ )
x[k] = x[k-1] + y[k];



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170811 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8386acd7348003d13a2db3c4dd3274653d8ffe10 20-Dec-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Fix a bug in the scalarization of instructions.
Before if-conversion we could check if a value is loop invariant
if it was declared inside the basic block. Now that loops have
multiple blocks this check is incorrect.

This fixes External/SPEC/CINT95/099_go/099_go



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170756 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d5d46ace89d2b7ec1a54610925bc0f13bb394335 20-Dec-2012 Nadav Rotem <nrotem@apple.com> Loop Vectorizer: turn-off if-conversion.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170708 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d3071bb042303ed239e5c770fd4026bfc339fcd0 20-Dec-2012 Nadav Rotem <nrotem@apple.com> Loop Vectorizer: Enable if-conversion.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170632 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
034b94b17006f51722886b0f2283fb6fb19aca1f 19-Dec-2012 Bill Wendling <isanbard@gmail.com> Rename the 'Attributes' class to 'Attribute'. It's going to represent a single attribute in the future.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170502 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0ef0e2e6d0a45cdbc792eee9d76f0a4b7cda5c8f 18-Dec-2012 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops instead of scalar operations.

For example on x86 with SSE4.2 a <8 x i8> add reduction becomes
movdqa %xmm0, %xmm1
movhlps %xmm1, %xmm1 ## xmm1 = xmm1[1,1]
paddw %xmm0, %xmm1
pshufd $1, %xmm1, %xmm0 ## xmm0 = xmm1[1,0,0,0]
paddw %xmm1, %xmm0
phaddw %xmm0, %xmm0
pextrb $0, %xmm0, %edx

instead of
pextrb $2, %xmm0, %esi
pextrb $0, %xmm0, %edx
addb %sil, %dl
pextrb $4, %xmm0, %esi
addb %dl, %sil
pextrb $6, %xmm0, %edx
addb %sil, %dl
pextrb $8, %xmm0, %esi
addb %dl, %sil
pextrb $10, %xmm0, %edi
pextrb $14, %xmm0, %edx
addb %sil, %dil
pextrb $12, %xmm0, %esi
addb %dil, %sil
addb %sil, %dl

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170439 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
9ad73e93a51c473a90e010a489a9c7c221112030 14-Dec-2012 Nadav Rotem <nrotem@apple.com> Enable the Loop Vectorizer by default for O2 and O3. Disable if-conversion by default. I plan to revert this patch later today.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170157 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
807dad62a0d4f1a1bbcb76fdc447634f76308252 13-Dec-2012 Nadav Rotem <nrotem@apple.com> Teach the cost model about the optimization in r169904: Truncation of induction variables costs the same as scalar trunc.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170051 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d0b144c04c10c62ad51d0e171680d9ce95af4078 12-Dec-2012 Nadav Rotem <nrotem@apple.com> Fix indentation.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170005 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ae3b652f5cc19d83b6466d4fa70a7d1c7fb6d06c 12-Dec-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to increase the function size.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170004 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
db65ff39faee4cab994d3b7ece1a31dfd9343818 12-Dec-2012 Nadav Rotem <nrotem@apple.com> Fix the ascii drawing that was ruined when I split the H and CPP

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169955 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6c645a35ccc658b20506d3137890d9003edd3527 12-Dec-2012 Nadav Rotem <nrotem@apple.com> fix a typo.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169953 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5da804150d418b8b4956b84013f8f67df553c543 12-Dec-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: When -Os is used, vectorize only loops that dont require a tail loop. There is no testcase because I dont know of a way to initialize the loop vectorizer pass without adding an additional hidden flag.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169950 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
655d2c5354fcd44c329d99428c7d9196bc78dbad 11-Dec-2012 Nadav Rotem <nrotem@apple.com> PR14574. Fix a bug in the code that calculates the mask the converted PHIs in if-conversion.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169916 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5e9efa10fc7eea582bb1ad216364fb2c467e4477 11-Dec-2012 Nadav Rotem <nrotem@apple.com> Loop Vectorize: optimize the vectorization of trunc(induction_var). The truncation is now done on scalars.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169904 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
cfb6285fdb27fba97ddc3ebf6485110d9a157ac5 11-Dec-2012 Nadav Rotem <nrotem@apple.com> Fix PR14565. Don't if-convert loops that have switch statements in them.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169813 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d1d92bf953d51560e979337cadcc9d7e62fdd79e 10-Dec-2012 Nadav Rotem <nrotem@apple.com> Split the LoopVectorizer into H and CPP.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169771 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f0d19bd1291ee1d2ffee4bbe0aef12b814aff789 10-Dec-2012 Nadav Rotem <nrotem@apple.com> Add support for reverse induction variables. For example:

while (i--)
sum+=A[i];



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169752 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
880166684e5af0f5b4bfe26870b9f7813e537354 09-Dec-2012 Paul Redmond <paul.redmond@intel.com> LoopVectorize: support vectorizing intrinsic calls

- added function to VectorTargetTransformInfo to query cost of intrinsics
- vectorize trivially vectorizable intrinsic calls such as sin, cos, log, etc.

Reviewed by: Nadav


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169711 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b48fdbc811b0bf7dec0780b4d03169d8ee835d03 09-Dec-2012 Paul Redmond <paul.redmond@intel.com> test commit.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169709 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
46c5f79789947b75ff3b0cc107f133fb0c5ffb5e 05-Dec-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Increase the number of pointers that can be tested at runtime. If we cant prove statically that the pointers are disjoint then we add the runtime check.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169334 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
dc5eff5b98aba0fe0e8104dfa7429c9a759684c2 04-Dec-2012 Nadav Rotem <nrotem@apple.com> Enable if-conversion during vectorization.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169331 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e570dee4b03cca54bbf27a7f7a3299c5cdc3d087 04-Dec-2012 Nadav Rotem <nrotem@apple.com> Fix a bug in vectorization of if-converted reduction variables. If the
reduction variable is not used outside the loop then we ran into an
endless loop. This change checks if we found the original PHI.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169324 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f6088d126e9110180f7db231bb58da9230a09537 04-Dec-2012 Nadav Rotem <nrotem@apple.com> Add support for reduction variables when IF-conversion is enabled.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169288 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e6f2df961065e3b723ef0fc78441f07c5577acd2 04-Dec-2012 Nadav Rotem <nrotem@apple.com> Give scalar if-converted blocks half the score because they are not always executed due to CF.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169223 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
319d594e22c3db55114f233ca398f9760e3f6ed9 04-Dec-2012 Nadav Rotem <nrotem@apple.com> Add the last part that is needed for vectorization of if-converted code.
Added the code that actually performs the if-conversion during vectorization.

We can now vectorize this code:

for (int i=0; i<n; ++i) {
unsigned k = 0;

if (a[i] > b[i]) <------ IF inside the loop.
k = k * 5 + 3;

a[i] = k; <---- K is a phi node that becomes vector-select.
}



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169217 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
915584ed136c29b8fd9304268967f3f359befbb0 04-Dec-2012 NAKAMURA Takumi <geek4civic@gmail.com> LoopVectorize.cpp: Suppress a warning. [-Wunused-variable]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169195 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
36422d887672dc9a21d9add99c675bdc9e4807dd 04-Dec-2012 NAKAMURA Takumi <geek4civic@gmail.com> Fix whitespace.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169194 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b8f842dce47e745b37505f156854755d8ad4c929 03-Dec-2012 Nadav Rotem <nrotem@apple.com> minor renaming, documentation and cleanups.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169175 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
fa72ee729a989ff340672034fd77832c1fd76326 03-Dec-2012 Nadav Rotem <nrotem@apple.com> IF-conversion: teach the cost-model how to grade if-converted loops.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169171 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
56f1ce22dc80cde198afb3ed153c109b951217fa 03-Dec-2012 Nadav Rotem <nrotem@apple.com> Now that we have a basic if-conversion infrastructure we can rename the
"single basic block loop vectorizer" to "innermost loop vectorizer".



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169158 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
dd8b1015c8e3dd8f7f9d6d5d63804d2e5ab89c20 03-Dec-2012 Nadav Rotem <nrotem@apple.com> Add initial support for IF-conversion. This patch implements the first 1/3,
which is the legality of the if-conversion transformation. The next step is to
implement the cost-model for the if-converted code as well as the
vectorization itself.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169152 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d04a8d4b33ff316ca4cf961e06c9e312eff8e64f 03-Dec-2012 Chandler Carruth <chandlerc@gmail.com> Use the new script to sort the includes of every file under lib.

Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.

Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169131 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d6964741f52d2553517209319fa214afe8b3812b 30-Nov-2012 Nadav Rotem <nrotem@apple.com> minor cleanups

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169048 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f735a7f88dcb87cca4622ce45effa8134759b7d8 30-Nov-2012 Nadav Rotem <nrotem@apple.com> Remove the use of LPPassManager. We can remove LPM because we dont need to run any additional loop passes on the new vector loop.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169016 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
dddaad624e388529f1f933939000d6dbc07c107d 29-Nov-2012 Nadav Rotem <nrotem@apple.com> When broadcasting invariant scalars into vectors, place the broadcast code in the preheader.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168927 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
22689b6034dbde1c08e39b162fd59e1fd5d35b96 26-Nov-2012 Nadav Rotem <nrotem@apple.com> Move the code that uses SCEVs prior to creating the new loops.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168601 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6bfc3481bd8995906af4c15131feeae665a197c6 25-Nov-2012 Nadav Rotem <nrotem@apple.com> Move the max vector width to a constant parameter. No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168570 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
00e8074ee24449e5d0e50b626290b499b8bd78b1 25-Nov-2012 Nadav Rotem <nrotem@apple.com> Fix the document style.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168569 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8c6b73666bdd08f15b31c00bd2fd663b632a1d65 25-Nov-2012 Nadav Rotem <nrotem@apple.com> Refactor the ptr runtime check generation code. No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168568 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a8e2b2b68fed9883bd41335f57f1193ffcc22ed2 25-Nov-2012 Nadav Rotem <nrotem@apple.com> Rename method. No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168560 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d7e0b7cdc59d04bec6cf5273cde3d2a16b13800e 25-Nov-2012 Nadav Rotem <nrotem@apple.com> The induction-pointer work is inspired by a research paper. This commit adds a reference.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168559 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
0af63ac245eeb0cce206ed4dbc9abdc0d86742cb 25-Nov-2012 Nadav Rotem <nrotem@apple.com> Add support for pointer induction variables even when there is no integer induction variable.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168558 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
9a6823516ffd6ed1787d923459c80a6fa4833914 17-Nov-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Add initial support for pointer induction variables (for example: *dst++ = *src++).
At the moment we still require to have an integer induction variable (for example: i++).



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168231 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
77d6ae1b01f86cc82593c8f67118b7c2d6fd17c2 16-Nov-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Division reductions generate incorrect code. Remove the part of the code that deals with divs.
Thanks to Paul Redmond for catching this while reviewing the code.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168142 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e123fd9c5956962d414693ee8ad83498413af6ce 14-Nov-2012 Nadav Rotem <nrotem@apple.com> use the getSplat API. Patch by Paul Redmond.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167892 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
9005b8d27d68caeb893d74aa75cd679972d86cef 11-Nov-2012 Nadav Rotem <nrotem@apple.com> Fix a comment typo and add comments.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167684 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b1bf1eeede72b8c93505dd80fdf21aed0e205c7d 09-Nov-2012 Nadav Rotem <nrotem@apple.com> Add support for memory runtime check. When we can, we calculate array bounds.
If the arrays are found to be disjoint then we run the vectorized version of
the loop. If they are not, we run the scalar code.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167608 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6e48f0307758096d06d0e87875294c76df81dec1 02-Nov-2012 Chandler Carruth <chandlerc@gmail.com> Fix sign compare warning. Patch by Mahesha HS.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167282 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e57b2cbce652d45ed5516e52ad82991bfa03cfd7 31-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Preserve NSW, NUW and IsExact flags.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167174 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4c1b4b1fe794437cbb245b11650d9e4001c9605e 31-Oct-2012 Nadav Rotem <nrotem@apple.com> Put the threshold magic number in a variable.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167134 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
2a5401cc75acf38092a7224cadaf895281d41b29 31-Oct-2012 Nadav Rotem <nrotem@apple.com> Remove enum values since they are not used anymore.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167131 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5b2c4dc5f8f70352f029d595797089821eb39b3c 31-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Do not vectorize loops with tiny constant trip counts.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167101 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
462d1ca42831df53a3c6435d247776f0b85bd594 31-Oct-2012 Nadav Rotem <nrotem@apple.com> Add support for loops that don't start with Zero.
This is important for loops in the LAPACK test-suite.
These loops start at 1 because they are auto-converted from fortran.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167084 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a368b89f2842530d07c0ac8e3b533882e165f197 30-Oct-2012 Nadav Rotem <nrotem@apple.com> Add documentation.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167055 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e709f5b600fd630c4f58b5dba14c8069a03093ea 30-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Add support for write-only loops when the write destination is a single pointer.
Speedup SciMark by 1%



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167035 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
cc03331caaeee00b1238654feda8c5a517e48c3a 30-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Fix a bug in the initialization of reduction variables. AND needs to start at all-one
while XOR, and OR need to start at zero.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167032 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
803ea1c40aa059adf5b1baef0bf4510483565d07 30-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: change debug prints: Print the module identifier when deciding to vectorize. When deciding not to vectorize do not print the called function name because it can be null.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166989 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
369ff7b740c154dc4357c6d063f4957498a1e7b9 29-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Update and preserve the dominator tree info.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166970 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f065a8467785015336432e3e6e584798d8b48d8e 27-Oct-2012 Nadav Rotem <nrotem@apple.com> 1. Fix a bug in getTypeConversion. When a *simple* type is split, we need to return the type of the split result.
2. Change the maximum vectorization width from 4 to 8.
3. A test for both.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166864 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a5a3a61c5fdcee972791d4e08441ba6edf131b88 27-Oct-2012 Nadav Rotem <nrotem@apple.com> Refactor the VectorTargetTransformInfo interface.

Add getCostXXX calls for different families of opcodes, such as casts, arithmetic, cmp, etc.

Port the LoopVectorizer to the new API.

The LoopVectorizer now finds instructions which will remain uniform after vectorization. It uses this information when calculating the cost of these instructions.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166836 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
3ef9dfa6858e25015c3e36b2f1a0ba5ebdea80d2 25-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Teach the cost model to query scalar costs as scalar types and not vectors of 1.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166715 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
8dbac7b529cfb73bcd0ceef514e5c1d247cf3baa 25-Oct-2012 Nadav Rotem <nrotem@apple.com> Add support for additional reduction variables: AND, OR, XOR.

Patch by Paul Redmond <paul.redmond@intel.com>.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166649 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
2652c50f74bc4a874c6a2e4b34ff2d52d479183f 25-Oct-2012 Nadav Rotem <nrotem@apple.com> Implement a basic cost model for vector and scalar instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166642 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
7c6de1d6e53c47f2943874ab9d63dddfd19b503d 24-Oct-2012 Nadav Rotem <nrotem@apple.com> whitespace

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166622 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
50bec6f8c494957b00dd225ddf580d3e0b97b871 24-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorizer: Add a basic cost model which uses the VTTI interface.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166620 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b8bce928f4ffdf50eff69334f3e25b27848536b6 24-Oct-2012 Micah Villmow <villmow@gmail.com> Back out r166591, not sure why this made it through since I cancelled the command. Bleh, sorry about this!


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166596 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
2f87640b86315beab8a5671cc23f524e59c58bd3 24-Oct-2012 Micah Villmow <villmow@gmail.com> Delete a directory that wasn't supposed to be checked in yet.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166591 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6457001f31713ff26a707ddef616341052b1b296 23-Oct-2012 Nadav Rotem <nrotem@apple.com> Use the AliasAnalysis isIdentifiedObj because it also understands mallocs and c++ news.

PR14158.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166491 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
782090aa02484d7d1d335e3a7e209f1132aaa1a7 22-Oct-2012 Nadav Rotem <nrotem@apple.com> Don't crash if the load/store pointer is not a GEP.

Fix by Shivarama Rao <Shivarama.Rao@amd.com>



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166427 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5f7d81022398f332b222552f5d980c4e3f1c542c 22-Oct-2012 Nadav Rotem <nrotem@apple.com> Rename a variable.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166410 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
565048e78a152aa69b66422037b67c0f627bbfec 22-Oct-2012 Nadav Rotem <nrotem@apple.com> Vectorizer: optimize the generation of selects. If the condition is uniform, generate a scalar-cond select (i1 as selector).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166409 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e0fa403e23ab5610471c2dbe5c032327ee1ed091 22-Oct-2012 Nadav Rotem <nrotem@apple.com> Update the loop vectorizer docs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166408 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5bb5a75b1e47443ba9b07a53e6e967d29108bab3 21-Oct-2012 Anders Carlsson <andersca@mac.com> Avoid an extra hash lookup when inserting a value into the widen map.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166395 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ffe5008fd3013451fd3cfe8dd878c9a1d4abad21 21-Oct-2012 Jakub Staszak <kubastaszak@gmail.com> Simplify code. No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166393 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b75209cb66ab5d7a155af3cffe0e552e69820e2c 21-Oct-2012 Jakub Staszak <kubastaszak@gmail.com> Simplify code. No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166392 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
bb950854acbb5966875763eaae7ab58e48e4f5a9 21-Oct-2012 Nadav Rotem <nrotem@apple.com> Fix a bug in the vectorization of wide load/store operations.
We used a SCEV to detect that A[X] is consecutive. We assumed that X was
the induction variable. But X can be any expression that uses the induction
for example: X = i + 2;



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166388 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c84787262942a400a8de6539ee79f4de72d3f872 21-Oct-2012 Nadav Rotem <nrotem@apple.com> Add support for reduction variables that do not start at zero.
This is important for nested-loop reductions such as :

In the innermost loop, the induction variable does not start with zero:

for (i = 0 .. n)
for (j = 0 .. m)
sum += ...



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166387 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
f01cad69c1ab1b025c524d1ac16060b2cc0f4668 21-Oct-2012 Nadav Rotem <nrotem@apple.com> Document change. Describe the pass and some papers that inspired the design of the pass.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166386 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5a418ba5f5a6498a25d5eacb0f876d9f358c977b 21-Oct-2012 Nadav Rotem <nrotem@apple.com> Vectorizer: fix a bug in the classification of induction/reduction phis.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166384 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ccaccfa8bf070b169847109513610cf41abfd1f3 20-Oct-2012 Nadav Rotem <nrotem@apple.com> Fix an infinite loop in the loop-vectorizer.

PR14134.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166379 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
bf8772ed2cc89a495e2692919331d7a03e76d791 20-Oct-2012 Nadav Rotem <nrotem@apple.com> Vectorize: teach cavVectorizeMemory to distinguish between A[i]+=x and A[B[i]]+=x.
If the pointer is consecutive then it is safe to read and write. If the pointer is non-loop-consecutive then
it is unsafe to vectorize it because we may hit an ordering issue.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166371 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
71a148223907504c78f90f835131d5e8921011ad 20-Oct-2012 Nadav Rotem <nrotem@apple.com> Fix a typo

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166367 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
e6748f91eaa69b91cc5c4081a45e013a251c2726 20-Oct-2012 Nadav Rotem <nrotem@apple.com> Vectorizer: refactor the memory checks to a new function. No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166366 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
4ade92b6145a0d298b3c5ae1af79aca2598a327d 20-Oct-2012 Nadav Rotem <nrotem@apple.com> LoopVectorize: Keep the IRBuilder on the stack.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166354 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
5dbe64e2bc2e4b96654703e85f909536df7ddb84 20-Oct-2012 Nadav Rotem <nrotem@apple.com> Vectorizer: Add support for loop reductions.

For example:

for (i=0; i<n; i++)
sum += A[i] + B[i] + i;



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166351 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
17f68c52d2f3b6b2472b07492d76520df230afcd 19-Oct-2012 Benjamin Kramer <benny.kra@googlemail.com> LoopVectorize: Keep the IRBuilder on the stack.

No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166274 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
89e7b356f270e29c2e9e18c6bbd30e5925585f06 19-Oct-2012 Nadav Rotem <nrotem@apple.com> vectorizer: Add support for reading and writing from the same memory location.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166255 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
ebd3f27c7e2c6c3a1b76786da0d0205a5fdb1ef5 19-Oct-2012 Nadav Rotem <nrotem@apple.com> cleanup the comment.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166247 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6220fb16c1d08534e770785f47717ee80baac517 18-Oct-2012 Nadav Rotem <nrotem@apple.com> fix a naming typo

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166232 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
b943d9d497175ce44cca7b7bb14b83a86dba7d76 18-Oct-2012 Nadav Rotem <nrotem@apple.com> Avoid reconstructing the pointer set when searching for duplicated read/write pointers.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166205 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
bef36ac2a47dd647af332b8594031df6fe2522c5 18-Oct-2012 Nadav Rotem <nrotem@apple.com> When looking for a vector representation of a scalar, do a single lookup. Also, cache the result of the broadcast instruction.
No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166191 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
3141d2db037bb3c31a1efd50118df45c1aebbb51 18-Oct-2012 Nadav Rotem <nrotem@apple.com> remove unused variable to fix a warning.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166170 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d22d5f9122de2c8527f1d8890ce2802a89e0db0c 18-Oct-2012 Nadav Rotem <nrotem@apple.com> Remove the use of dominators and AA.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166167 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
1953ace81d3121808967a7fa47ad6d631499933d 18-Oct-2012 Nadav Rotem <nrotem@apple.com> Vectorizer: Add support for loops with an unknown count. For example:

for (i=0; i<n; i++){
a[i] = b[i+1] + c[i+3];
}



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166165 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
6dc0050f6d42520ad92e6786551f27ee3f365ecd 18-Oct-2012 NAKAMURA Takumi <geek4civic@gmail.com> LoopVectorize.cpp: Fix a warning. [-Wunused-variable]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166153 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
c2358eb361ee3304e553c0d283c5c3a44f28950f 18-Oct-2012 Jakub Staszak <kubastaszak@gmail.com> Remove redundant SetInsertPoint call.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166138 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
a5401d67304b9814fc13521563a9634f37f2047b 17-Oct-2012 Roman Divacky <rdivacky@freebsd.org> Fix some typos and wrong indenting.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166128 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
d15c0c7ac118cb23241b002e7206221283e36e2d 17-Oct-2012 Nadav Rotem <nrotem@apple.com> Add a loop vectorizer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166112 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp