| cd81d94322a39503e4a3e87b6ee03d4fcb3465fb |
|
21-Jul-2014 |
Stephen Hines <srhines@google.com> |
Update LLVM for rebase to r212749.
Includes a cherry-pick of:
r212948 - fixes a small issue with atomic calls
Change-Id: Ib97bd980b59f18142a69506400911a6009d9df18
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| dce4a407a24b04eebc6a376f8e62b41aaa7b071f |
|
29-May-2014 |
Stephen Hines <srhines@google.com> |
Update LLVM for 3.5 rebase (r209712).
Change-Id: I149556c940fb7dc92d075273c87ff584f400941f
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 36b56886974eae4f9c5ebc96befd3e7bfe5de338 |
|
24-Apr-2014 |
Stephen Hines <srhines@google.com> |
Update to LLVM 3.5a.
Change-Id: Ifadecab779f128e62e430c2b4f6ddd84953ed617
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 43f41cc550e74346f927dc45cb82c57ddcf07117 |
|
20-Nov-2013 |
Bill Wendling <isanbard@gmail.com> |
Merging r195118:
------------------------------------------------------------------------
r195118 | chandlerc | 2013-11-19 01:03:18 -0800 (Tue, 19 Nov 2013) | 22 lines
Fix an issue where SROA computed different results based on the relative
order of slices of the alloca which have exactly the same size and other
properties. This was found by a perniciously unstable sort
implementation used to flush out buggy uses of the algorithm.
The fundamental idea is that findCommonType should return the best
common type it can find across all of the slices in the range. There
were two bugs here previously:
1) We would accept an integer type smaller than a byte-width multiple,
and if there were different bit-width integer types, we would accept
the first one. This caused an actual failure in the testcase updated
here when the sort order changed.
2) If we found a bad combination of types or a non-load, non-store use
before an integer typed load or store we would bail, but if we found
the integere typed load or store, we would use it. The correct
behavior is to always use an integer typed operation which covers the
partition if one exists.
While a clever debugging sort algorithm found problem #1 in our existing
test cases, I have no useful test case ideas for #2. I spotted in by
inspection when looking at this code.
------------------------------------------------------------------------
git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@195217 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 7f80b75b963781d81b772ae2f3a35dc74e1b6457 |
|
22-Sep-2013 |
Benjamin Kramer <benny.kra@googlemail.com> |
Drop spurious handle in comment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191172 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 1ce1525ed453aea78d17f28ec3c353d0cde5341f |
|
21-Sep-2013 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Handle casts involving vectors of pointers and integer scalars.
SROA wants to convert any types of equivalent widths but it's not possible to
convert vectors of pointers to an integer scalar with a single cast. As a
workaround we add a bitcast to the corresponding int ptr type first. This type
of cast used to be an edge case but has become common with SLP vectorization.
Fixes PR17271.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191143 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 6c1fa7caaefc88a5a867add402d90115823bd0eb |
|
14-Aug-2013 |
Nick Lewycky <nicholas@mxc.ca> |
Revert r187191, which broke opt -mem2reg on the testcases included in PR16867.
However, opt -O2 doesn't run mem2reg directly so nobody noticed until r188146
when SROA started sending more things directly down the PromoteMemToReg path.
In order to revert r187191, I also revert dependent revisions r187296, r187322
and r188146. Fixes PR16867. Does not add the testcases from that PR, but both
of them should get added for both mem2reg and sroa when this revert gets
unreverted.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188327 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 5b854f1ea55601790d9191c9720e77da35095340 |
|
11-Aug-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Re-instate r187323 which fast-tracks promotable allocas as soon as the
SROA-based analysis has enough information. This should work now that
both mem2reg *and* the SSAUpdater-based AllocaPromoter have been updated
to be able to promote the types of allocas that the SROA analysis
detects.
I've included tests for the AllocaPromoter that were only possible to
write once we fast-tracked promotable allocas without rewriting them.
This includes a test both for r187347 and r188145.
Original commit log for r187323:
"""
Now that mem2reg understands how to cope with a slightly wider set of uses of
an alloca, we can pre-compute promotability while analyzing an alloca for
splitting in SROA. That lets us short-circuit the common case of a bunch of
trivially promotable allocas. This cuts 20% to 30% off the run time of SROA for
typical frontend-generated IR sequneces I'm seeing. It gets the new SROA to
within 20% of ScalarRepl for such code. My current benchmark for these numbers
is PR15412, but it fits the general pattern of IR emitted by Clang so it should
be widely applicable.
"""
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188146 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 37508bb842d9beedd75139a589c6f538f90efbaa |
|
11-Aug-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Finish fixing the SSAUpdater-based AllocaPromoter strategy in SROA to cope with
the more general set of patterns that are now handled by mem2reg and that we
can detect quickly while doing SROA's initial analysis. Notably, this allows it
to promote through no-op bitcast and GEP sequences. A core part of the
SSAUpdater approach is the ability to test whether a particular instruction is
part of the set being promoted. Testing this becomes significantly more complex
in the world where the operand to every load and store isn't the alloca itself.
I ended up using the approach of walking up the def-chain until we find the
alloca. I benchmarked this against keeping a set of pointer operands and
keeping a set of the loads and stores we care about, and this one seemed faster
although the difference was very small.
No test case yet because currently the rewriting always "fixes" the inputs to
not require this. The next patch which re-enables early promotion of easy cases
in SROA will include a test case that specifically exercises this aspect of the
alloca promoter.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188145 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 3c7a446059133de68c912242cb3b0cc934b8e6b1 |
|
11-Aug-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Reformat some bits of AllocaPromoter and simplify the name and type of
our visiting datastructures in the AllocaPromoter/SSAUpdater path of
SROA. Also shift the order if clears around to be more consistent.
No functionality changed here, this is just a cleanup.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188144 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| e1361ec325834dc05b94c3e2764fb32173b363ac |
|
29-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Teach the AllocaPromoter which is wrapped around the SSAUpdater
infrastructure to do promotion without a domtree the same smarts about
looking through GEPs, bitcasts, etc., that I just taught mem2reg about.
This way, if SROA chooses to promote an alloca which still has some
noisy instructions this code can cope with them.
I've not used as principled of an approach here for two reasons:
1) This code doesn't really need it as we were already set up to zip
through the instructions used by the alloca.
2) I view the code here as more of a hack, and hopefully a temporary one.
The SSAUpdater path in SROA is a real sore point for me. It doesn't make
a lot of architectural sense for many reasons:
- We're likely to end up needing the domtree anyways in a subsequent
pass, so why not compute it earlier and use it.
- In the future we'll likely end up needing the domtree for parts of the
inliner itself.
- If we need to we could teach the inliner to preserve the domtree. Part
of the re-work of the pass manager will allow this to be very powerful
even in large SCCs with many functions.
- Ultimately, computing a domtree has gotten significantly faster since
the original SSAUpdater-using code went into ScalarRepl. We no longer
use domfrontiers, and much of domtree is lazily done based on queries
rather than eagerly.
- At this point keeping the SSAUpdater-based promotion saves a total of
0.7% on a build of the 'opt' tool for me. That's not a lot of
performance given the complexity!
So I'm leaving this a bit ugly in the hope that eventually we just
remove all of this nonsense.
I can't even readily test this because this code isn't reachable except
through SROA. When I re-instate the patch that fast-tracks allocas
already suitable for promotion, I'll add a testcase there that failed
before this change. Before that, SROA will fix any test case I give it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187347 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 65f12f1d0542163e4e40b9e0ffbfcb3fd1df8f3a |
|
28-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Temporarily revert r187323 until I update SSAUpdater to match mem2reg.
I forgot that we had two totally independent things here. :: sigh ::
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187327 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| cea60aff34ada256a77f5760863218a976786f45 |
|
28-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Now that mem2reg understands how to cope with a slightly wider set of
uses of an alloca, we can pre-compute promotability while analyzing an
alloca for splitting in SROA. That lets us short-circuit the common case
of a bunch of trivially promotable allocas. This cuts 20% to 30% off the
run time of SROA for typical frontend-generated IR sequneces I'm seeing.
It gets the new SROA to within 20% of ScalarRepl for such code. My
current benchmark for these numbers is PR15412, but it fits the general
pattern of IR emitted by Clang so it should be widely applicable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187323 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 6c3a95dab561ef9747876d14014e965f8688f55b |
|
28-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Thread DataLayout through the callers and into mem2reg. This will be
useful in a subsequent patch, but causes an unfortunate amount of noise,
so I pulled it out into a separate patch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187322 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 89934cbd344b6839f4d298b281998a9f48355bf1 |
|
27-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Don't use all the #ifdefs to hide the stats counters and instead rely on
their being optimized out in debug mode. Realistically, this just isn't
going to be the slow part anyways. This also fixes unused variable
warnings that are breaking LLD build bots. =/ I didn't see these at
first, and kept losing track of the fact that they were broken.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187297 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| b7f27824fb63210a67040affd43a82890b6784fb |
|
24-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a problem I introduced in r187029 where we would over-eagerly
schedule an alloca for another iteration in SROA. This only showed up
with a mixture of promotable and unpromotable selects and phis. Added
a test case for this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187031 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 9b3b2862478e5baec967d479a2d00731758e1bb8 |
|
24-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Fix PR16687 where we were incorrectly promoting an alloca that had
pending speculation for a phi node. The problem here is that we were
using growth of the specluation set as an indicator of whether
speculation would occur, and if the phi node is already in the set we
don't see it grow. This is a symptom of the fact that this signal is
a total hack.
Unfortunately, I couldn't really come up with a non-hacky way of
signaling that promotion remains valid *after* speculation occurs, such
that we only speculate when all else looks good for promotion. In the
end, I went with at least a much more explicit approach of doing the
work of queuing inside the phi and select processing and setting
a preposterously named flag to convey that we're in the special state of
requiring speculating before promotion.
Thanks to Richard Trieu and Nick Lewycky for the excellent work reducing
a testcase for this from a pretty giant, nasty assert in a big
application. =] The testcase was excellent.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187029 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 1579a0f8a690455b76cd87e7883196ffe58465aa |
|
23-Jul-2013 |
Nick Lewycky <nicholas@mxc.ca> |
Remove extraneous null statement. No functionality change!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186893 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 85f6cbd1a5dc0071b3b4a7387e66479bbdfb3d13 |
|
23-Jul-2013 |
Jakub Staszak <kubastaszak@gmail.com> |
OldPtr is llvm::Instruction. Remove unneeded cast<>.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186880 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 916cde6416f0ba10101da8f59d6b0a9ca5b654a0 |
|
20-Jul-2013 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Microoptimization: Remove dead entries first, then sort.
While there replace an explicit struct with std::mem_fun.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186761 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 47042bcc266285676f8ff284e5d46a2c196c367b |
|
19-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Cleanup the stats counters for the new implementation. These actually
count the right things and have the right names.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186667 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| fbf2a026224b80d9c1513060082195dc10a99d75 |
|
19-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Fix another assert failure very similar to PR16651's test case. This
test case came from Benjamin and found the parallel bug in the vector
promotion code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186666 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| c09228dba3be474d9835cad19adc4419224872f3 |
|
19-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Try to move to a more reasonable set of naming conventions given the new
implementation of the SROA algorithm. We were using the term 'partition'
in many places that no longer ever represented an actual partition, but
rather just an arbitrary slice of an alloca.
No functionality change intended here. Mostly just renaming of types,
functions, variables, and rewording of comments. Several comments were
rewritten to make a lot more sense in the new structure of things.
The stats are still weird and not reflective of how this really works.
I'll fix those up in a separate patch as it is a touch more semantic of
a change...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186659 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| df5ed3f642ff5dfd19634c80c4106ace247dc1bf |
|
19-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
A long overdue cleanup in SROA to use 'DL' instead of 'TD' for the
DataLayout variables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186656 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 8f0a1cecc582145903a2bc0d0e53f06f874fc831 |
|
19-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Fix PR16651, an assert introduced in my recent re-work of the innards of
SROA.
The crux of the issue is that now we track uses of a partition of the
alloca in two places: the iterators over the partitioning uses and the
previously collected split uses vector. We weren't accounting for the
fact that the split uses might invalidate integer widening in ways other
than due to their width (in this case due to being volatile).
Further reduced testcase added to the tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186655 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| f7c45ce3f59d993b1b7f8edbe07304328ac71b12 |
|
18-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Reapply r186316 with a fix for one bug where the code could walk off the
end of a vector. This was found with ASan. I've had one other report of
a crasher, but thus far been unable to reproduce the crash. It may well
be fixed with this version, and if not I'd like to get more information
from the build bots about what is happening.
See r186316 for the full commit log for the new implementation of the
SROA algorithm.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186565 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ebf72b330164dca40566bbce89ce832c705c5bf3 |
|
15-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Revert r186316 while I track down an ASan failure and an assert from
a bot.
This reverts the commit which introduced a new implementation of the
fancy SROA pass designed to reduce its overhead. I'll skip the huge
commit log here, refer to r186316 if you're looking for how this all
works and why it works that way.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186332 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ea2e90df1519e5de2f30cd5f63a38d82bf4f0653 |
|
15-Jul-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Reimplement SROA yet again. Same fundamental principle, but a totally
different core implementation strategy.
Previously, SROA would build a relatively elaborate partitioning of an
alloca, associate uses with each partition, and then rewrite the uses of
each partition in an attempt to break apart the alloca into chunks that
could be promoted. This was very wasteful in terms of memory and compile
time because regardless of how complex the alloca or how much we're able
to do in breaking it up, all of the datastructure work to analyze the
partitioning was done up front.
The new implementation attempts to form partitions of the alloca lazily
and on the fly, rewriting the uses that make up that partition as it
goes. This has a few significant effects:
1) Much simpler data structures are used throughout.
2) No more double walk of the recursive use graph of the alloca, only
walk it once.
3) No more complex algorithms for associating a particular use with
a particular partition.
4) PHI and Select speculation is simplified and happens lazily.
5) More precise information is available about a specific use of the
alloca, removing the need for some side datastructures.
Ultimately, I think this is a much better implementation. It removes
about 300 lines of code, but arguably removes more like 500 considering
that some code grew in the process of being factored apart and cleaned
up for this all to work.
I've re-used as much of the old implementation as possible, which
includes the lion's share of code in the form of the rewriting logic.
The interesting new logic centers around how the uses of a partition are
sorted, and split into actual partitions.
Each instruction using a pointer derived from the alloca gets
a 'Partition' entry. This name is totally wrong, but I'll do a rename in
a follow-up commit as there is already enough churn here. The entry
describes the offset range accessed and the nature of the access. Once
we have all of these entries we sort them in a very specific way:
increasing order of begin offset, followed by whether they are
splittable uses (memcpy, etc), followed by the end offset or whatever.
Sorting by splittability is important as it simplifies the collection of
uses into a partition.
Once we have these uses sorted, we walk from the beginning to the end
building up a range of uses that form a partition of the alloca.
Overlapping unsplittable uses are merged into a single partition while
splittable uses are broken apart and carried from one partition to the
next. A partition is also introduced to bridge splittable uses between
the unsplittable regions when necessary.
I've looked at the performance PRs fairly closely. PR15471 no longer
will even load (the module is invalid). Not sure what is up there.
PR15412 improves by between 5% and 10%, however it is nearly impossible
to know what is holding it up as SROA (the entire pass) takes less time
than reading the IR for that test case. The analysis takes the same time
as running mem2reg on the final allocas. I suspect (without much
evidence) that the new implementation will scale much better however,
and it is just the small nature of the test cases that makes the changes
small and noisy. Either way, it is still simpler and cleaner I think.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186316 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 365ef0b197d7c841f8e501da64296df65be4ca23 |
|
03-Jul-2013 |
Craig Topper <craig.topper@gmail.com> |
Use SmallVectorImpl::iterator/const_iterator instead of SmallVector to avoid specifying the vector size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185540 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| a1fe2948ed4039e68d1784494c3b23a4ce4126b4 |
|
25-Jun-2013 |
Bob Wilson <bob.wilson@apple.com> |
Fix SROA to avoid unnecessary scalar conversions for 1-element vectors.
When a 1-element vector alloca is promoted, a store instruction can often be
rewritten without converting the value to a scalar and using an insertelement
instruction to stuff it into the new alloca. This patch just adds a check
to skip that conversion when it is unnecessary. This turns out to be really
important for some ARM Neon operations where <1 x i64> is used to get around
the fact that i64 is not a legal type.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184870 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| fee6969463d455201a5179620ccd8c3171effaa6 |
|
01-May-2013 |
Nadav Rotem <nrotem@apple.com> |
SROA: Generate selects instead of shuffles when blending values because this is the cannonical form.
Shuffles are more difficult to lower and we usually don't touch them, while we do optimize selects more often.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180875 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| d81a0dee5b7df712c584e2a3e556ff2f0677e9df |
|
21-Apr-2013 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Don't crash on a select with two identical operands.
This is an edge case that can happen if we modify a chain of multiple selects.
Update all operands in that case and remove the assert. PR15805.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179982 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 05c7e7f99dfa1ff353c3d13e71143b61baa583f6 |
|
07-Apr-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Fix PR15674 (and PR15603): a SROA think-o.
The fix for PR14972 in r177055 introduced a real think-o in the *store*
side, likely because I was much more focused on the load side. While we
can arbitrarily widen (or narrow) a loaded value, we can't arbitrarily
widen a value to be stored, as that changes the width of memory access!
Lock down the code path in the store rewriting which would do this to
only handle the intended circumstance.
All of the existing tests continue to pass, and I've added a test from
the PR.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178974 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 6f7becfe23f38e8c28e9422d883263974058aeed |
|
24-Mar-2013 |
Jakub Staszak <kubastaszak@gmail.com> |
Minor cleanups. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177837 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| d647ec52396ed0f1886537114ab587c71fc43565 |
|
21-Mar-2013 |
Chandler Carruth <chandlerc@gmail.com> |
[SROA] Prefix names using a custom IRBuilder inserter.
The key part of this is ensuring that name prefixes remain in a Twine
form until we get to a point where we can nuke them under NDEBUG. This
is tricky using the old APIs as they played fast and loose with Twine,
which is prone to serious error. The inserter is much cleaner as it is
actually in the call stack leading to the setName call, and so has
a good opportunity to prepend the prefix.
This matters more than you might imagine because most runs over an
alloca find a single partition, and rewrite 3 or 4 instructions
referring to it. As a consequence doing this lazily and exclusively with
Twine allows the optimizer to delete more of it and shaves another 2% to
3% off of the release build's SROA run time for PR15412. I also think
the APIs are cleaner, and the use of Twine is more reliable, so
I consider it a win-win despite the churn required to reach this state.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177631 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| fd060a94a46db75c45b2a1789a8bc597a03e6ac1 |
|
20-Mar-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a silly search-and-replace goof with r177495 that only broke
non-release builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177498 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 05c6d0b16f8c442a29b03b312f9fb6910955a8ec |
|
20-Mar-2013 |
Chandler Carruth <chandlerc@gmail.com> |
[SROA] Don't preserve the IR names in release builds.
This is espcially important because the new SROA pass goes to great
lengths to provide helpful names for debugging, and as a consequence
they can become very slow to render.
Good for between 5% and 15% of the SROA runtime on some slow test cases
such as the one in PR15412.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177495 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 30ee9c2093897a0edfe3bd23e6c754c4d88cbd88 |
|
20-Mar-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Move the endif to the correct line so we don't have warnings about
unused statistics variables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177494 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| f2b649da0f433511e9e0d602ee59e9f687dd846c |
|
20-Mar-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Introduce some new statistics to help track the exact behavior of the
new SROA pass.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177493 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 5e8da1773c8a16a9e3f08df444420f04d71ba673 |
|
18-Mar-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Mark internal classes as POD-like to get better behavior out of
SmallVector and DenseMap.
This speeds up SROA by 25% on PR15412.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177259 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 41b55f5556d1332934cefa7c14862313eb87fa29 |
|
14-Mar-2013 |
Chandler Carruth <chandlerc@gmail.com> |
PR14972: SROA vs. GVN exposed a really bad bug in SROA.
The fundamental problem is that SROA didn't allow for overly wide loads
where the bits past the end of the alloca were masked away and the load
was sufficiently aligned to ensure there is no risk of page fault, or
other trapping behavior. With such widened loads, SROA would delete the
load entirely rather than clamping it to the size of the alloca in order
to allow mem2reg to fire. This was exposed by a test case that neatly
arranged for GVN to run first, widening certain loads, followed by an
inline step, and then SROA which miscompiles the code. However, I see no
reason why this hasn't been plaguing us in other contexts. It seems
deeply broken.
Diagnosing all of the above took all of 10 minutes of debugging. The
really annoying aspect is that fixing this completely breaks the pass.
;] There was an implicit reliance on the fact that no loads or stores
extended past the alloca once we decided to rewrite them in the final
stage of SROA. This was used to encode information about whether the
loads and stores had been split across multiple partitions of the
original alloca. That required threading explicit tracking of whether
a *use* of a partition is split across multiple partitions.
Once that was done, another problem arose: we allowed splitting of
integer loads and stores iff they were loads and stores to the entire
alloca. This is a really arbitrary limitation, and splitting at least
some integer loads and stores is crucial to maximize promotion
opportunities. My first attempt was to start removing the restriction
entirely, but currently that does Very Bad Things by causing *many*
common alloca patterns to be fully decomposed into i8 operations and
lots of or-ing together to produce larger integers on demand. The code
bloat is terrifying. That is still the right end-goal, but substantial
work must be done to either merge partitions or ensure that small i8
values are eagerly merged in some other pass. Sadly, figuring all this
out took essentially all the time and effort here.
So the end result is that we allow splitting only when the load or store
at least covers the alloca. That ensures widened loads and stores don't
hurt SROA, and that we don't rampantly decompose operations more than we
have previously.
All of this was already fairly well tested, and so I've just updated the
tests to cover the wide load behavior. I can add a test that crafts the
pass ordering magic which caused the original PR, but that seems really
brittle and to provide little benefit. The fundamental problem is that
widened loads should Just Work.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@177055 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 11687d4982200fe99a820ea513100f237dfc1609 |
|
07-Mar-2013 |
Jakub Staszak <kubastaszak@gmail.com> |
Keep coding stanard.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176661 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 9497005d38b1c0d5183374493c08982eb8586ead |
|
07-Mar-2013 |
Jakub Staszak <kubastaszak@gmail.com> |
Don't create IRBuilder if we can return from the method earlier.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176660 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| bcff7b77348e299d25e7ada9ea5bffb80aab55b2 |
|
19-Feb-2013 |
Jakub Staszak <kubastaszak@gmail.com> |
Remove unused variable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175568 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 50573b1c274d290a81ab76331104206ea0b09bde |
|
19-Feb-2013 |
Jakub Staszak <kubastaszak@gmail.com> |
Minor cleanups. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175567 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 4263ed33a77a3679545e784da1ab26e79032a467 |
|
19-Feb-2013 |
Jakub Staszak <kubastaszak@gmail.com> |
Remove unneeded #includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175565 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ba6f722d6a80efeacb69c12f9322d858affb4d2b |
|
19-Feb-2013 |
Jakub Staszak <kubastaszak@gmail.com> |
Fix typos.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175562 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| f1af1feeee0f0ec797410762c006211f9c1e2a0f |
|
29-Jan-2013 |
Edwin Vane <edwin.vane@intel.com> |
Fixing warnings revealed by gcc release build
Fixed set-but-not-used warnings.
Reviewer: gribozavr
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173810 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 0b8c9a80f20772c3793201ab5b251d3520b9cea3 |
|
02-Jan-2013 |
Chandler Carruth <chandlerc@gmail.com> |
Move all of the header files which are involved in modelling the LLVM IR
into their new header subdirectory: include/llvm/IR. This matches the
directory structure of lib, and begins to correct a long standing point
of file layout clutter in LLVM.
There are still more header files to move here, but I wanted to handle
them in separate commits to make tracking what files make sense at each
layer easier.
The only really questionable files here are the target intrinsic
tablegen files. But that's a battle I'd rather not fight today.
I've updated both CMake and Makefile build systems (I think, and my
tests think, but I may have missed something).
I've also re-sorted the includes throughout the project. I'll be
committing updates to Clang, DragonEgg, and Polly momentarily.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171366 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 6c3074958370bf25dc6e4e4b757f0c083e245dbe |
|
01-Jan-2013 |
Benjamin Kramer <benny.kra@googlemail.com> |
Add IRBuilder::CreateVectorSplat and use it to simplify code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171349 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 0ea4c3d8c0d6b0815aa1381167d0f53baf186dbd |
|
01-Jan-2013 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Clean up unused assignment warnings from clang's analyzer.
No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171348 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 98281a20503896349bd152e2dfe87435d3a6aada |
|
30-Dec-2012 |
Nuno Lopes <nunoplopes@sapo.pt> |
convert a bunch of callers from DataLayout::getIndexedOffset() to GEP::accumulateConstantOffset().
The later API is nicer than the former, and is correct regarding wrap-around offsets (if anyone cares).
There are a few more places left with duplicated code, which I'll remove soon.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171259 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| e21708e4aaf741d0c9ccb5a5ddc75738fea7b61f |
|
18-Dec-2012 |
Nadav Rotem <nrotem@apple.com> |
SROA: Replace calls to getScalarSizeInBits to DataLayout's API because
getScalarSizeInBits could not handle vectors of pointers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170412 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| b0de1e31d11056037c4db3e2ecfe1547e85c3e1c |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix another SROA crasher, PR14601.
This was a silly oversight, we weren't pruning allocas which were used
by variable-length memory intrinsics from the set that could be widened
and promoted as integers. Fix that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170353 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 99a54942ae0fb6fdca03e91b2e492e9738fa4436 |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Teach the rewriting of memcpy calls to support subvector copies.
This also cleans up a bit of the memcpy call rewriting by sinking some
irrelevant code further down and making the call-emitting code a bit
more concrete.
Previously, memcpy of a subvector would actually miscompile (!!!) the
copy into a single vector element copy. I have no idea how this ever
worked. =/ This is the memcpy half of PR14478 which we probably weren't
noticing previously because it didn't actually assert.
The rewrite relies on the newly refactored insert- and extractVector
functions to do the heavy lifting, and those are the same as used for
loads and stores which makes the test coverage a bit more meaningful
here.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170338 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 8bbff2348d378192b332db38394498d83ed4feeb |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a secondary bug I introduced while fixing the first part of PR14478.
The first half of fixing this bug was actually in r170328, but was
entirely coincidental. It did however get me to realize the nature of
the bug, and adapt the test case to test more interesting behavior. In
turn, that uncovered the rest of the bug which I've fixed here.
This should fix two new asserts that showed up in the vectorize nightly
tester.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170333 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 5814a88ce9ebe415015b37477c841304fa8b7b1a |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Hoist a convertValue call to the two paths where it is needed.
I noticed this while looking at r170328. We only ever do a vector
rewrite when the alloca *is* the vector type, so it's good to not paper
over bugs here by doing a convertValue that isn't needed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170331 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 7f7d201d737ecb354abd683d63ab8abbf83158c3 |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Hoist the insertVector helper to be a static helper.
This will allow its use inside of memcpy rewriting as well. This routine
is more complex than extractVector, and some of its uses are not 100%
where I want them to be so there is still some work to do here.
While this can technically change the output in some cases, it shouldn't
be a change that matters -- IE, it can leave some dead code lying around
that prior versions did not, etc.
Yet another step in the refactorings leading up to the solution to the
last component of PR14478.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170328 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 5c531eb8b121b83ad14ac5cf94c8fd9c50ef6f5b |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Lift the extractVector helper all the way out to a static helper function.
The method helpers all implicitly act upon the alloca, and what we
really want is a fully generic helper. Doing memcpy rewrites is more
special than all other rewrites because we are at times rewriting
instructions which touch pointers *other* than the alloca. As
a consequence all of the helpers needed by memcpy rewriting of
sub-vector copies will need to be generalized fully.
Note that all of these helpers ({insert,extract}{Integer,Vector}) are
woefully uncommented. I'm going to go back through and document them
once I get the factoring correct.
No functionality changed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170325 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 8ab1efd5fcff755b4ccc86d6d3da52cfe7cf0b9d |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Factor the vector load rewriting into a more generic form.
This makes it suitable for use in rewriting memcpy in the presence of
subvector memcpy intrinsics.
No functionality changed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170324 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 17c84ea594c6f10cb13c84ebe765b54f234c82ef |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix the first part of PR14478: memset now works.
PR14478 highlights a serious problem in SROA that simply wasn't being
exercised due to a lack of vector input code mixed with C-library
function calls. Part of SROA was written carefully to handle subvector
accesses via memset and memcpy, but the rewriter never grew support for
this. Fixing it required refactoring the subvector access code in other
parts of SROA so it could be shared, and then fixing the splat formation
logic and using subvector insertion (this patch).
The PR isn't quite fixed yet, as memcpy is still broken in the same way.
I'm starting on that series of patches now.
Hopefully this will be enough to bring the bullet benchmark back to life
with the bb-vectorizer enabled, but that may require fixing memcpy as
well.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170301 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| d6e4397a5b68836d9dae1d4a30da6f54ece51219 |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Extract the logic for inserting a subvector into a vector alloca.
No functionality changed. Another step of refactoring toward solving
PR14487.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170300 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 225d25de490386dd6398842c19f567b921a97db6 |
|
17-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Lift the integer splat computation into a helper function.
No functionality changed. Refactoring leading up to the fix for PR14478
which requires some significant changes to the memset and memcpy
rewriting.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170299 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 19820053fe46dbc91c43edb80a693fa6aae09251 |
|
15-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Relax an overly aggressive assert to fix PR14572.
The alloca width is based on the alloc size, not the type size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170270 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ed90ed077a58d6eff6aede206273ae0aeefa3a94 |
|
10-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Add a new visitor for walking the uses of a pointer value.
This visitor provides infrastructure for recursively traversing the
use-graph of a pointer-producing instruction like an alloca or a malloc.
It maintains a worklist of uses to visit, so it can handle very deep
recursions. It automatically looks through instructions which simply
translate one pointer to another (bitcasts and GEPs). It tracks the
offset relative to the original pointer as long as that offset remains
constant and exposes it during the visit as an APInt offset. Finally, it
performs conservative escape analysis.
However, currently it has some limitations that should be addressed
going forward:
1) It doesn't handle vectors of pointers.
2) It doesn't provide a cheaper visitor when the constant offset
tracking isn't needed.
3) It doesn't support non-instruction pointer values.
The current functionality is exactly what is required to implement the
SROA pointer-use visitors in terms of this one, rather than in terms of
their own ad-hoc base visitor, which was always very poorly specified.
SROA has been converted to use this, and the code there deleted which
this utility now provides.
Technically speaking, using this new visitor allows SROA to handle a few
more cases than it previously did. It is now more aggressive in ignoring
chains of instructions which look like they would defeat SROA, but in
fact do not because they never result in a read or write of memory.
While this is "neat", it shouldn't be interesting for real programs as
any such chains should have been removed by others passes long before we
get to SROA. As a consequence, I've not added any tests for these
features -- it shouldn't be part of SROA's contract to perform such
heroics.
The goal is to extend the functionality of this visitor going forward,
and re-use it from passes like ASan that can benefit from doing
a detailed walk of the uses of a pointer.
Thanks to Ben Kramer for the code review rounds and lots of help
reviewing and debugging this patch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169728 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 3d9afa8e9714fb4d0fd0d210a35dcffbc6e33742 |
|
10-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix PR14548: SROA was crashing on a mixture of i1 and i8 loads and stores.
When SROA was evaluating a mixture of i1 and i8 loads and stores, in
just a particular case, it would tickle a latent bug where we compared
bits to bytes rather than bits to bits. As a consequence of the latent
bug, we would allow integers through which were not byte-size multiples,
a situation the later rewriting code was never intended to handle.
In release builds this could trigger all manner of oddities, but the
reported issue in PR14548 was forming invalid bitcast instructions.
The only downside of this fix is that it makes it more clear that SROA
in its current form is not capable of handling mixed i1 and i8 loads and
stores. Sometimes with the previous code this would work by luck, but
usually it would crash, so I'm not terribly worried. I'll watch the LNT
numbers just to be sure.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169719 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 0da9175d90bcd202810ebef1b51d18d9ed0f9a98 |
|
09-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Switch SROA to pop Uses off the back of its visitors' queues.
This will more closely match the behavior of the new PtrUseVisitor that
I am adding. Hopefully this will not change the actual behavior in any
way, but by making the processing order more similar help in debugging.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169697 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| d04a8d4b33ff316ca4cf961e06c9e312eff8e64f |
|
03-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Use the new script to sort the includes of every file under lib.
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.
Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169131 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| e74a4a791535b9756fa7312e5cd6f746ecc1c989 |
|
03-Dec-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Remove some buggy and apparantly unnecessary code from SROA.
The partitioning logic attempted to handle uses of an alloca with an
offset starting before the alloca so long as the use had some overlap
with the alloca itself. However, there was a bug where we tested
'(uint64_t)Offset >= AllocSize' without first checking whether 'Offset'
was positive. As a consequence, essentially every negative offset (that
is, starting *before* the alloca does) would be thrown out, even if it
was overlapping. The subsequent code to throw out negative offsets which
were actually non-overlapping was essentially dead. The code to *handle*
overlapping negative offsets was actually dead!
I've just removed all of this, and taught SROA to discard any uses which
start prior to the alloca from the beginning. It has the lovely property
of simplifying the code. =] All the tests still pass, and in fact no new
tests are needed as this is already covered by our testsuite. Fixing the
code so that negative offsets work the way the comments indicate they
were supposed to work causes regressions. That's how I found this.
Anyways, this is all progress in the correct direction -- tightening up
SROA to be maximally aggressive. Some day, I really hope to turn
out-of-bounds accesses to an alloca into 'unreachable'.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169120 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 5bded7525b4eb1ae4ce3f7d05159de95d0f595f2 |
|
01-Dec-2012 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Avoid struct and array types early to avoid creating an overly large integer type.
Fixes PR14465.
Differential Revision: http://llvm-reviews.chandlerc.com/D148
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169084 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 84bcf93e0fd225de2217d1b712c01586a633a6d8 |
|
30-Nov-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Move the InstVisitor utility into VMCore where it belongs. It heavily
depends on the IR infrastructure, there is no sense in it being off in
Support land.
This is in preparation to start working to expand InstVisitor into more
special-purpose visitors that are still generic and can be re-used
across different passes. The expansion will go into the Analylis tree
though as nothing in VMCore needs it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168972 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 07df765e65204203f0185a7a243e5ec3a5c4b21c |
|
21-Nov-2012 |
Chandler Carruth <chandlerc@gmail.com> |
PR14055: Implement support for sub-vector operations in SROA.
Now if we can transform an alloca into a single vector value, but it has
subvector, non-element accesses, we form the appropriate shufflevectors
to allow SROA to proceed. This fixes PR14055 which pointed out a very
common pattern that SROA couldn't handle -- mixed vec3 and vec4
operations on a single alloca.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168418 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 3a902d0ae7763e952eb2fa70e2b07ccb038ae235 |
|
20-Nov-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Use LLVM_ENABLE_DUMP for the variables used in printing as well as the
printing functions themselves.
Part of PR14324 (which should have just been a patch to the list, but
hey...)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168362 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 176792990e1b8df4894d0dabddb735abbb254e7c |
|
20-Nov-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix PR14132 and handle OOB loads speculated throuh PHI nodes.
The issue is that we may end up with newly OOB loads when speculating
a load into the predecessors of a PHI node, and this confuses the new
integer splitting logic in some cases, triggering an assertion failure.
In fact, the branch in question must be dead code as it loads from
a too-narrow alloca. Add code to handle this gracefully and leave the
requisite FIXMEs for both optimizing more aggressively and doing more to
aid sanitizing invalid code which triggers these patterns.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168361 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| a1237b16c9dcc3d40ed47025b3e6042a99c5c1b5 |
|
20-Nov-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Add a comment to associate a FIXME with a PR where it is matters.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168347 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| f5837aacd4b04e551fb194f4095006b7e6f2b991 |
|
20-Nov-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Rework the rewriting of loads and stores for vector and integer allocas
to properly handle the combinations of these with split integer loads
and stores. This essentially replaces Evan's r168227 by refactoring the
code in a different way, and trynig to mirror that refactoring in both
the load and store sides of the rewriting.
Generally speaking there was some really problematic duplicated code
here that led to poorly founded assumptions and then subtle bugs. Now
much of the code actually flows through and follows a more consistent
style and logical path. There is still a tiny bit of duplication on the
store side of things, but it is much less bad.
This also changes the logic to never re-use a load or store instruction
as that was simply too error prone in practice.
I've added a few tests (one a reduction of the one in Evan's original
patch, which happened to be the same as the report in PR14349). I'm
going to look at adding a few more tests for things I found and fixed in
passing (such as the volatile tests in the vectorizable predicate).
This patch has survived bootstrap, and modulo one bugfix survived
Duncan's test suite, but let me know if anything else explodes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168346 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| fd22019ec36e8335254430e61a1e5a3f8a681f6a |
|
17-Nov-2012 |
Evan Cheng <evan.cheng@apple.com> |
Teach SROA rewriteVectorizedStoreInst to handle cases when the loaded value is narrower than the stored value. rdar://12713675
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168227 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 426c2bf5cdd2173e4a33aea8cb92cf684a724f4b |
|
01-Nov-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Revert the majority of the next patch in the address space series:
r165941: Resubmit the changes to llvm core to update the functions to
support different pointer sizes on a per address space basis.
Despite this commit log, this change primarily changed stuff outside of
VMCore, and those changes do not carry any tests for correctness (or
even plausibility), and we have consistently found questionable or flat
out incorrect cases in these changes. Most of them are probably correct,
but we need to devise a system that makes it more clear when we have
handled the address space concerns correctly, and ideally each pass that
gets updated would receive an accompanying test case that exercises that
pass specificaly w.r.t. alternate address spaces.
However, from this commit, I have retained the new C API entry points.
Those were an orthogonal change that probably should have been split
apart, but they seem entirely good.
In several places the changes were very obvious cleanups with no actual
multiple address space code added; these I have not reverted when
I spotted them.
In a few other places there were merge conflicts due to a cleaner
solution being implemented later, often not using address spaces at all.
In those cases, I've preserved the new code which isn't address space
dependent.
This is part of my ongoing effort to clean out the partial address space
code which carries high risk and low test coverage, and not likely to be
finished before the 3.2 release looms closer. Duncan and I would both
like to see the above issues addressed before we return to these
changes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167222 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ece6c6bb6329748b92403c06ac87f45c43485911 |
|
01-Nov-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Revert the series of commits starting with r166578 which introduced the
getIntPtrType support for multiple address spaces via a pointer type,
and also introduced a crasher bug in the constant folder reported in
PR14233.
These commits also contained several problems that should really be
addressed before they are re-committed. I have avoided reverting various
cleanups to the DataLayout APIs that are reasonable to have moving
forward in order to reduce the amount of churn, and minimize the number
of commits that were reverted. I've also manually updated merge
conflicts and manually arranged for the getIntPtrType function to stay
in DataLayout and to be defined in a plausible way after this revert.
Thanks to Duncan for working through this exact strategy with me, and
Nick Lewycky for tracking down the really annoying crasher this
triggered. (Test case to follow in its own commit.)
After discussing with Duncan extensively, and based on a note from
Micah, I'm going to continue to back out some more of the more
problematic patches in this series in order to ensure we go into the
LLVM 3.2 branch with a reasonable story here. I'll send a note to
llvmdev explaining what's going on and why.
Summary of reverted revisions:
r166634: Fix a compiler warning with an unused variable.
r166607: Add some cleanup to the DataLayout changes requested by
Chandler.
r166596: Revert "Back out r166591, not sure why this made it through
since I cancelled the command. Bleh, sorry about this!
r166591: Delete a directory that wasn't supposed to be checked in yet.
r166578: Add in support for getIntPtrType to get the pointer type based
on the address space.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167221 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 5801ff93e3b81ee18fdd57f59d0e776ac3b27384 |
|
01-Nov-2012 |
Jakub Staszak <kubastaszak@gmail.com> |
Don't insert and erase load instruction. Simply create (new) and delete it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167196 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 70dace3052984e843202ec98844a2a6636debb89 |
|
30-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix PR14212: For some strange reason I treated vectors differently from
integers in that the code to handle split alloca-wide integer loads or
stores doesn't come first. It should, for the same reasons as with
integers, and the PR attests to that. Also had to fix a busted assert in
that this test case also covers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167051 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| a2b88163af30f59d12ae0172565f3406bdbf6c45 |
|
25-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Teach SROA how to split whole-alloca integer loads and stores into
smaller integer loads and stores.
The high-level motivation is that the frontend sometimes generates
a single whole-alloca integer load or store during ABI lowering of
splittable allocas. We need to be able to break this apart in order to
see the underlying elements and properly promote them to SSA values. The
hope is that this fixes some performance regressions on x86-32 with the
new SROA pass.
Unfortunately, this causes quite a bit of churn in the test cases, and
bloats some IR that comes out. When we see an alloca that consists soley
of bits and bytes being extracted and re-inserted, we now do some
splitting first, before building widened integer "bucket of bits"
representations. These are always well folded by instcombine however, so
this shouldn't actually result in missed opportunities.
If this splitting of all-integer allocas does cause problems (perhaps
due to smaller SSA values going into the RA), we could potentially go to
some extreme measures to only do this integer splitting trick when there
are non-integer component accesses of an alloca, but discovering this is
quite expensive: it adds yet another complete walk of the recursive use
tree of the alloca.
Either way, I will be watching build bots and LNT bots to see what
fallout there is here. If anyone gets x86-32 numbers before & after this
change, I would be very interested.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166662 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| b52fb876171e3670e7307fda4459ca005d49d9f5 |
|
24-Oct-2012 |
Micah Villmow <villmow@gmail.com> |
Add some cleanup to the DataLayout changes requested by Chandler.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166607 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| b8bce928f4ffdf50eff69334f3e25b27848536b6 |
|
24-Oct-2012 |
Micah Villmow <villmow@gmail.com> |
Back out r166591, not sure why this made it through since I cancelled the command. Bleh, sorry about this!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166596 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 2f87640b86315beab8a5671cc23f524e59c58bd3 |
|
24-Oct-2012 |
Micah Villmow <villmow@gmail.com> |
Delete a directory that wasn't supposed to be checked in yet.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166591 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 2a13242b208b80f2d20cdb3067ccb82edb7d1d0f |
|
20-Oct-2012 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Simplify code. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166375 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 2360b7ad99eedecaae512373e7be49c2143550cf |
|
18-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Refactor insert and extract of sub-integers into static helpers that
operate purely on values. Sink the alloca loading and storing logic into
the rewrite routines that are specific to alloca-integer-rewrite
driving. This is just a refactoring here, but the subsequent step will
be to reuse the insertion and extraction logic when rewriting integer
loads and stores that have been split and decomposed into narrower loads
and stores.
No functionality changed other than different names for instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166176 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 02d5333eea0078e2551b541403898f4422e83251 |
|
18-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
This FIXME was fixed some time ago. =]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166175 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 02bf98ab38355aafbe9a763caa019bc06ad9ce3d |
|
17-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
This just in, it is a *bad idea* to use 'udiv' on an offset of
a pointer. A very bad idea. Let's not do that. Fixes PR14105.
Note that this wasn't *that* glaring of an oversight. Originally, these
routines were only called on offsets within an alloca, which are
intrinsically positive. But over the evolution of the pass, they ended
up being called for arbitrary offsets, and things went downhill...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166095 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 020d9d5feb59b4e92b4b55c5850fe3a5f77671b0 |
|
17-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a really annoying "bug" introduced in r165941. The change from that
revision makes no sense. We cannot use the address space of the *post
indexed* type to conclude anything about a *pre indexed* pointer type's
size. More importantly, this index can never be over a pointer. We are
indexing over arrays and vectors here.
Of course, I have no test case here. Neither did the original patch. =/
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166091 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 2c39b15073db81d93bb629303915b7d7e5d088dc |
|
15-Oct-2012 |
Micah Villmow <villmow@gmail.com> |
Resubmit the changes to llvm core to update the functions to support different pointer sizes on a per address space basis.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165941 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| d2cd73f6a5103f5de221a67410e35e6bee211fa3 |
|
15-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Update the memcpy rewriting to fully support widened int rewriting. This
includes extracting ints for copying elsewhere and inserting ints when
copying into the alloca. This should fix the CanSROA assertion coming
out of Clang's regression test suite.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165931 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 94fc64c42fe27aa16dd5611eb3657ff0297801c8 |
|
15-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Follow-up fix to r165928: handle memset rewriting for widened integers,
and generally clean up the memset handling. It had rotted a bit as the
other rewriting logic got polished more.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165930 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 81ff90db4403f9b4ee3574f00ebdfba4a12177c9 |
|
15-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
First major step toward addressing PR14059. This teaches SROA to handle
cases where we have partial integer loads and stores to an otherwise
promotable alloca to widen[1] those loads and stores to cover the entire
alloca and bitcast them into the appropriate type such that promotion
can proceed.
These partial loads and stores stem from an annoying confluence of ARM's
calling convention and ABI lowering and the FCA pre-splitting which
takes place in SROA. Clang lowers a { double, double } in-register
function argument as a [4 x i32] function argument to ensure it is
placed into integer 32-bit registers (a really unnerving implicit
contract between Clang and the ARM backend I would add). This results in
a FCA load of [4 x i32]* from the { double, double } alloca, and SROA
decomposes this into a sequence of i32 loads and stores. Inlining
proceeds, code gets folded, but at the end of the day, we still have i32
stores to the low and high halves of a double alloca. Widening these to
be i64 operations, and bitcasting them to double prior to loading or
storing allows promotion to proceed for these allocas.
I looked quite a bit changing the IR which Clang produces for this case
to be more friendly, but small changes seem unlikely to help. I think
the best representation we could use currently would be to pass 4 i32
arguments thereby avoiding any FCAs, but that would still require this
fix. It seems like it might eventually be nice to somehow encode the ABI
register selection choices outside of the parameter type system so that
the parameter can be a { double, double }, but the CC register
annotations indicate that this should be passed via 4 integer registers.
This patch does not address the second problem in PR14059, which is the
reverse: when a struct alloca is loaded as a *larger* single integer.
This patch also does not address some of the code quality issues with
the FCA-splitting. Those don't actually impede any optimizations really,
but they're on my list to clean up.
[1]: Pedantic footnote: for those concerned about memory model issues
here, this is safe. For the alloca to be promotable, it cannot escape or
have any use of its address that could allow these loads or stores to be
racing. Thus, widening is always safe.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165928 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 11cb6ba5d0fe65c7249ef94c83df294b88d575f0 |
|
15-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Hoist the canConvertValue predicate and the convertValue transform out
into static helper functions. They're really quite generic and are going
to be needed elsewhere shortly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165927 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 07525a6be6bce604f3b528c91973ac4e66742266 |
|
13-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Teach SROA to cope with wrapper aggregates. These show up a lot in ABI
type coercion code, especially when targetting ARM. Things like [1
x i32] instead of i32 are very common there.
The goal of this logic is to ensure that when we are picking an alloca
type, we look through such wrapper aggregates and across any zero-length
aggregate elements to find the simplest type possible to form a type
partition.
This logic should (generally speaking) rarely fire. It only ends up
kicking in when an alloca is accessed using two different types (for
instance, i32 and float), and the underlying alloca type has wrapper
aggregates around it. I noticed a significant amount of this occurring
looking at stepanov_abstraction generated code for arm, and suspect it
happens elsewhere as well.
Note that this doesn't yet address truly heinous IR productions such as
PR14059 is concerning. Those result in mismatched *sizes* of types in
addition to mismatched access and alloca types.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165870 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ac104272d9fc42af8dc6853853b96d489685e5a7 |
|
13-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Speculatively harden the conversion logic. I have no idea if this will
help the dragonegg builders, and no test case at this point, but this
was one dimly plausible case I spotted by inspection. Hopefully will get
a testcase from those bots soon-ish, and will tidy this up with proper
testing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165869 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| c2fcf1a67126a079e34f04bd09a50ae85fdd9469 |
|
13-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Silence a warning in -assert builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165867 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 520eeaeffd5ef724ea5c68192f77331571f38060 |
|
13-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Clean up how we rewrite loads and stores to the whole alloca. When these
are single value types, the load and store should be directly based upon
the alloca and then bitcasting can fix the type as needed afterward.
This might in theory improve some of the IR coming out of SROA, but
I don't expect big changes yet and don't have any test cases on hand.
This is really just a cleanup/refactoring patch. The next patch will
cause this code path to be hit a lot more, actually get SROA to promote
more allocas and include several more test cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165864 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| fb384d61c78b60787ed65475d8403aee65023962 |
|
11-Oct-2012 |
Micah Villmow <villmow@gmail.com> |
Revert 165732 for further review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165747 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| f3840d2c16a4ec4c879a8ded402835746de380f8 |
|
11-Oct-2012 |
Micah Villmow <villmow@gmail.com> |
Add in the first iteration of support for llvm/clang/lldb to allow variable per address space pointer sizes to be optimized correctly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165726 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 2fdb25b5a9c096f38bbd3f622c34ff9b2744e56d |
|
09-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix PR14034, an infloop / heap corruption / crash bug in the new SROA.
Thanks to Benjamin for the raw test case. This one took about 50 times
longer to reduce than to fix. =/
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165476 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 3574eca1b02600bac4e625297f4ecf745f4c4f32 |
|
08-Oct-2012 |
Micah Villmow <villmow@gmail.com> |
Move TargetData to DataLayout.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165402 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 0559d31c374050096e1071208661f97f24dbe299 |
|
05-Oct-2012 |
NAKAMURA Takumi <geek4civic@gmail.com> |
SROA.cpp: Fix a warning, [-Wunused-variable]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165309 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| fca3f4021ae9a561edb6d2fcb4a282b6f25ab144 |
|
05-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Teach the new SROA a new trick. Now we zap any memcpy or memmoves which
are in fact identity operations. We detect these and kill their
partitions so that even splitting is unaffected by them. This is
particularly important because Clang relies on emitting identity memcpy
operations for struct copies, and these fold away to constants very
often after inlining.
Fixes the last big performance FIXME I have on my plate.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165285 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 0e9da58af0e480af2fa87f8a61aa28b465fec280 |
|
05-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Lift the speculation visitor above all the helpers that are targeted at
the rewrite visitor to make the fact that the speculation is completely
independent a bit more clear.
I promise that this is just a cut/paste of the one visitor and adding
the annonymous namespace wrappings. The diff may look completely
preposterous, it does in git for some reason.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165284 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| b2d98c29170d99fb43de0244a7f4f93a8893c208 |
|
04-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix PR13969, a mini-phase-ordering issue with the new SROA pass.
Currently, we re-visit allocas when something changes about the way they
might be *split* to allow better scalarization to take place. However,
we weren't handling the case when the *promotion* is what would change
the behavior of SROA. When an address derived from an alloca is stored
into another alloca, we consider the first to have escaped. If the
second is ever promoted to an SSA value, we will suddenly be able to run
the SROA pass on the first alloca.
This patch adds explicit support for this form if iteration. When we
detect a store of a pointer derived from an alloca, we flag the
underlying alloca for reprocessing after promotion. The logic works hard
to only do this when there is definitely going to be promotion and it
might remove impediments to the analysis of the alloca.
Thanks to Nick for the great test case and Benjamin for some sanity
check review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165223 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| aa3cb334af09ae30a3283f748cacb4a919785035 |
|
04-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Teach the integer-promotion rewrite strategy to be endianness aware.
Sorry for this being broken so long. =/
As part of this, switch all of the existing tests to be Little Endian,
which is the behavior I was asserting in them anyways! Add in a new
big-endian test that checks the interesting behavior there.
Another part of this is to tighten the rules abotu when we perform the
full-integer promotion. This logic now rejects cases where there fully
promoted integer is a non-multiple-of-8 bitwidth or cases where the
loads or stores touch bits which are in the allocated space of the
alloca but are not loaded or stored when accessing the integer. Sadly,
these aren't really observable today as the rest of the pass will
already ensure the invariants hold. However, the latter situation is
likely to become a potential concern in the future.
Thanks to Benjamin and Duncan for early review of this patch. I'm still
looking into whether there are further endianness issues, please let me
know if anyone sees BE failures persisting past this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165219 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 322e9ba2cb92330a345bd6e606c64a7b16bd929f |
|
03-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix an issue where we failed to adjust the alignment constraint on
a memcpy to reflect that '0' has a different meaning when applied to
a load or store. Now we correctly use underaligned loads and stores for
the test case added.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165101 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| f710fb14ee951fe533b94eb1046d422e94d4f1f3 |
|
03-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Try to use a better set of abstractions for computing the alignment
necessary during rewriting. As part of this, fix a real think-o here
where we might have left off an alignment specification when the address
is in fact underaligned. I haven't come up with any way to trigger this,
as there is always some other factor that reduces the alignment, but it
certainly might have been an observable bug in some way I can't think
of. This also slightly changes the strategy for placing explicit
alignments on loads and stores to only do so when the alignment does not
match that required by the ABI. This causes a few redundant alignments
to go away from test cases.
I've also added a couple of tests that really push on the alignment that
we end up with on loads and stores. More to come here as I try to fix an
underlying bug I have conjectured and produced test cases for, although
it's not clear if this bug is the one currently hitting dragonegg's
gcc47 bootstrap.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165100 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 75eac5f0ebff4d0ffe10ce6bc8f2867c5f15315b |
|
03-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Switch the SetVector::remove_if implementation to use partition which
preserves the values of the relocated entries, unlikely remove_if. This
allows walking them and erasing them.
Also flesh out the predicate we are using for this to support the
various constraints actually imposed on a UnaryPredicate -- without this
we can't compose it with std::not1.
Thanks to Sean Silva for the review here and noticing the issue with
std::remove_if.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165073 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 5c5b3cf5b8af06b8e9347f3f45e8c67438ffd446 |
|
03-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Teach the new SROA to handle cases where an alloca that has already been
scheduled for processing on the worklist eventually gets deleted while
we are processing another alloca, fixing the original test case in
PR13990.
To facilitate this, add a remove_if helper to the SetVector abstraction.
It's not easy to use the standard abstractions for this because of the
specifics of SetVectors types and implementation.
Finally, a nice small test case is included. Thanks to Benjamin for the
fantastic reduced test case here! All I had to do was delete some empty
basic blocks!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165065 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| fdb15850e6f2d395b128bde0f2fff6b20c96adc6 |
|
02-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix another crasher in SROA, reported by Joel.
We require that the indices into the use lists are stable in order to
build fast lookup tables to locate a particular partition use from an
operand of a PHI or select. This is (obviously in hind sight)
incompatible with erasing elements from the array. Really, we don't want
to erase anyways. It is expensive, and a rare operation. Instead, simply
weaken the contract of the PartitionUse structure to allow null Use
pointers to represent dead uses. Now we can clear out the pointer to
mark things as dead, and all it requires is adding some 'continue'
checks to the various loops.
I'm still reducing a test case for this, as the test case I have is
huge. I think this one I can get a nice test case for though, as it was
much more deterministic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165032 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| a346f462716b4eb32d27ffe20fc8696473f225f6 |
|
02-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a silly coding error on my part. The whole point of the speculator
being separate was that it can grow the use list. As a consequence, we
can't use the iterator-pair interface, we need an index based interface.
Expose such an interface from the AllocaPartitioning, and use it in the
speculator.
This should at least fix a use-after-free bug found by Duncan, and may
fix some of the other crashers.
I don't have a nice deterministic test case yet, but if I get a good
one, I'll add it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165027 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| c7a4ca7a34c7b618e00badeaf2b3d6d23d72e36d |
|
01-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Make this plural. Spotted by Duncan in review (and a very old typo, this
is the second time I've moved this comment around...)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164939 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 0a7ca313ec39c8b3b1f215b1f11a2f784d7f0400 |
|
01-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Prune some unnecessary includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164938 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 673850aa2dceb8fe1b7a20b72339bf803af8609f |
|
01-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix several issues with alignment. We weren't always accounting for type
alignment requirements of the new alloca. As one consequence which was
reported as a bug by Duncan, we overaligned memcpy calls to ranges of
allocas after they were rewritten to types with lower alignment
requirements. Other consquences are possible, but I don't have any test
cases for them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164937 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 1e1b16c48515d6d54525ab279e4acb5ddd05c83d |
|
01-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Factor the PHI and select speculation into a separate rewriter. This
could probably be factored still further to hoist this logic into
a generic helper, but currently I don't have particularly clean ideas
about how to handle that.
This at least allows us to drop custom load rewriting from the
speculation logic, which in turn allows the existing load rewriting
logic to fire. In theory, this could enable vector promotion or other
tricks after speculation occurs, but I've not dug into such issues. This
is primarily just cleaning up the factoring of the code and the
resulting logic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164933 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 77c12700339496cfa18e3004aabb807ec1c5a1a2 |
|
01-Oct-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Refactor the PartitionUse structure to actually use the Use* instead of
a pair of instructions, one for the used pointer and the second for the
user. This simplifies the representation and also makes it more dense.
This was noticed because of the miscompile in PR13926. In that case, we
were running up against a fundamental "bad idea" in the speculation of
PHI and select instructions: the speculation and rewriting are
interleaved, which requires phi speculation to also perform load
rewriting! This is bad, and causes us to miss opportunities to do (for
example) vector rewriting only exposed after PHI speculation, etc etc.
It also, in the old system, required us to insert *new* load uses into
the current partition's use list, which would then be ignored during
rewriting because we had already extracted an end iterator for the use
list. The appending behavior (and much of the other oddities) stem from
the strange de-duplication strategy in the PartitionUse builder.
Amusingly, all this went without notice for so long because it could
only be triggered by having *different* GEPs into the same partition of
the same alloca, where both different GEPs were operands of a single
PHI, and where the GEP which was not encountered first also had multiple
uses within that same PHI node... Hence the insane steps required to
reproduce.
So, step one in fixing this fundamental bad idea is to make the
PartitionUse actually contain a Use*, and to make the builder do proper
deduplication instead of funky de-duplication. This is enough to remove
the appending behavior, and fix the miscompile in PR13926, but there is
more work to be done here. Subsequent commits will lift the speculation
into its own visitor. It'll be a useful step toward potentially
extracting all of the speculation logic into a generic utility
transform.
The existing PHI test case for repeated operands has been made more
extreme to catch even these issues. This test case, run through the old
pass, will exactly reproduce the miscompile from PR13926. ;] We were so
close here!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164925 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| b67c9a5b0295faaab40c6c64c433ab679d225a70 |
|
29-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a somewhat surprising miscompile where code relying on an ABI
alignment could lose it due to the alloca type moving down to a much
smaller alignment guarantee.
Now SROA will actively compute a proper alignment, factoring the target
data, any explicit alignment, and the offset within the struct. This
will in some cases lower the alignment requirements, but when we lower
them below those of the type, we drop the alignment entirely to give
freedom to the code generator to align it however is convenient.
Thanks to Duncan for the lovely test case that pinned this down. =]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164891 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| d0ac06d18812dd1c20ad55f9324c550609138593 |
|
26-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Analogous fix to memset and memcpy rewriting. Don't have a test case
contrived for these yet, as I spotted them by inspection and the test
cases are a bit more tricky to phrase.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164691 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 238fd1591ee004130ca1e857cda1d270b15efe3d |
|
26-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
When rewriting the pointer operand to a load or store which has
alignment guarantees attached, re-compute the alignment so that we
consider offsets which impact alignment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164690 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 81b001a2209be99e42aceed74f411c7cefc4d022 |
|
26-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Teach all of the loads, stores, memsets and memcpys created by the
rewriter in SROA to carry a proper alignment. This involves
interrogating various sources of alignment, etc. This is a more complete
and principled fix to PR13920 as well as related bugs pointed out by Eli
in review and by inspection in the area.
Also by inspection fix the integer and vector promotion paths to create
aligned loads and stores. I still need to work up test cases for
these... Sorry for the delay, they were found purely by inspection.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164689 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| b3dca3f50e649caa29f97736f0fa4b2fe1097034 |
|
26-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Revert the business end of r164636 and try again. I'll come in again. ;]
This should really, really fix PR13916. For real this time. The
underlying bug is... a bit more subtle than I had imagined.
The setup is a code pattern that leads to an @llvm.memcpy call with two
equal pointers to an alloca in the source and dest. Now, not any pattern
will do. The alloca needs to be formed just so, and both pointers should
be wrapped in different bitcasts etc. When this precise pattern hits,
a funny sequence of events transpires. First, we correctly detect the
potential for overlap, and correctly optimize the memcpy. The first
time. However, we do simplify the set of users of the alloca, and that
causes us to run the alloca back through the SROA pass in case there are
knock-on simplifications. At this point, a curious thing has happened.
If we happen to have an i8 alloca, we have direct i8 pointer values. So
we don't bother creating a cast, we rewrite the arguments to the memcpy
to dircetly refer to the alloca.
Now, in an unrelated area of the pass, we have clever logic which
ensures that when visiting each User of a particular pointer derived
from an alloca, we only visit that User once, and directly inspect all
of its operands which refer to that particular pointer value. However,
the mechanism used to detect memcpy's with the potential to overlap
relied upon getting visited once per *Use*, not once per *User*. This is
always true *unless* the same exact value is both source and dest. It
turns out that almost nothing actually produces that pattern though.
We can hand craft test cases that more directly test this behavior of
course, and those are included. Also, note that there is a significant
missed optimization here -- we prove in many cases that there is
a non-volatile memcpy call with identical source and dest addresses. We
shouldn't prevent splitting the alloca in that case, and in fact we
should just remove such memcpy calls eagerly. I'll address that in
a subsequent commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164669 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 051a318e67ae601c46b4fc7ceb5c5b0b605a296d |
|
26-Sep-2012 |
Nick Lewycky <nicholas@mxc.ca> |
Don't drop the alignment on a memcpy intrinsic when producing a store. This is
only a missed optimization opportunity if the store is over-aligned, but a
miscompile if the store's new type has a higher natural alignment than the
memcpy did. Fixes PR13920!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164641 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 48c0f65be363eeec4d69a96864e06b1570aedbfe |
|
25-Sep-2012 |
Nick Lewycky <nicholas@mxc.ca> |
Revert the business end of r164634, and replace it with a different fix. The
reason we were getting two of the same alloca is because of a memmove/memcpy
which had the same alloca in both the src and dest. Now we detect that case
directly. This has the same testcase as before, but fixes a clang test
CodeGenObjC/exceptions.m which runs clang -O2.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164636 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| c3f10e43fc4b1084bc109ff740d9ba4b6eaced0a |
|
25-Sep-2012 |
Nick Lewycky <nicholas@mxc.ca> |
Don't try to promote the same alloca twice. Fixes PR13916!
Chandler, it's not obvious that it's okay that this alloca gets into the list
twice to begin with. Please review and see whether this is the fix you really
want, but I wanted to get a fix checked in quickly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164634 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| c303463dcceb6a3b15e6f4d11d7c31c1ac2ce276 |
|
25-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a case where SROA did not correctly detect dead PHI or selects due
to chains or cycles between PHIs and/or selects. Also add a couple of
really nice test cases reduced from Kostya's reports in PR13905 and
PR13906. Both are fixed by this patch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164596 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 72bf29f45d9da6a2a6be910f55a4787d30d3d2eb |
|
25-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a crash in SROA. This was reported independently by Takumi and
David (I think), but I would appreciate folks verifying that this fixes
the big crasher.
I'm still working on a reduced test case, but because this was causing
problems I wanted to get the fix checked in quickly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164585 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| bc4021f31eaa97ee52655828da3e3de14a39e4a6 |
|
24-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Address one of the original FIXMEs for the new SROA pass by implementing
integer promotion analogous to vector promotion. When there is an
integer alloca being accessed both as its integer type and as a narrower
integer type, promote the narrower access to "insert" and "extract" the
smaller integer from the larger one, and make the integer alloca
a candidate for promotion.
In the new formulation, we don't care about target legal integer or use
thresholds to control things. Instead, we only perform this promotion to
an integer type which the frontend has already emitted a load or store
for. This bounds the scope and prevents optimization passes from
coalescing larger and larger entities into a single integer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164479 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 02e92a0b5dd75514c8489d3f87fe7c5262ca6290 |
|
23-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Switch to a signed representation for the dynamic offsets while walking
across the uses of the alloca. It's entirely possible for negative
numbers to come up here, and in some rare cases simply doing the 2's
complement arithmetic isn't the correct decision. Notably, we can't zext
the index of the GEP. The definition of GEP is that these offsets are
sign extended or truncated to the size of the pointer, and then wrapping
2's complement arithmetic used.
This patch fixes an issue that comes up with *no* input from the
buildbots or bootstrap afaict. The only place where it manifested,
disturbingly, is Clang's own regression test suite. A reduced and
targeted collection of tests are added to cope with this. Note that I've
tried to pin down the potential cases of overflow, but may have missed
some cases. I've tried to add a few cases to test this, but its hard
because LLVM has quite limited support for >64bit constructs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164475 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| d54a6b56a91c72cc33ce3dafcf1653123e6fdd87 |
|
22-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a case where the new SROA pass failed to zap dead operands to
selects with a constant condition. This resulted in the operands
remaining live through the SROA rewriter. Most of the time, this just
caused some dead allocas to persist and get zapped by later passes, but
in one case found by Joerg, it caused a crash when we tried to *promote*
the alloca despite it having this dead use. We already have the
mechanisms in place to handle this, just wire select up to them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164427 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 38f35fd3b74ab608362bb871fc84481f54624e17 |
|
19-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix the last crasher I've gotten a reproduction for in SROA. This one
from the dragonegg build bots when we turned on the full version of the
pass. Included a much reduced test case for this pesky bug, despite
bugpoint's uncooperative behavior.
Also, I audited all the similar code I could find and didn't spot any
other cases where this mistake cropped up.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164178 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 7c8df7aa0c02908a29f0b5e6cfccb531dccdb96b |
|
18-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix getCommonType in a different way from the way I fixed it when
working on FCA splitting. Instead of refusing to form a common type when
there are uses of a subsection of the alloca as well as a use of the
entire alloca, just skip the subsection uses and continue looking for
a whole-alloca use with a type that we can use.
This produces slightly prettier IR I think, and also fixes the other
failure in the test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164146 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 3b682bdf671e5dd0eb70c0498ba18e5c08874fc9 |
|
18-Sep-2012 |
Benjamin Kramer <benny.kra@googlemail.com> |
Fix build for compilers that don't understand injected class names properly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164142 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 371d5d86bd70756d2066692114fec599c33baf92 |
|
18-Sep-2012 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Use CRTP for OpSplitter to get rid of virtual dispatch and the virtual-dtor warnings that come with it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164140 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 6e67b254ae2cf2411cb2d1ae41bf0fbc05079f40 |
|
18-Sep-2012 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Replace the member function template contraption for recursively splitting aggregates into a real class.
No intended functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164135 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ad83d08ed04c541a2fbc8262a45077e565bb20e3 |
|
18-Sep-2012 |
NAKAMURA Takumi <geek4civic@gmail.com> |
SROA.cpp: Appease msvc.
...I don't know why this could appease msvc...baad.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164130 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 9e3f63957955461fce1f383b12eb70605a361f6f |
|
18-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Fix a warning in release builds and a test case I forgot to update with
a fix to getCommonType in the previous patch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164120 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| c370acdf9698b7eee11f3d8e3732f1d72cd25943 |
|
18-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Add a major missing piece to the new SROA pass: aggressive splitting of
FCAs. This is essential in order to promote allocas that are used in
struct returns by frontends like Clang. The FCA load would block the
rest of the pass from firing, resulting is significant regressions with
the bullet benchmark in the nightly test suite.
Thanks to Duncan for repeated discussions about how best to do this, and
to both him and Benjamin for review.
This appears to have blocked many places where the pass tries to fire,
and so I'm expect somewhat different results with this fix added.
As with the last big patch, I'm including a change to enable the SROA by
default *temporarily*. Ben is going to remove this as soon as the LNT
bots pick up the patch. I'm just trying to get a round of LNT numbers
from the stable machines in the lab.
NOTE: Four clang tests are expected to fail in the brief window where
this is enabled. Sorry for the noise!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164119 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 2d1c2a2fb8aadc180ce486670e6600afb14f7096 |
|
17-Sep-2012 |
Benjamin Kramer <benny.kra@googlemail.com> |
NewSROA: Provide a full set of operator< for ByteRanges.
MSVC8 won't compile lower_bound if one is missing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164035 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 63392ea3ba295d59260553245b14a435b5f71a3e |
|
16-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Refactor the SROA visitors for partitioning an alloca and building
partition use lists a bit. No functionality changed.
These visitors are actually visiting a tuple of a Use and an offset into
the alloca. However, we use the InstVisitor to handle the dispatch over
the users, and so the Use and Offset are stored in class member
variables and set just before each call to visit(). This is fairly
awkward and makes the functions a bit harder to read, but its the only
real option we have until InstVisitor can be rewritten to use variadic
templates.
However, this pattern shouldn't be followed on the helper member
functions where there is no interface constraint from the visitor. We
already were passing the instruction as a normal parameter rather than
use the Use to get at it, start passing the offset as well. This will
become more important in subsequent patches as the offset will in some
cases change while visiting a single instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164003 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 1c8db50a9aca4229ebee75ed9cfbf8b222292787 |
|
15-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Port the SSAUpdater-based promotion logic from the old SROA pass to the
new one, and add support for running the new pass in that mode and in
that slot of the pass manager. With this the new pass can completely
replace the old one within the pipeline.
The strategy for enabling or disabling the SSAUpdater logic is to do it
by making the requirement of the domtree analysis optional. By default,
it is required and we get the standard mem2reg approach. This is usually
the desired strategy when run in stand-alone situations. Within the
CGSCC pass manager, we disable requiring of the domtree analysis and
consequentially trigger fallback to the SSAUpdater promotion.
In theory this would allow the pass to re-use a domtree if one happened
to be available even when run in a mode that doesn't require it. In
practice, it lets us have a single pass rather than two which was
simpler for me to wrap my head around.
There is a hidden flag to force the use of the SSAUpdater code path for
the purpose of testing. The primary testing strategy is just to run the
existing tests through that path. One notable difference is that it has
custom code to handle lifetime markers, and one of the tests has been
enhanced to exercise that code.
This has survived a bootstrap and the test suite without serious
correctness issues, however my run of the test suite produced *very*
alarming performance numbers. I don't entirely understand or trust them
though, so more investigation is on-going.
To aid my understanding of the performance impact of the new SROA now
that it runs throughout the optimization pipeline, I'm enabling it by
default in this commit, and will disable it again once the LNT bots have
picked up one iteration with it. I want to get those bots (which are
much more stable) to evaluate the impact of the change before I jump to
any conclusions.
NOTE: Several Clang tests will fail because they run -O3 and check the
result's order of output. They'll go back to passing once I disable it
again.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163965 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| d0807690de6c735af84e35dd76a30420eaded16b |
|
14-Sep-2012 |
Benjamin Kramer <benny.kra@googlemail.com> |
SROA: Silence unused variable warnings in Release builds.
The NDEBUG hack is ugly, but I see no better solution.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163900 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 6b547a2af560fac18f7f7c9b82cd685fee2dde10 |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Rework the computation of a sub-structure natural type. There were
pointless checks in here, bad asserts, and just confusing code. I've
also added a bit more to the comment to clarify what this function is
really trying to do as it was not obvious to Duncan when studying it.
Thanks to Duncan for helping me dig through the issue.
No real functionality changed here in practical cases, and certainly no
test case. This is just cleanup spotted by inspection.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163897 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 01f85f02a426617e095143f1ac2c19c2e59ee76d |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Rely on the recursive check for pointer types rather than adding an
explicit check before recursing. A simplification requested by Duncan
during review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163896 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ad41dcfd875e8d3421a5f0970b6a886b6e10b3b7 |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Be a bit more aggressive in bailing out of this routine. Spotted by
inspection by Duncan during review. My suspicion is that we would still
have returned 0 anyways in this case, but doing it sooner is better.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163895 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 8ed1ed8dfd2cf9215446c637a416a5a31f96051e |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Add some comments clarifying that the GEP analysis for vector GEPs is
deeply suspicious and likely to go away eventually. Also fix a bogus
comment about one of the checks in the vector GEP analysis. Based on
review from Duncan.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163894 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 8615cd236ed9f9540b4a3ad05dd0a1d202a3f6b4 |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Move an instance variable to a local variable based on review by Duncan.
Originally I had anticipated needing to thread this through more bits of
the SROA pass itself, but that ended up not happening. In the end, this
is a much simpler way to manange the variable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163893 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 50754f089265bd6a4208d8b89c653e7f7854796a |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Add a comment about debug intrinsics that I *really* don't want to
forget from Duncan's review as a FIXME.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163892 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| b3dd9a13e8454fe069fafc1b259b64851f70e6e7 |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Add two asserts that Duncan thought would help ensure things don't rot
unexpectedly in the future. More fixes from his code review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163891 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 1c2fe28109ef84b5c2f8d839e18a08c3ce33c854 |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Remove some dead, commented out code Duncan spotted in review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163889 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ba13d2efbac767ff1b19217ee26cd2e548562cfd |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Wrap the dumping and printing routines in NDEBUG and LLVM_ENABLE_DUMP macros.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163888 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 7f5bedec7406f76383a2367c4d051d7aaf0d6308 |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Lots of comment fixes and cleanups from Duncan's review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163887 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| ad9f5b89677583ab6ac036dfb4227b7a59f394ee |
|
14-Sep-2012 |
NAKAMURA Takumi <geek4civic@gmail.com> |
SROA.cpp: Unbreak gcc, sorry!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163886 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 9b1e62454a4e3d41292157404f787fe1d463ddf4 |
|
14-Sep-2012 |
NAKAMURA Takumi <geek4civic@gmail.com> |
SROA.cpp: Appease msvc. LLVM_ATTRIBUTE(s) should come front of "const".
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163885 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 2a9bf25f60a136ff6cd9f15262974630888e2252 |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Speculative change to try to fix older GCC versions that can't handle
the injected class name of a dependent base class here.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163884 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|
| 713aa9431d84805c5b7ddce57159d14ee94cd548 |
|
14-Sep-2012 |
Chandler Carruth <chandlerc@gmail.com> |
Introduce a new SROA implementation.
This is essentially a ground up re-think of the SROA pass in LLVM. It
was initially inspired by a few problems with the existing pass:
- It is subject to the bane of my existence in optimizations: arbitrary
thresholds.
- It is overly conservative about which constructs can be split and
promoted.
- The vector value replacement aspect is separated from the splitting
logic, missing many opportunities where splitting and vector value
formation can work together.
- The splitting is entirely based around the underlying type of the
alloca, despite this type often having little to do with the reality
of how that memory is used. This is especially prevelant with unions
and base classes where we tail-pack derived members.
- When splitting fails (often due to the thresholds), the vector value
replacement (again because it is separate) can kick in for
preposterous cases where we simply should have split the value. This
results in forming i1024 and i2048 integer "bit vectors" that
tremendously slow down subsequnet IR optimizations (due to large
APInts) and impede the backend's lowering.
The new design takes an approach that fundamentally is not susceptible
to many of these problems. It is the result of a discusison between
myself and Duncan Sands over IRC about how to premptively avoid these
types of problems and how to do SROA in a more principled way. Since
then, it has evolved and grown, but this remains an important aspect: it
fixes real world problems with the SROA process today.
First, the transform of SROA actually has little to do with replacement.
It has more to do with splitting. The goal is to take an aggregate
alloca and form a composition of scalar allocas which can replace it and
will be most suitable to the eventual replacement by scalar SSA values.
The actual replacement is performed by mem2reg (and in the future
SSAUpdater).
The splitting is divided into four phases. The first phase is an
analysis of the uses of the alloca. This phase recursively walks uses,
building up a dense datastructure representing the ranges of the
alloca's memory actually used and checking for uses which inhibit any
aspects of the transform such as the escape of a pointer.
Once we have a mapping of the ranges of the alloca used by individual
operations, we compute a partitioning of the used ranges. Some uses are
inherently splittable (such as memcpy and memset), while scalar uses are
not splittable. The goal is to build a partitioning that has the minimum
number of splits while placing each unsplittable use in its own
partition. Overlapping unsplittable uses belong to the same partition.
This is the target split of the aggregate alloca, and it maximizes the
number of scalar accesses which become accesses to their own alloca and
candidates for promotion.
Third, we re-walk the uses of the alloca and assign each specific memory
access to all the partitions touched so that we have dense use-lists for
each partition.
Finally, we build a new, smaller alloca for each partition and rewrite
each use of that partition to use the new alloca. During this phase the
pass will also work very hard to transform uses of an alloca into a form
suitable for promotion, including forming vector operations, speculating
loads throguh PHI nodes and selects, etc.
After splitting is complete, each newly refined alloca that is
a candidate for promotion to a scalar SSA value is run through mem2reg.
There are lots of reasonably detailed comments in the source code about
the design and algorithms, and I'm going to be trying to improve them in
subsequent commits to ensure this is well documented, as the new pass is
in many ways more complex than the old one.
Some of this is still a WIP, but the current state is reasonbly stable.
It has passed bootstrap, the nightly test suite, and Duncan has run it
successfully through the ACATS and DragonEgg test suites. That said, it
remains behind a default-off flag until the last few pieces are in
place, and full testing can be done.
Specific areas I'm looking at next:
- Improved comments and some code cleanup from reviews.
- SSAUpdater and enabling this pass inside the CGSCC pass manager.
- Some datastructure tuning and compile-time measurements.
- More aggressive FCA splitting and vector formation.
Many thanks to Duncan Sands for the thorough final review, as well as
Benjamin Kramer for lots of review during the process of writing this
pass, and Daniel Berlin for reviewing the data structures and algorithms
and general theory of the pass. Also, several other people on IRC, over
lunch tables, etc for lots of feedback and advice.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163883 91177308-0d34-0410-b5e6-96231b3b80d8
/external/llvm/lib/Transforms/Scalar/SROA.cpp
|