• Home
  • History
  • Annotate
  • only in /art/runtime/gc/collector/
History log of /art/runtime/gc/collector/
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
b94ed50a53ecbfbd360a2771a22ec91283b9a5f3 23-Jul-2014 Mathieu Chartier <mathieuc@google.com> resolved conflicts for merge of 38c488bc to lmp-dev-plus-aosp

Change-Id: If8fbbd51232fd01a481cb47df426de169c0c830c
38c488bcd41ba632a646d7a1d790ec71a2fcf6fa 16-Jul-2014 Mathieu Chartier <mathieuc@google.com> Recycle mem-maps for collector transitions.

We now create spaces when we need them for collector transitions or
homogeneous compaction by recycling mem maps. Change the bump
pointer space size to be as large as the heap capacity instead of
1/2 heap capacity like it used to be. For GSS, bump pointer spaces
are set to 32MB currently.

Changed GSS to have main space == non moving space since we don't
need to copy from the main space.

Fixes GC stress tests 074, 096.
Fixed test 080 oom throw with -Xmx2m for GC stress test, this was
broken since it was allocating a 4 MB array before starting the
OOM process.

Bug: 14059466
Bug: 16406852

(cherry picked from commit b363f666883860d40823d5528df3c98c897f74f4)

Change-Id: I62877cfa24ec944a6f34ffac30334f454a8002fd
emi_space.cc
8060f68b0eb74acb0b89a0a32373fd1d66a9fc7f 23-Jul-2014 Mathieu Chartier <mathieuc@google.com> am b76cac63: Revert "Revert "Disable adding main and non moving spaces to immune region in GSS""

* commit 'b76cac637691c29daa9c44e493b5bc26346ed116':
Revert "Revert "Disable adding main and non moving spaces to immune region in GSS""
b76cac637691c29daa9c44e493b5bc26346ed116 23-Jul-2014 Mathieu Chartier <mathieuc@google.com> Revert "Revert "Disable adding main and non moving spaces to immune region in GSS""

This reverts commit f85c2fb317399ab540854cd7551ac47690366543.
ark_sweep-inl.h
ark_sweep.cc
emi_space-inl.h
emi_space.cc
emi_space.h
f63063c7f9f37786bc174e78c57309a7888d7b98 22-Jul-2014 Mathieu Chartier <mathieuc@google.com> am 01bd8e3d: Merge "Revert "Disable adding main and non moving spaces to immune region in GSS"" into lmp-dev

* commit '01bd8e3d8e663fcc48d6941e4876a09657fee6c9':
Revert "Disable adding main and non moving spaces to immune region in GSS"
f85c2fb317399ab540854cd7551ac47690366543 22-Jul-2014 Mathieu Chartier <mathieuc@google.com> Revert "Disable adding main and non moving spaces to immune region in GSS"

Bug: 16399257

This reverts commit be0562fb14e6754ee932b8d9c97e2a6df3a91119.

Change-Id: I29e07a8fa1e972990e5bf1ddf8c9a3538ea5f9cf
ark_sweep-inl.h
ark_sweep.cc
emi_space-inl.h
emi_space.cc
emi_space.h
be0562fb14e6754ee932b8d9c97e2a6df3a91119 14-Jul-2014 Mathieu Chartier <mathieuc@google.com> Disable adding main and non moving spaces to immune region in GSS

Disabled adding the main and non moving space to the immune region.
This will enable us to recycle bump pointer spaces for malloc space
-> malloc space compaction as well as collector transitions.

Also added logic for falling back to the non moving space, we may
copy objects there.

Refactored mod union table logic into MarkReachableObjects.

No measurable performance benefit or regression.

Bug: 14059466
Bug: 16291259

(cherry picked from commit 4c13a3ff475f206c4d0a86ee2595c45392fd942f)

Change-Id: I858b4fbddca888e164052ad247565a0bdbea68b5
ark_sweep-inl.h
ark_sweep.cc
emi_space-inl.h
emi_space.cc
emi_space.h
479f131d4bd3829dd512312020808b05f5a591f1 16-Jul-2014 Mathieu Chartier <mathieuc@google.com> Merge "Disable adding main and non moving spaces to immune region in GSS"
4c13a3ff475f206c4d0a86ee2595c45392fd942f 14-Jul-2014 Mathieu Chartier <mathieuc@google.com> Disable adding main and non moving spaces to immune region in GSS

Disabled adding the main and non moving space to the immune region.
This will enable us to recycle bump pointer spaces for malloc space
-> malloc space compaction as well as collector transitions.

Also added logic for falling back to the non moving space, we may
copy objects there.

Refactored mod union table logic into MarkReachableObjects.

No measurable performance benefit or regression.

Bug: 14059466
Bug: 16291259

Change-Id: If663d9fdbde943b988173b7f6ac844e5f78a0327
ark_sweep-inl.h
ark_sweep.cc
emi_space-inl.h
emi_space.cc
emi_space.h
e63db27db913f1a88e2095a1ee8239b2bb9124e8 16-Jul-2014 Ian Rogers <irogers@google.com> Break apart header files.

Create libart-gtest for common runtime and compiler gtest routines.
Rename CompilerCallbacksImpl that is quick compiler specific.
Rename trace clock source constants to not use the overloaded profiler term.

Change-Id: I4aac4bdc7e7850c68335f81e59a390133b54e933
mmune_region.h
ark_compact.h
emi_space.h
f5997b4d3f889569d5a2b724d83d764bfbb8d106 20-Jun-2014 Mathieu Chartier <mathieuc@google.com> More advanced timing loggers.

The new timing loggers have lower overhead since they only push into
a vector. The new format has two types, a start timing and a stop
timing. You can thing of these as brackets associated with a
timestamp. It uses these to construct various statistics when needed,
such as: Total time, exclusive time, and nesting depth.

Changed PrettyDuration to have a default of 3 digits after the decimal
point.

Exaple of a GC dump with exclusive / total times and indenting:
I/art (23546): GC iteration timing logger [Exclusive time] [Total time]
I/art (23546): 0ms InitializePhase
I/art (23546): 0.305ms/167.746ms MarkingPhase
I/art (23546): 0ms BindBitmaps
I/art (23546): 0ms FindDefaultSpaceBitmap
I/art (23546): 0ms/1.709ms ProcessCards
I/art (23546): 0.183ms ImageModUnionClearCards
I/art (23546): 0.916ms ZygoteModUnionClearCards
I/art (23546): 0.610ms AllocSpaceClearCards
I/art (23546): 1.373ms AllocSpaceClearCards
I/art (23546): 0.305ms/6.318ms MarkRoots
I/art (23546): 2.106ms MarkRootsCheckpoint
I/art (23546): 0.153ms MarkNonThreadRoots
I/art (23546): 4.287ms MarkConcurrentRoots
I/art (23546): 43.461ms UpdateAndMarkImageModUnionTable
I/art (23546): 0ms/112.712ms RecursiveMark
I/art (23546): 112.712ms ProcessMarkStack
I/art (23546): 0.610ms/2.777ms PreCleanCards
I/art (23546): 0.305ms/0.855ms ProcessCards
I/art (23546): 0.153ms ImageModUnionClearCards
I/art (23546): 0.610ms ZygoteModUnionClearCards
I/art (23546): 0.610ms AllocSpaceClearCards
I/art (23546): 0.549ms AllocSpaceClearCards
I/art (23546): 0.549ms MarkRootsCheckpoint
I/art (23546): 0.610ms MarkNonThreadRoots
I/art (23546): 0ms MarkConcurrentRoots
I/art (23546): 0.610ms ScanGrayImageSpaceObjects
I/art (23546): 0.305ms ScanGrayZygoteSpaceObjects
I/art (23546): 0.305ms ScanGrayAllocSpaceObjects
I/art (23546): 1.129ms ScanGrayAllocSpaceObjects
I/art (23546): 0ms ProcessMarkStack
I/art (23546): 0ms/0.977ms (Paused)PausePhase
I/art (23546): 0.244ms ReMarkRoots
I/art (23546): 0.672ms (Paused)ScanGrayObjects
I/art (23546): 0ms (Paused)ProcessMarkStack
I/art (23546): 0ms/0.610ms SwapStacks
I/art (23546): 0.610ms RevokeAllThreadLocalAllocationStacks
I/art (23546): 0ms PreSweepingGcVerification
I/art (23546): 0ms/10.621ms ReclaimPhase
I/art (23546): 0.610ms/0.702ms ProcessReferences
I/art (23546): 0.214ms/0.641ms EnqueueFinalizerReferences
I/art (23546): 0.427ms ProcessMarkStack
I/art (23546): 0.488ms SweepSystemWeaks
I/art (23546): 0.824ms/9.400ms Sweep
I/art (23546): 0ms SweepMallocSpace
I/art (23546): 0.214ms SweepZygoteSpace
I/art (23546): 0.122ms SweepMallocSpace
I/art (23546): 6.226ms SweepMallocSpace
I/art (23546): 0ms SweepMallocSpace
I/art (23546): 2.144ms SweepLargeObjects
I/art (23546): 0.305ms SwapBitmaps
I/art (23546): 0ms UnBindBitmaps
I/art (23546): 0.275ms FinishPhase
I/art (23546): GC iteration timing logger: end, 178.971ms

Change-Id: Ia55b65609468f212b3cd65cda66b843da42be645
arbage_collector.cc
ark_compact.cc
ark_sweep.cc
emi_space.cc
10fb83ad7442c8cf3356a89ec918e0786f110981 16-Jun-2014 Mathieu Chartier <mathieuc@google.com> Shared single GC iteration accounting for all GCs.

Previously, each garbage collector had data that was only used
during collection. Since only one collector can be running at any
given time, we can make this data be shared between all collectors.
This reduces memory usage since we don't need to have redundant
information for each GC types. Also reduced how much code is required
to sweep spaces.

Bug: 9969166
Change-Id: I31caf0ee4d572f75e0c66863fe7db12c08ae08e7
arbage_collector.cc
arbage_collector.h
ark_compact.cc
ark_sweep.cc
emi_space.cc
52e4b43d62896b56f8c2bd041e528472bb4a0d8d 10-Jun-2014 Mathieu Chartier <mathieuc@google.com> Add mark compact collector.

The mark compact collector is a 4 phase collection, doing a normal
full mark_sweep, calculating forwarding addresses of objects in the
from space, updating references of objects in the from space, and
moving the objects in the from space.

Support is diabled by default since it needs to have non movable
classes and field arrays. Performance numbers is around 50% as fast.

The main advantage that this has over semispace is that the worst
case memory usage is 50% since we only need one space isntead of two.

TODO: Make field arrays and classes movable. This causes complication
since Object::VisitReferences relies on these, so if we update the
fields of an object but another future object uses this object to
figure out what fields are reference fields it doesn't work.

Bug: 14059466

Change-Id: I661ed3b71ad4dde124ef80312c95696b4a5665a1
ark_compact.cc
ark_compact.h
ark_sweep.cc
ark_sweep.h
emi_space.cc
19d46b44f2abe742be22e32908dbfd9e6dd9bfea 18-Jun-2014 Mathieu Chartier <mathieuc@google.com> Fix systrace logging, total paused time, and bytes saved message.

Moved the GC top level systrace logging to be inside of Collector::Run.
This prevents cases where we forgot to call it such as background
compaction. Fixed a unit error regarding total pause time. Fixed
negative bytes saved to use the word "expanded".

Bug: 15702709

Change-Id: Ic2991ecad2daa000d0aee9d559b8bc77d8c160aa
arbage_collector.cc
arbage_collector.h
308351ada0008b0cbe1a5afc31c302c975554ee4 15-Jun-2014 Mathieu Chartier <mathieuc@google.com> Change reference processing to use heap references.

Removes several SetReferents for updating moved referents. Cleaned
up other aspects of the code.

Change-Id: Ibcb4d713fadea617efee7e936352ddf77ff4c370
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
c5f17732d8144491c642776b6b48c85dfadf4b52 06-Jun-2014 Ian Rogers <irogers@google.com> Remove deprecated WITH_HOST_DALVIK.

Bug: 13751317
Fix the Mac build:
- disable x86 selector removal that causes OS/X 10.9 kernel panics,
- madvise don't need does zero memory on the Mac, factor into MemMap
routine,
- switch to the elf.h in elfutils to avoid Linux kernel dependencies,
- we can't rely on exclusive_owner_ being available from other pthread
libraries so maintain our own when futexes aren't available (we
can't rely on the OS/X 10.8 hack any more),
- fix symbol naming in assembly code,
- work around C library differences,
- disable backtrace in DumpNativeStack to avoid a broken libbacktrace
dependency,
- disable main thread signal handling logic,
- align the stack in stub_test,
- use $(HOST_SHLIB_SUFFIX) rather than .so in host make file variables.

Not all host tests are passing on the Mac with this change. dex2oat
works as does running HelloWorld.
Change-Id: I5a232aedfb2028524d49daa6397a8e60f3ee40d3
ark_sweep.cc
e8aff40f8222ea75d2af89773156cb07fb8653c7 28-May-2014 Hiroshi Yamauchi <yamauchi@google.com> Merge "Use a heap allocated free buffer in MarkSweep::SweepArray()."
bbdc5bc5fd5141711879a6c85d80ac45b7aad5d0 28-May-2014 Hiroshi Yamauchi <yamauchi@google.com> Use a heap allocated free buffer in MarkSweep::SweepArray().

This is for avoiding a large stack frame size.

Bug: 15278350
Change-Id: I63e92c2dd0ad7078fd74ee2392717dc44b431e9a
ark_sweep.cc
ark_sweep.h
4240c5112aedb174a3e1c1f071ea54e82168b130 27-May-2014 Mathieu Chartier <mathieuc@google.com> Don't create bump pointer spaces unless necessary.

We now only create these spaces if the foreground or background
collector is a moving collector.

Bug: 13564922
Bug: 14059466
Change-Id: I9f83c7881bc28423b78d5249c8e8854add91b60c
emi_space.cc
3e5cf305db800b2989ad57b7cde8fb3cc9fa1b9e 21-May-2014 Ian Rogers <irogers@google.com> Begin migration of art::Atomic to std::atomic.

Change-Id: I4858d9cbed95e5ca560956b9dabd976cebe68333
ark_sweep.cc
ark_sweep.h
700a402244a1a423da4f3ba8032459f4b65fa18f 20-May-2014 Ian Rogers <irogers@google.com> Now we have a proper C++ library, use std::unique_ptr.

Also remove the Android.libcxx.mk and other bits of stlport compatibility
mechanics.

Change-Id: Icdf7188ba3c79cdf5617672c1cfd0a68ae596a61
ark_sweep.h
emi_space.h
13bf2e6a6c14bccf5377998b7568100ffd417f8e 19-May-2014 Hiroshi Yamauchi <yamauchi@google.com> Check and log a to-space overflow explicitly.

Measured no noticeable performance regression with Ritzperf
EvaluateAndApplyChanges and -Xgc:SS on Mako.

Change-Id: Ib5ab934b867f91789a82ae771b45c95e9d01390c
emi_space.cc
507dfdd147c97bfbadebfd63584d094b6a4e7b47 16-May-2014 Ian Rogers <irogers@google.com> Compatibility layer to transition from UniquePtr to std::unique_ptr.

Use ART_WITH_STLPORT (enabled for the target) to cause the use of UniquePtr,
for the host switch to std::unique_ptr. For now the type remains called
UniquePtr.
Make dalvik compile with clang on the host, move its build to C++11.

Change-Id: I5ba8d2757904bc089ed62047ea03de3c0853fb12
ark_sweep.h
emi_space.h
eb8167a4f4d27fce0530f6724ab8032610cd146b 08-May-2014 Mathieu Chartier <mathieuc@google.com> Add Handle/HandleScope and delete SirtRef.

Delete SirtRef and replaced it with Handle. Handles are value types
which wrap around StackReference*.

Renamed StackIndirectReferenceTable to HandleScope.

Added a scoped handle wrapper which wraps around an Object** and
restores it in its destructor.

Renamed Handle::get -> Get.

Bug: 8473721

Change-Id: Idbfebd4f35af629f0f43931b7c5184b334822c7a
arbage_collector.h
ark_sweep.cc
ark_sweep.h
emi_space-inl.h
emi_space.cc
emi_space.h
ticky_mark_sweep.cc
a17288e3b9dd6751005fed3386251d2d0949be74 09-May-2014 Mathieu Chartier <mathieuc@google.com> Add null check in MarkSweepMarkObjectSlowPath.

Bug: 14626564
Change-Id: Ief387e96b813440a35a7c480897576d1543891f6
ark_sweep.cc
8254783763226255a90c1ecf0a8e285501ba9df5 08-May-2014 Mathieu Chartier <mathieuc@google.com> Merge "Add concurrent reference processing."
78f7b4c04ab6e8b5581921bc95b67a9beee1c246 06-May-2014 Mathieu Chartier <mathieuc@google.com> Add concurrent reference processing.

Concurrent reference processing currently works by going into native
code from java.lang.ref.Reference.get(). From there, we have a fast
path if the references aren't being processed which returns the
referent without needing to access any locks. In the slow path we
block until reference processing is complete. It may be possible to
improve the slow path if the referent is blackened.

TODO: Investigate doing the fast path in java code by using racy reads
of a static volatile boolean. This will work as long as there are no
suspend points inbetween the boolean read and referent read.

Bug: 14381653

Change-Id: I1546b55be4691fe4ff4aa6d857b234cce7187d87
ark_sweep.cc
ark_sweep.h
emi_space.cc
24faeb23de9aa3038cac6655b575a4b991a6add4 07-May-2014 Hiroshi Yamauchi <yamauchi@google.com> Trigger GSS whole-heap collection based on large object allocations.

This is to avoid a large amount of accumulated dead large objects. For
example, in Maps, there was a case where ~100 MB of large objects were
accumulated before collected.

Bug: 11650816
Change-Id: Ie0ab1561282902a823e110d45f87146ba41fe6f8
emi_space.cc
emi_space.h
e76e70f424468f311c2061c291e8384263f3968c 03-May-2014 Mathieu Chartier <mathieuc@google.com> Add RecordFree to the GarbageCollector interface

RecordFree now calls the Heap::RecordFree as well as updates the
garbage collector's internal bytes freed accounting.

Change-Id: I8cb03748b0768e3c8c50ea709572960e6e4ad219
arbage_collector.cc
arbage_collector.h
ark_sweep.cc
emi_space.cc
0651d41e41341fb2e9ef3ee41dc1f1bfc832dbbb 29-Apr-2014 Mathieu Chartier <mathieuc@google.com> Add thread unsafe allocation methods to spaces.

Used by SS/GSS collectors since these run with mutators suspended and
only allocate from a single thread. Added AllocThreadUnsafe to
BumpPointerSpace and RosAllocSpace. Added AllocThreadUnsafe which uses
current runs as thread local runs for a thread unsafe allocation.
Added code to revoke current runs which are the same idx as thread
local runs.

Changed:
The number of thread local runs in each thread is now the the number
of thread local runs in RosAlloc instead of the number of size
brackets.

Total GC time / time on EvaluateAndApplyChanges.
TLAB SS:
Before: 36.7s / 7254
After: 16.1s / 4837

TLAB GSS:
Before: 6.9s / 3973
After: 5.7s / 3778

Bug: 8981901

Change-Id: Id1d264ade3799f431bf7ebbdcca6146aefbeb632
emi_space.cc
emi_space.h
b0fa5dc7769c1e054032f39de0a3f6d6dd06f8cf 29-Apr-2014 Ian Rogers <irogers@google.com> Force inlining on trivial accessors.

Make volatility for GetFieldObject a template parameter.
Move some trivial mirror::String routines to a -inl.h.

Bug: 14285442

Change-Id: Ie23b11d4f18cb15a62c3bbb42837a8aaf6b68f92
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
6f365cc033654a5a3b45eaa1379d4b5f156b0cee 23-Apr-2014 Mathieu Chartier <mathieuc@google.com> Enable concurrent sweeping for non-concurrent GC.

Refactored the GarbageCollector to let all of the phases be run by
the collector's RunPhases virtual method. This lets the GC decide
which phases should be concurrent and reduces how much baked in GC
logic resides in GarbageCollector.

Enabled concurrent sweeping in the semi space and non concurrent
mark sweep GCs. Changed the semi-space collector to have a swap semi
spaces boolean which can be changed with a setter.

Fixed tests to pass with GSS collector, there was an error related to
the large object space limit.

Before (EvaluateAndApplyChanges):
GSS paused GC time 7.81s/7.81s, score: 3920

After (EvaluateAndApplyChanges):
GSS paused GC time 6.94s/7.71s, score: 3900

Benchmark score doesn't go up since the GC happens in the allocating
thread. There is a slight reduction in pause times experienced by
other threads (0.8s total).

Added options for pre sweeping GC heap verification and pre sweeping
rosalloc verification.

Bug: 14226004
Bug: 14250892
Bug: 14386356

Change-Id: Ib557d0590c1ed82a639d0f0281ba67cf8cae938c
oncurrent_copying.h
arbage_collector.cc
arbage_collector.h
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
cb905718826da268d8d8e09296806256f202c9f4 29-Apr-2014 Hiroshi Yamauchi <yamauchi@google.com> Merge "Improve GSS reference processing."
4db7449c0065971ec3a64ca04aeb64cfd2e802f0 23-Apr-2014 Hiroshi Yamauchi <yamauchi@google.com> Improve GSS reference processing.

Support the case where the reference object is in the free list space
and the referent object is in the bump pointer space at a bump pointer
space collection.

Bug: 11650816
Change-Id: If98b08edc9e37351c74ee07cb3f2d30c2b4d0056
emi_space.cc
emi_space.h
62ab87bb3ff4830def25a1716f6785256c7eebca 28-Apr-2014 Mathieu Chartier <mathieuc@google.com> Always log explicit GC.

People who use DDMS want to see that a GC actually occurs when they
press GC button.

Bug: 14325353
Change-Id: I44e0450c92abf7223d33552ed37f626fe63e1c28
arbage_collector.cc
ark_sweep.cc
emi_space.cc
bbd695c71e0bf518f582e84524e1cdeb3de3896c 16-Apr-2014 Mathieu Chartier <mathieuc@google.com> Replace ObjectSet with LargeObjectBitmap.

Speeds up large object marking since large objects no longer required
a lock. Changed the GCs to use the heap bitmap for marking objects
which aren't in the fast path. This eliminates the need for a
MarkLargeObject function.

Maps before (10 GC iterations):
Mean partial time: 180ms
Mean sticky time: 151ms

Maps after:
Mean partial time: 161ms
Mean sticky time: 101ms

Note: the GC durations are long due to recent ergonomic changes and
because the fast bulk free hasn't yet been enabled. Over 50% of the
GC time is spent in RosAllocSpace::FreeList.

Bug: 13571028

Change-Id: Id8f94718aeaa13052672ccbae1e8edf77d653f62
arbage_collector.cc
ark_sweep.cc
ark_sweep.h
emi_space-inl.h
emi_space.cc
emi_space.h
4d7f61d44a732cfbc8573e5d93364983fd746888 17-Apr-2014 Mathieu Chartier <mathieuc@google.com> Use non volatile lock words in semispace collector.

GSS FormulaEvaluationActions.EvaluateAndApplyChanges:
Before GC time: 9.1s
After GC time: 7.98s

Fixed timing logger errors.

Change-Id: I4193c6ccbbbe7a7220dfaabbf3472a5dcebae616
emi_space-inl.h
emi_space.cc
emi_space.h
a8e8f9c0a8e259a807d7b99a148d14104c24209d 09-Apr-2014 Mathieu Chartier <mathieuc@google.com> Refactor space bitmap to support different alignments.

Required for:
Using space bitmaps instead of std::set in mod union table +
remembered set.
Using a bitmap instead of set for large object marking.

Bug: 13571028

Change-Id: Id024e9563d4ca4278f79607cdb2f81895121b113
arbage_collector.cc
ark_sweep.cc
ark_sweep.h
emi_space-inl.h
emi_space.cc
emi_space.h
b272cd3013e046b7b001a091b0925b99a844e382 12-Apr-2014 Mathieu Chartier <mathieuc@google.com> Fix heap verification with SS/GSS.

Since I had removed the SwapSemiSpaces it was trying to verify
references in the from space of the semispace collector, which isn't
necessarily valid.

Bug: 13989162

Change-Id: I8e0b7119a1a53d63f2c71ecdb276290eda5bae07
emi_space.cc
31f441464c0c8f840aba37e236ad133f30308d70 08-Apr-2014 Mathieu Chartier <mathieuc@google.com> Clean-up heap spaces.

We now use the CMS collector instead of the semispace collector when
the phone is booting. We still perform compaction during the zygote
space creation. This reduces time spent in GC by ~2s during boot
and doesn't affect zygote space size.

Changed the space creation logic to create the temp space when a
background transition occurs.

Added a flag to each space which is true if you are allowed to
move objects that are within this space.

Removed SwapSemiSpaces call from the semi space collector, it is now
the job of the caller to do this with threads suspended. This
simplifies the logic in the zygote compaction / heap transition code
since these do not copy from one semispace to another.

Added Space::Clear to RosAllocSpace and DlMallocSpace. This greatly
simplifies the code used for collector transitions.

Time spent in GC creating zygote space:
Before: 3.4s, After: 1.28s
No change in zygote space size.

Bug: 13878055

Change-Id: I700348ab7d5bf3aa537c0cd70c0fed09aa4b0623
emi_space.cc
8edd574b5105f714961159ee5f8d04a264ae83eb 09-Apr-2014 Hiroshi Yamauchi <yamauchi@google.com> Merge "GSS: Fix the bump pointer space only collection."
df386c551405ce9668e827584f744c6f098761fa 09-Apr-2014 Hiroshi Yamauchi <yamauchi@google.com> GSS: Fix the bump pointer space only collection.

Fixes b/13912464 where the clear soft reference behavior accidentally
disabled the bump pointer space only collection.

Changed the collector name so that the GC logs would indicate the
generational mode and the collection mode.

Peformance enhancement: instead of triggering the whole heap
collection every 5 collections, count the bytes promoted since the
last whole heap collection and use it to decide when to trigger the
whole heap collection. This improves MemAllocTest by 5-10% (N4 and
host).

Bug: 13912464
Bug: 11650816
Bug: 9986565
Change-Id: I653a0dca62a8b54adf69abe2940a41eac70f809b
ark_sweep.cc
emi_space.cc
emi_space.h
5a48719b516a52d1a6800d8ae6f7dcba3d883bdc 08-Apr-2014 Mathieu Chartier <mathieuc@google.com> Reset GC timings after SIGQUIT.

We now reset the GC timings when a SIGQUIT happens, this is useful
for excluding GCs which happen during the initialization of an app
when measuring GC performance.

Change-Id: I68c79bdb279290c12ae588bc7e95ac24908c157e
arbage_collector.cc
arbage_collector.h
440e4ceb310349ee8eb569495bc04d3d7fbe71cb 01-Apr-2014 Mathieu Chartier <mathieuc@google.com> Add monitor deflation.

We now deflate the monitors when we perform a heap trim. This causes
a pause but it shouldn't matter since we should be in a state where
we don't care about pauses. Memory savings are hard to measure.

Fixed integer overflow bug in GetEstimatedLastIterationThroughput.

Bug: 13733906
Change-Id: I4e0e68add02e7f43370b3a5ea763d6fe8a5b212c
arbage_collector.cc
7bf9f190cd33a7e2f8584299eb889e9df66e0323 04-Apr-2014 Mathieu Chartier <mathieuc@google.com> Improve invalid root dumping.

The invalid root dumping now attempts to print the root type.

Change-Id: Ie821296d569f34909ba6e2705f5c347cd2143a3a
ark_sweep.cc
ark_sweep.h
79b1bf0ba1580cef94412ac3a62904ac89e6dfbc 01-Apr-2014 Hiroshi Yamauchi <yamauchi@google.com> Merge "Make the support code for read barriers a bit more general."
624468cd401cc1ac0dd70c746301e0788a597759 01-Apr-2014 Hiroshi Yamauchi <yamauchi@google.com> Make the support code for read barriers a bit more general.

Add an option for Baker in addition to Brooks.

Bug: 12687968
Change-Id: I8a31db817ff6686c72951b6534f588228e270b11
ark_sweep.cc
emi_space-inl.h
emi_space.cc
05a48b1f8e62564abb7c2fe674e3234d5861647f 01-Apr-2014 Mathieu Chartier <mathieuc@google.com> Fix stack overflow slow path error.

The frame size without spill was being passed into the slow path
instead of the spill size. This was incorrect since only the spills
will have been pushed at the point of the overflow check.

Also addressed an other comment.

Change-Id: Ic6e455122473a8f796b291d71f945bcf72788662
arbage_collector.h
0f7bf6a3ad1798fde328a2bff48a4bf2d750a36b 28-Mar-2014 Mathieu Chartier <mathieuc@google.com> Swap allocation stacks in pause.

This enables us to collect objects allocated during the GC for both
sticky, partial, and full GC. This also significantly simplifies GC
code. No measured performance impact on benchmarks, but this should
slightly increase sticky GC throughput.

Changed RevokeRosAllocThreadLocalBuffers to happen at most once per
GC. Previously it occured twice if pre-cleaning was enabled.

Renamed HandleDirtyObjectsPhase to PausePhase and enabled it for
non-concurrent GC. This helps reduce duplicated code which was in
both HandleDirtyObjectsPhase for concurrent GC and ReclaimPhase for
non-concurrent GC.

Change-Id: I533414b5c2cd2800f00724418e0ff90e7fdb0252
arbage_collector.cc
arbage_collector.h
ark_sweep.cc
ark_sweep.h
26162488da23d1cbd04e6112ea458847818f1dae 28-Mar-2014 Mathieu Chartier <mathieuc@google.com> Merge "Refactor some GC code."
f805f7772efe8f87055befb03647a970b5a4d59f 28-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> Merge "An empty collector skeleton for a read barrier-based collector."
4aeec176eaf11fe03f342aadcbb79142230270ed 28-Mar-2014 Mathieu Chartier <mathieuc@google.com> Refactor some GC code.

Reduced amount of code in mark sweep / semi space by moving
common logic to garbage_collector.cc. Cleaned up mod union tables
and deleted an unused implementation.

Change-Id: I4bcc6ba41afd96d230cfbaf4d6636f37c52e37ea
arbage_collector.cc
ark_sweep-inl.h
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
ticky_mark_sweep.h
d5307ec41c8344be0c32273ec4f574064036187d 28-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> An empty collector skeleton for a read barrier-based collector.

Bug: 12687968

Change-Id: Ic2a3a7b9943ca64e7f60f4d6ed552a316ea4a6f3
oncurrent_copying.cc
oncurrent_copying.h
arbage_collector.cc
afe4998fc15b8de093d6b282c9782d7182829e36 27-Mar-2014 Mathieu Chartier <mathieuc@google.com> Change sticky GC ergonomics to use GC throughput.

The old sticky ergonomics used partial/full GC when the bytes until
the footprint limit was < min free. This was suboptimal. The new
sticky GC ergonomics do partial/full GC when the throughput
of the current sticky GC iteration is <= mean throughput of the
partial/full GC.

Total GC time on FormulaEvaluationActions.EvaluateAndApplyChanges.
Before: 26.4s
After: 24.8s
No benchmark score change measured.

Bug: 8788501

Change-Id: I90000305e93fd492a8ef5a06ec9620d830eaf90d
arbage_collector.cc
arbage_collector.h
580a8dfda74e7999f6b3e28e239ae8461a39dfcb 26-Mar-2014 Mathieu Chartier <mathieuc@google.com> Less redundant verification.

~3 less objects verified per object scanned in the GC. Helps the
irogers dogfood experience.

Change-Id: I6efeab7842a6c702adecef73fb573c19291fecf2
emi_space.cc
3b05e9ba874449dbff65b01b8781001f7d93eea6 25-Mar-2014 Mathieu Chartier <mathieuc@google.com> Add missing debugger root visiting.

Bug: 13634574
Change-Id: I2a76f6c43f1d0ad1922f06deb40a71ff651129fd
emi_space-inl.h
emi_space.cc
emi_space.h
407f702da4f867c074fc3c8c688b8f8c32279eff 18-Feb-2014 Mathieu Chartier <mathieuc@google.com> Refactor object reference visiting logic.

Refactored the reference visiting logic to be in mirror::Object
instead of MarkSweep.

Change-Id: I773249478dc463d83b465e85c2402320488577c0
ark_sweep-inl.h
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
0e54cd0d8fc635d3dc8bf88a465fdade151a098f 20-Mar-2014 Mathieu Chartier <mathieuc@google.com> Refactor and optimize GC code.

Fixed the reference cache mod union table, and re-enabled it by
default. Added a boolean flag to count how many null objects,
immune, fast path, slow path objects we marked.

Slight speedup in mark stack processing, large speedup in image mod
union table scanning.
EvaluateAndApplyChanges Before:
Process mark stack time for full GC only:
12.464089s, 12.357870s, 12.538028s
Time spent marking mod image union table ~240ms.
After:
Process mark stack time: 12.299375s, 12.217142s, 12.187076s
Time spent marking mod image union table ~40ms.

TODO: Refactor reference visiting logic into mirror::Object.

Change-Id: I91889ded9d3f2bf127bc0051c1b1ff77e792e94f
mmune_region.cc
mmune_region.h
ark_sweep-inl.h
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
c93c530efc175954160c3834c93961a1a946a35a 21-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> Revoke rosalloc thread-local buffers at the checkpoint.

In the mark sweep collector, rosalloc thread-local buffers were
revoked during the pause. Now, they are revoked at the thread
checkpoint, as opposed to during the pause, which appears to help
reduce the pause time.

In Ritz MemAllocTest, the average sticky pause time went down ~20%
(925 us -> 724 us).

Bug: 13394464
Bug: 9986565
Change-Id: I104992a11b46d59264c0b9aa2db82b1ccf2826bc
arbage_collector.cc
arbage_collector.h
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
c7b4dfb8db21528b179689a80b69d868de394ef1 21-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> Merge "Refactor the garbage collector driver (GarbageCollector::Run)."
601276abdb746b03675ff945745aa936694d3439 20-Mar-2014 Mathieu Chartier <mathieuc@google.com> Fix RecordFree to take signed parameters.

RecordFree can get negative bytes allocated when background
compaction foreground transitions occur. This caused a DCHECK to
fail on debug builds. Also did some refactoring in
PreProcessReferences.

Bug: 13568814
Change-Id: I57543f1c78544a94f1d241459698b736dba8cfa8
ark_sweep.cc
ark_sweep.h
3e41780cb3bcade3b724908e00443a9caf6977ef 20-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> Refactor the garbage collector driver (GarbageCollector::Run).

Bug: 12687968

Change-Id: Ifc9ee86249f7938f51495ea1498cf0f7853a27e8
arbage_collector.cc
arbage_collector.h
ark_sweep.cc
ark_sweep.h
emi_space.h
1ad2784ad9f311ebf9fe0677d33818648f423f9c 20-Mar-2014 Mathieu Chartier <mathieuc@google.com> Add soft reference pre processing.

Soft reference pre-processing does soft reference preservation with
mutators running. After this is done, it does another pass with
mutators paused in the ProcessReference code. This helps lower pauses
since most preserved soft references have their referents recursive
marked outside the pause.

Changed ergonomics to have non sticky collectors always clear the
soft references.

Maps pauses ~10ms -> ~3ms on Nexus 4.

Bug: 13421927

Change-Id: I1370f7bb6934034869aa5afca0c377876267aa8e
ark_sweep.cc
ark_sweep.h
38e68e9978236db87c9008bbe47db80525d2fa16 07-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> Use the card table to speed up the GSS collector.

Scan only dirty cards, as opposed to the whole space, to find
references from the non-moving spaces to the bump pointer spaces at
bump pointer space only collections.

With this change, the Ritz MemAllocTest speeds up by 8-10% on host and
2-3% on N4. The Ritz EvaluateFibonacci speeds up by 8% and its average
pause time is reduced by 43% on N4.

Bug: 11650816
Change-Id: I1eefe75776bc37e24673b301ffa65a25f9bd4cde
mmune_region.cc
ark_sweep.cc
emi_space.cc
emi_space.h
8fa2dad7fe7909c8335101d6c8904ae997cdf29f 13-Mar-2014 Mathieu Chartier <mathieuc@google.com> Refactor reference code into mirror namespace.

Added two new files: mirror/reference.h and mirror/reference-inl.h.

Change-Id: Ibe3ff6379aef7096ff130594535b7f7c0b7dabce
ark_sweep.cc
emi_space.cc
8d562103c3a3452fb15ef4b1c64df767b70507a4 13-Mar-2014 Mathieu Chartier <mathieuc@google.com> Refactor immune region logic into its own file.

Added immune_region.cc/.h in the collector directory. Changed the
functionality to no longer assume spaces are added to immune region
in ascending order.

Change-Id: Id1d643b3849ad2695e8a151dbbb74a5035644472
mmune_region.cc
mmune_region.h
ark_sweep.cc
ark_sweep.h
artial_mark_sweep.cc
emi_space.cc
emi_space.h
d6534315596326f1a65aa2d300144c09205c5122 10-Mar-2014 Mathieu Chartier <mathieuc@google.com> Add timing split for RevokeAllThreadLocalBuffers.

This is part of the pause and should be accounted for.

Change-Id: I3165324de810e8fab02719098977402a18013da1
arbage_collector.cc
arbage_collector.h
719d1a33f6569864f529e5a3fff59e7bca97aad0 06-Mar-2014 Ian Rogers <irogers@google.com> Enable annotalysis on clang ART builds.

Fix clang build errors aswell as restructure locking/mutex code for correct
thread safety analysis support.
Reorder make dependencies so that host builds build first as they should
provide better compilation errors than target.
Remove host's use of -fno-omit-frame-pointer as it has no value with correct
use of CFI, which we should have.

Change-Id: I72cea8da9a3757b1a0b3acb4081feccb7c6cef90
arbage_collector.h
ark_sweep.h
artial_mark_sweep.h
emi_space.cc
ticky_mark_sweep.h
a5f9de040a7a740b7b4979bc8d47f95eb05718ae 01-Mar-2014 Mathieu Chartier <mathieuc@google.com> Defer heap transitions to heap trimming daemon.

This fixes the case where quick back and forth process state changes
would cause a lot of heap transitions.

Change-Id: Ia5be792edcf26079b2aa23b9c115f6b0a9a39a1c
ark_sweep.cc
893263b7d5bc2ca43a91ecb8071867f5134fc60a 04-Mar-2014 Mathieu Chartier <mathieuc@google.com> Avoid marking old class linker and intern table roots during pause.

The new root visiting logic has a concept of a root log which holds
new roots which were added since the start of the GC. This is an
optimization since it lets us only mark these newly added roots
during the pause (or pre-cleaning) since the other roots intern table
and class linker roots were marked concurrently at the start of the
GC.

Before (EvaluateAndApplyChanges):
MarkConcurrentRoots: Sum: 605.193ms
After:
MarkConcurrentRoots: Sum: 271.858ms

This should also reduce pathological GC pauses which used to be able
to happen when the intern table or class linker became "dirty"
during the concurrent GC.

Change-Id: I433fab021f2c339d50c35aaae7161a50a0901dec
ark_sweep.cc
ark_sweep.h
emi_space.cc
b48b9eb6d181a1f52e2e605cf26a21505f1d46ed 01-Mar-2014 Ian Rogers <irogers@google.com> Fix clang to compile and run host tests.

Don't use the computed goto interpreter with clang 3.4 as it causes compilation
to hang.
Avoid inclusion of LLVM_(HOST|DEVICE)_BUILD_MK except for with portable as it
sets clang incompatible cflags.
Most fixes are self-evident, for the quick dex file method inliner the enums
were being used with ostreams, so fix the enums and operator out python script
to allow this.
Note this change effects portable but this is untestable as portable was broken
by ELF file and mc linker changes.

Change-Id: Ia54348f6b1bd3f76d3b71c6e8c5f97626386b903
ark_sweep.cc
15d3402bbf8265eb1165694da2e4117eb128f3bc 27-Feb-2014 Mathieu Chartier <mathieuc@google.com> Add custom SIGSEGV handler to help find heap corruption.

The new signal handler prints heap diagnostics when you get a SIGSEGV.
Added a fault message member in runtime which is modifiable by
Runtime::SetFaultMessage. When you get a SIGSEGV it will print out
whatever is stored in this string as well as the normal information.
This is useful for debugging heap corruption since it lets you see
which threads were in which methods when the last GC occured.

Added some smarter object dumping logic when the faulting address is
in the heap.

Bug: 12934910

Change-Id: Ia72be2c39f70ad711cbd746d66fad2b617d5d29f
emi_space.cc
6fac447555dc94a935b78198479cce645c837b89 26-Feb-2014 Ian Rogers <irogers@google.com> Make allocations report usable size.

Work-in-progress to allow arrays to fill usable size. Bug: 13028925.
Use C++11's override keyword on GCC >= 2.7 to ensure that we override GC and
allocator methods.
Move initial mirror::Class set up into a Functor so that all allocated objects
have non-zero sizes. Use this property to assert that all objects are never
larger than their usable size.
Other bits of GC related clean-up, missing initialization, missing use of
const, hot methods in .cc files, "unimplemented" functions that fail at
runtime in header files, reducing header file includes, move valgrind's space
into its own files, reduce number of array allocation routines.

Change-Id: Id5760041a2d7f94dcaf17ec760f6095ec75dadaa
ark_sweep.h
artial_mark_sweep.h
emi_space.cc
emi_space.h
ticky_mark_sweep.cc
ticky_mark_sweep.h
930f7b843ddc6e6530439d3fdb0e2133a6292f1e 26-Feb-2014 Hiroshi Yamauchi <yamauchi@google.com> Merge "(Experimental) Add Brooks pointers."
eb7bbad163ce7a2c60bba6e0d3d7488e5cfb2c05 26-Feb-2014 Mathieu Chartier <mathieuc@google.com> Remove duplicate card pre-cleaning.

Sticky GC was pre-cleaning cards twice since MarkingPhase calls
MarkReachableObjects.

Change-Id: I61572b79c855bcd02085a1f7ff96dd0089db95fb
ark_sweep.cc
ticky_mark_sweep.cc
9d04a20bde1b1855cefc64aebc1a44e253b1a13b 31-Jan-2014 Hiroshi Yamauchi <yamauchi@google.com> (Experimental) Add Brooks pointers.

This feature is disabled by default.

Verified that the Brooks pointers are installed correctly by using the
CMS/SS collectors.

Change-Id: Ia9be9814ab6e29169ac85edc4792ce8c81d552a9
ark_sweep.cc
emi_space.cc
c22c59ef8513b4cbbfd25073d1afbf58196b522a 25-Feb-2014 Mathieu Chartier <mathieuc@google.com> Remove started runtime check in RevokeAllThreadLocalAllocationStacks

This check occasionally caused some thread local allocation stacks
to incorrectly not get revoked when multiple threads were allocating
without a started runtime. This showed up in image_test with
compaction enabled when we were initializing classes in the compiler
driver.

Change-Id: I7f28d072feea333c2503e35265ba25c51a6308fe
ark_sweep.cc
ark_sweep.h
dda54f59271464b5a72bf4cde6d9010e8dc1f337 24-Feb-2014 Mathieu Chartier <mathieuc@google.com> Card pre-cleaning.

We now pre-clean cards before the pause in the concurrent mark sweep
collectors. This provides substantial a pause time reduction for GC
iterations which have a lot of dirty cards. The only downside is a
slight GC time increase for large heaps.

Benchmark FormulaEvaluationActions.EvaluateAndApplyChanges:

Before:
Partial average pause: 5.47ms
Sticky average pause: 2.91ms
Total GC time: 25.8s

After:
Partial average pause: 1.98ms
Sticky average pause: 1.66ms
Total GC time: 27.0s

Benchmark score difference in the noise.

Change-Id: If9f01f8c1501f122e19432438108d48e723b332e
ark_sweep.cc
ark_sweep.h
ticky_mark_sweep.cc
4e30541a92381fb280cd0be9a1763b713ee4d64c 19-Feb-2014 Mathieu Chartier <mathieuc@google.com> Fix and optimize verify object.

VerifyObject no longer resides in heap. You can now enable
VerifyObject for non-debug builds. VerifyStack is still slow, so it
is now guarded by its own flag.

Fixed the image writer to not use verification at places where
verification fails due to invalid reads.

Fixed RosAlloc to use SizeOf which doesn't call verify object.

Added a flag paremeter to some of the mirror getters / setters to
be able to selectively disable VerifyObject on certain calls.

Optimized the GC to not verify each object multiple times during
object scanning if verify object is enabled.

Added 3 verification options: verify reads, verify this, and verify
writes so that you can select how much verification you want for
mirror getters and setters.

Removed some useless DCHECKs which would slow debug builds without
providing any benefits.

TODO: RosAlloc verification doesn't currently work with verify
objects.

Bug: 12934910
Bug: 12879358

Change-Id: Ic61033104dfc334543f89b0fc0ad8cd4f4015d69
ark_sweep-inl.h
ark_sweep.cc
emi_space.cc
3bb57c7b41bf5419fe895e7aa664d8d430205ba8 18-Feb-2014 Mathieu Chartier <mathieuc@google.com> Change ProcessReferences to not use RecursiveMarkObject.

Calling ProcessMarkStack in RecursiveMarkObject caused a lot of
overhead due to timing logger splits. Changed the logic to be the
same as prior to the reference queue refactoring which involves
calling process mark stack after preserving soft references and
enqueueing finalizer references.

FinalizingGC longest pause is reduced by around 1/2 down to ~300ms.
Benchmark score ~400000 -> ~600000.

Also changed the timing logger splits in the GC to have (Paused) if
the split is a paused part of the GC.

Bug: 12129382

Change-Id: I7476d4f23670b19d70738e2fd48e37ec2f57e9f4
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
815873ecc312b1d231acce71e1a16f42cdaf09f2 14-Feb-2014 Mathieu Chartier <mathieuc@google.com> Change root visitor to use Object**.

Simplifies code and improves the performance of root visiting since
we usually don't need to check to see if the object moved.

Change-Id: Iba998f5a15ae1fa1b53ca5226dd2168a411196cf
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
d2fe10a3a34af171bf1631219cd2d6ff6b7778b5 15-Jan-2014 Sebastien Hertz <shertz@google.com> Remove blacklist

Removes the class initialization blacklist and use transaction to detect and
revert class initialization attempting to invoke native method. This only
concerns class initialization happening at compilation time when generating an
image (like boot.art for the system).

In transactional mode, we log every object's field assignment and array update.
Therefore we're able to abort a transaction to restore values of fields and
array as they were before the transaction starts. We also log changes to the
intern string table so we can restore its state prior to transaction start.

Since transactional mode only happens at compilation time, we don't need to log
all these changes at runtime. In order to reduce the overhead of testing if
transactional mode is on/off, we templatize interfaces of mirror::Object and
mirror::Array, respectively responsible for setting a field and setting an
array element.

For various reasons, we skip some specific fields from transaction:
- Object's class and array's length must remain unchanged so garbage collector
can compute object's size.
- Immutable fields only set during class loading: list of fields, method,
dex caches, vtables, ... as all classes have been loaded and verified before a
transaction occurs.
- Object's monitor for performance reason.

Before generating the image, we browse the heap to collect objects that need to
be written into it. Since the heap may still holds references to unreachable
objects due to aborted transactions, we trigger one collection at the end of
the class preinitialization phase.

Since the transaction is held by the runtime and all compilation threads share
the same runtime, we need to ensure only one compilation thread has exclusive
access to the runtime. To workaround this issue, we force class initialization
phase to run with only one thread. Note this is only done when generating image
so application compilation is not impacted. This issue will be addressed in a
separate CL.

Bug: 9676614
Change-Id: I221910a9183a5ba6c2b99a277f5a5a68bc69b5f9
emi_space.cc
f5b0e20b5b31f5f5465784adcf2a204dcd69c7fd 12-Feb-2014 Hiroshi Yamauchi <yamauchi@google.com> Thread-local allocation stack.

With this change, Ritz MemAllocTest gets ~14% faster on N4.

Bug: 9986565
Change-Id: I2fb7d6f7c5daa63dd4fc73ba739e6ae4ed820617
ark_sweep.cc
emi_space.cc
ad35d904c0b835f7e005c471d2c6b18b84894595 12-Feb-2014 Mathieu Chartier <mathieuc@google.com> Avoid unnecessarily dirtying pages in SemiSpace collector.

Added a special version of memcpy which has handling to avoid dirty
pages for objects which are larger than one page. No non noise GC
performance regression measured on ritzperf benchmark. Seems to save
4k to 148k memory per app on AOSP mako.

Bug: 10802951
Change-Id: I5976c83643e06beebf5c86a65c584bf7c1ab1e2e
emi_space.cc
emi_space.h
83c8ee000d525017ead8753fce6bc1020249b96a 28-Jan-2014 Mathieu Chartier <mathieuc@google.com> Add root types and thread id to root visiting.

Enables us to pass the root type and thread id to hprof.

Bug: 12680863
Change-Id: I6a0f1f9e3aa8f9b4033d695818ae7ca3460d67cb
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
a4adbfd44032d70e166e6f18096bbbed05a990ba 05-Feb-2014 Hiroshi Yamauchi <yamauchi@google.com> RosAlloc verification.

If enabled, RosAlloc verification checks the allocator internal
metadata and invariants to detect bugs, heap corruptions, and race
conditions. Added runtime options for enabling and disabling
it. Enable it for the debug build.

Bug: 9986565
Bug: 12592026
Change-Id: I923742b87805ae839f1549d78d0d492733da6a58
arbage_collector.cc
emi_space.cc
ef7d42fca18c16fbaf103822ad16f23246e2905d 06-Jan-2014 Ian Rogers <irogers@google.com> Object model changes to support 64bit.

Modify mirror objects so that references between them use an ObjectReference
value type rather than an Object* so that functionality to compress larger
references can be captured in the ObjectRefererence implementation.
ObjectReferences are 32bit and all other aspects of object layout remain as
they are currently.

Expand fields in objects holding pointers so they can hold 64bit pointers. Its
expected the size of these will come down by improving where we hold compiler
meta-data.
Stub out x86_64 architecture specific runtime implementation.
Modify OutputStream so that reads and writes are of unsigned quantities.
Make the use of portable or quick code more explicit.
Templatize AtomicInteger to support more than just int32_t as a type.
Add missing, and fix issues relating to, missing annotalysis information on the
mutator lock.
Refactor and share implementations for array copy between System and uses
elsewhere in the runtime.
Fix numerous 64bit build issues.

Change-Id: I1a5694c251a42c9eff71084dfdd4b51fff716822
ark_sweep-inl.h
ark_sweep.h
emi_space.cc
emi_space.h
d1e05bf325fa4f57e8d79155a328313da800cf27 05-Feb-2014 Mathieu Chartier <mathieuc@google.com> Change marking objects in to_space DCHECK to CHECK

Should have no visible performance hit since the check should only
happen for large objects.

Working on pinpointing bug 12897879

Change-Id: I5cacf5275af8367997a595b72d42474a909ff10f
emi_space.cc
1f3b5358b28a83f0929bdd8ce738f06908677fb7 03-Feb-2014 Mathieu Chartier <mathieuc@google.com> Move SwapBitmaps to ContinuousMemMapAllocSpace.

Moved the SwapBitmaps function to ContinuousMemMapAllocSpace since
the zygote space uses this function during full GC.

Fixed a place where we were casting a ZygoteSpace to a MallocSpace,
somehow this didn't cause any issues in non-debug builds.

Moved the CollectGarbage in PreZygoteFork before the lock to prevent
an occasional lock level violation caused by attempting to enqueue
java lang references with the a lock.

Bug: 12876255

Change-Id: I77439e46d5b26b37724bdcee3a0948410f1b0eb4
arbage_collector.cc
ticky_mark_sweep.cc
9aeeeac4ba0b136652f213d60a5a1990a333a629 31-Jan-2014 Mathieu Chartier <mathieuc@google.com> Merge "Add zygote space as its own space type."
a1602f28c0e3127ad511712d4b08db89737ae901 14-Jan-2014 Mathieu Chartier <mathieuc@google.com> Add zygote space as its own space type.

Helps prevent errors caused from doing invalid operations on the
old alloc space.

Removed some duplicated code in mark_sweep.cc and semi_space.cc.

Change-Id: I67a772cab30d698744c918aad581053f282a4a99
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
ticky_mark_sweep.cc
ba5870d1cb6c6320bfd2eb818772352df71a8269 30-Jan-2014 Hiroshi Yamauchi <yamauchi@google.com> Fix a large object space issue with the GSS collector.

Objects in the large object space weren't properly collected because
large objects on the allocation stack weren't copied from the live set
to the marked set.

(Cherrypick commit c7c9fdff117cd53a12aaefb0f5bc9b3245de6a02 from master.)

Bug: 12553120
Bug: 11650816
Change-Id: Ib67394eb6de0520fe215c9e6e6988a7dd59cdd48
emi_space.cc
6f4ffe41649f1e6381e8cda087ad3749206806e5 13-Jan-2014 Hiroshi Yamauchi <yamauchi@google.com> Improve the generational mode.

- Turn the compile-time flags for generational mode into a command
line flag.

- In the generational mode, always collect the whole heap, as opposed
to the bump pointer space only, if a collection is an explicit,
native allocation-triggered or last attempt one.

Change-Id: I7a14a707cc47e6e3aa4a3292db62533409f17563
arbage_collector.cc
arbage_collector.h
emi_space.cc
emi_space.h
05e713a3a009a0825826ce6d494582fcd6dd6a8d 09-Jan-2014 Hiroshi Yamauchi <yamauchi@google.com> Bump pointer space only collection.

Add a mode of collection that collects only the bump pointer spaces,
as opposed to the whole heap. It's disabled behind a flag.

TODO: It still scans the entire non-moving space to look for
inter-space references. Use a card table like technique to limit the
scope of scanning and speed it up.

Bug: 11650816
Change-Id: I56cb11e78e47a578bff644e6e6c63d978cfedf39
emi_space.cc
emi_space.h
db7f37d57b6ac83abe6815d0cd5c50701b6be821 10-Jan-2014 Mathieu Chartier <mathieuc@google.com> Refactor large object sweeping.

Moved basic sweeping logic into large_object_space.cc.
Renamed SpaceSetMap -> ObjectSet.

Change-Id: I938c1f29f69b0682350347da2bd5de021c0e0224
arbage_collector.cc
ark_sweep.cc
emi_space.cc
5dc08a680696554ffc76d58dd591b23e79f5b5a1 10-Jan-2014 Mathieu Chartier <mathieuc@google.com> Fix ART debug build boot.

Some of the checks were no longer valid after the zygote bin packing.

Bug: 12480317
Change-Id: Iaa169b0aafe9fec06e75e9b58a3de28aa70a9421
emi_space.cc
emi_space.h
85a43c055fcdc366293c61df6f65e586d6967841 08-Jan-2014 Mathieu Chartier <mathieuc@google.com> Bin packing the zygote (best fit).

We now use bin packing to fill holes in the non movable space with
objects from the bump pointer space when we create the zygote.

Zygote space PSS reduction on AOSP: ~300k.
Zygote size on AOSP: 2121040 bytes -> 1597944 bytes.

Deleted some unreachable code.

Bug: 11902311

Change-Id: Idc957d69e93b3a941e0c298d47a21b73526dd286
emi_space.cc
emi_space.h
e6da9af8dfe0a3e3fbc2be700554f6478380e7b9 16-Dec-2013 Mathieu Chartier <mathieuc@google.com> Background compaction support.

When the process state changes to a state which does not perceives
jank, we copy from the main free-list backed allocation space to
the bump pointer space and enable the semispace allocator.

When we transition back to foreground, we copy back to a free-list
backed space.

Create a seperate non-moving space which only holds non-movable
objects. This enables us to quickly wipe the current alloc space
(DlMalloc / RosAlloc) when we transition to background.

Added multiple alloc space support to the sticky mark sweep GC.

Added a -XX:BackgroundGC option which lets you specify
which GC to use for background apps. Passing in
-XX:BackgroundGC=SS makes the heap compact the heap for apps which
do not perceive jank.

Results:
Simple background foreground test:
0. Reboot phone, unlock.
1. Open browser, click on home.
2. Open calculator, click on home.
3. Open calendar, click on home.
4. Open camera, click on home.
5. Open clock, click on home.
6. adb shell dumpsys meminfo

PSS Normal ART:
Sample 1:
88468 kB: Dalvik
3188 kB: Dalvik Other
Sample 2:
81125 kB: Dalvik
3080 kB: Dalvik Other

PSS Dalvik:
Total PSS by category:
Sample 1:
81033 kB: Dalvik
27787 kB: Dalvik Other
Sample 2:
81901 kB: Dalvik
28869 kB: Dalvik Other

PSS ART + Background Compaction:
Sample 1:
71014 kB: Dalvik
1412 kB: Dalvik Other
Sample 2:
73859 kB: Dalvik
1400 kB: Dalvik Other

Dalvik other reduction can be explained by less deep allocation
stacks / less live bitmaps / less dirty cards.

TODO improvements: Recycle mem-maps which are unused in the current
state. Not hardcode 64 MB capacity of non movable space (avoid
returning linear alloc nightmares). Figure out ways to deal with low
virtual address memory problems.

Bug: 8981901

Change-Id: Ib235d03f45548ffc08a06b8ae57bf5bada49d6f3
arbage_collector.cc
ark_sweep.cc
emi_space.cc
emi_space.h
ticky_mark_sweep.cc
ec05007f8619f8b0cc868d06731e07f84bb74c5b 08-Jan-2014 Mathieu Chartier <mathieuc@google.com> Refactor sweeping logic into malloc space.

Removes duplicated code in MarkSweep/SemiSpace.

Deleted VerifyImageRoots since it had race conditions and is tested
by pre/post GC heap verification.

Change-Id: I9636359ff6adb3e93d56ce77a3e15299ed23dfd5
ark_sweep.cc
ark_sweep.h
emi_space.cc
b122a4bbed34ab22b4c1541ee25e5cf22f12a926 20-Nov-2013 Ian Rogers <irogers@google.com> Tidy up memory barriers.

Change-Id: I937ea93e6df1835ecfe2d4bb7d84c24fe7fc097b
ark_sweep.cc
emi_space.cc
692fafd9778141fa6ef0048c9569abd7ee0253bf 30-Nov-2013 Mathieu Chartier <mathieuc@google.com> Thread local bump pointer allocator.

Added a thread local allocator to the heap, each thread has three
pointers which specify the thread local buffer: start, cur, and
end. When the remaining space in the thread local buffer isn't large
enough for the allocation, the allocator allocates a new thread
local buffer using the bump pointer allocator.

The bump pointer space had to be modified to accomodate thread
local buffers. These buffers are called "blocks", where a block
is a buffer which contains a set of adjacent objects. Blocks
aren't necessarily full and may have wasted memory towards the
end. Blocks have an 8 byte header which specifies their size and is
required for traversing bump pointer spaces.

Memory usage is in between full bump pointer and ROSAlloc since
madvised memory limits wasted ram to an average of 1/2 page per
block.

Added a runtime option -XX:UseTLAB which specifies whether or
not to use the thread local allocator. Its a NOP if the garbage
collector is not the semispace collector.

TODO: Smarter block accounting to prevent us reading objects until
we either hit the end of the block or GetClass() == null which
signifies that the block isn't 100% full. This would provide a
slight speedup to BumpPointerSpace::Walk.

Timings: -XX:HeapMinFree=4m -XX:HeapMaxFree=8m -Xmx48m
ritzperf memalloc:
Dalvik -Xgc:concurrent: 11678
Dalvik -Xgc:noconcurrent: 6697
-Xgc:MS: 5978
-Xgc:SS: 4271
-Xgc:CMS: 4150
-Xgc:SS -XX:UseTLAB: 3255

Bug: 9986565
Bug: 12042213

Change-Id: Ib7e1d4b199a8199f3b1de94b0a7b6e1730689cad
arbage_collector.cc
emi_space.cc
460536209b741bc469f1b0857775449abb2102fb 12-Dec-2013 Bernhard Rosenkränzer <Bernhard.Rosenkranzer@linaro.org> Don't rely on gcc extensions

Make the code more compatible with different compilers.
clang doesn't allow extra static qualifiers on template
specializations, const qualifiers on function types,
or inline attributes on lambda functions, and is more
picky about casting away constness with reinterpret_cast.

These modifications are compatible with both gcc and
clang.

Change-Id: I739b10df2780bec537827a13679fd2bcc2cc7188
Signed-off-by: Bernhard Rosenkränzer <Bernhard.Rosenkranzer@linaro.org>
ark_sweep.cc
emi_space.cc
8755359a35a4aa915fe3753633015263c7e97b74 11-Dec-2013 Mathieu Chartier <mathieuc@google.com> Merge "Enable moving classes."
c528dba35b5faece51ca658fc008b688f8b690ad 26-Nov-2013 Mathieu Chartier <mathieuc@google.com> Enable moving classes.

Slight reduction in Zygote size, memory savings are in the noise.
Before: Zygote size: 8739224
After: Zygote size: 8733568

Fixed a bug where we didn't set the concurrent start bytes after
switching the allocator from bump pointer to ROSAlloc in the
zygote. This caused excessive memory usage.

Added the method verifiers as roots to fix an issue caused by
RegTypes holding a Class*.

Added logic to clear card table in the SemiSpace collector, this
reduces DalvikOther from ~2400k -> ~1760k when using the SemiSpace
collector.

Added a missing lock to the timing loggers which caused a rare
one time crash in std::set.

Bug: 11771255
Bug: 8499494
Bug: 10802951

Change-Id: I99d2b528cd51c1c5ed7012e3220b3aefded680ae
ark_sweep-inl.h
ark_sweep.h
emi_space.cc
4b1782f1c1af136a18f2886e76ef32216c3db3e4 06-Dec-2013 Hiroshi Yamauchi <yamauchi@google.com> A simple promotion-like mechanism.

If enabled, this mechanism moves objects that survive the first
semi-space copying collection to the non-moving space.

Bug: 11650816
Change-Id: Ia2fc902d08dc983449416f420a39449f9ed96255
emi_space.cc
emi_space.h
b2f9936cab87a187f078187c22d9b29d4a188a62 21-Nov-2013 Mathieu Chartier <mathieuc@google.com> Add histogram for GC pause times.

Printed when you dump the GC performance info.

Bug: 10855285
Change-Id: I3bf7f958305f97c52cb31c03bdd6218c321575b9
arbage_collector.cc
arbage_collector.h
ark_sweep.cc
emi_space.cc
cf58d4adf461eb9b8e84baa8019054c88cd8acc6 26-Sep-2013 Hiroshi Yamauchi <yamauchi@google.com> A custom 'runs-of-slots' memory allocator.

Bug: 9986565
Change-Id: I0eb73b9458752113f519483616536d219d5f798b
arbage_collector.cc
ark_sweep.cc
emi_space.cc
ticky_mark_sweep.cc
39e3261168e7761fea6d873494d7c5d191285791 13-Nov-2013 Mathieu Chartier <mathieuc@google.com> Refactor reference queues.

Refactored the reference queue processing to reside in the heap code.
This removes significant code duplication in the semispace and
marksweep garbage collectors.

Changed the soft reference behaviour to preserve all soft references
unless the GC requires them to be cleared to avoid an out of memory
error. It may be worth investigating a better heuristic in the
future to preserve soft references by LRU order.

Change-Id: I1f3ff5bd4b3c5149271f4bb4fc94ba199e2f9bc2
ark_sweep.cc
ark_sweep.h
emi_space.cc
emi_space.h
5fe9af720048673e62ee29597a30bb9e54c903c5 14-Nov-2013 Ian Rogers <irogers@google.com> Fix memory leaks relating to timing logger.

Bug: 11670287.
We use pointers to uninitialized values for control-flow in the timing logger
code, add TODO comments to clean this up later.
Remove base namespace and other bits of tidying.

Change-Id: I1e6600a1e92f974c8f58f3a405a4e4abb4d9f80f
arbage_collector.h
ark_sweep.cc
emi_space.cc
906457c326d505f511fae42fc693cade1656c19e 14-Nov-2013 Ian Rogers <irogers@google.com> Add missing field initialization in mark sweep.

Change-Id: Ied7deb35c1f1d975ef44c7b425128e0349ee44b0
ark_sweep.cc
590fee9e8972f872301c2d16a575d579ee564bee 13-Sep-2013 Mathieu Chartier <mathieuc@google.com> Compacting collector.

The compacting collector is currently similar to semispace. It works by
copying objects back and forth between two bump pointer spaces. There
are types of objects which are "non-movable" due to current runtime
limitations. These are Classes, Methods, and Fields.

Bump pointer spaces are a new type of continuous alloc space which have
no lock in the allocation code path. When you allocate from these it uses
atomic operations to increase an index. Traversing the objects in the bump
pointer space relies on Object::SizeOf matching the allocated size exactly.

Runtime changes:
JNI::GetArrayElements returns copies objects if you attempt to get the
backing data of a movable array. For GetArrayElementsCritical, we return
direct backing storage for any types of arrays, but temporarily disable
the GC until the critical region is completed.

Added a new runtime call called VisitObjects, this is used in place of
the old pattern which was flushing the allocation stack and walking
the bitmaps.

Changed image writer to be compaction safe and use object monitor word
for forwarding addresses.

Added a bunch of added SIRTs to ClassLinker, MethodLinker, etc..

TODO: Enable switching allocators, compacting on background, etc..

Bug: 8981901

Change-Id: I3c886fd322a6eef2b99388d19a765042ec26ab99
arbage_collector.cc
arbage_collector.h
ark_sweep-inl.h
ark_sweep.cc
ark_sweep.h
artial_mark_sweep.cc
emi_space-inl.h
emi_space.cc
emi_space.h
ticky_mark_sweep.cc
ticky_mark_sweep.h
a3d2718d1fac53210b2a311b1728409d6c8e7b9d 06-Nov-2013 Brian Carlstrom <bdc@google.com> Change thread.h to thread-inl.h to pick up missing Thread::Currnet for debug build in master

Change-Id: I56a4dd18ec1c212f9dbb73b14c0c0623b23c87bd
arbage_collector.cc
artial_mark_sweep.cc
ticky_mark_sweep.cc
5cc16c08ae22e7ba12c81a2f362560a623cbd01d 05-Nov-2013 Hiroshi Yamauchi <yamauchi@google.com> Merge "Fix a DCHECK failure due to unmatching numbers of cards scanned." into dalvik-dev
0941b0423537a6a5d7c1df6dd23e9864ea8f319c 05-Nov-2013 Hiroshi Yamauchi <yamauchi@google.com> Fix a DCHECK failure due to unmatching numbers of cards scanned.

- See the bug for details of the failure.
- After a discussion, we decided to get rid of the DCHECK as a simple
solution would not detect corner failure cases and a full solution
would add undesired complexity, and left a comment that explains
what situation had caused a DCHECK failure.
- Fix a potential error of failing to scan the last card that the end
of the image space falls on as a result of the image end being not
necessarily aligned by the card size.
- Remove dead/unused MarkSweep::ScanRoot().
- Add AlignUp and AlignDown for aligning pointers.

Bug: 11465268
Change-Id: Iee3018a42c48a159feb0e9cf77b1a6b303f5d245
ark_sweep.cc
ark_sweep.h
ad2541a59c00c2c69e8973088891a2b5257c9780 25-Oct-2013 Mathieu Chartier <mathieuc@google.com> Fix object identity hash.

The object identity hash is now stored in the monitor word after
being computed. Hashes are computed by a pseudo random number
generator.

When we write the image, we eagerly compute object hashes to
prevent pages getting dirtied.

Bug: 8981901

Change-Id: Ic8edacbacb0afc7055fd740a52444929f88ed564
ark_sweep.h
11409ae81a3eaf84d7fd2b3c85b8b06d2bae27f0 23-Sep-2013 Mathieu Chartier <mathieuc@google.com> Refactor and improve mod-union tables.

Allow support for adding more mod union tables, reduces the amount
of baked in logic. Adds support for updating mod union table references
from compaction (not for ReferenceCache table yet).

Change-Id: I1beeda00839ed86ef0e853beff5ce10d0ab2b9d1
ark_sweep-inl.h
ark_sweep.cc
ark_sweep.h
ticky_mark_sweep.h
810b1d704f2db0d935bf5dddae3545f79cabd435 20-Sep-2013 Mathieu Chartier <mathieuc@google.com> resolved conflicts for merge of bb0140e0 to dalvik-dev

Change-Id: I31e0c05c5e917cf3275296a7f65bd8b78ec9b78b
72aaacfd30a9d2ad8ffdde038510706d58b0fd68 20-Sep-2013 Mathieu Chartier <mathieuc@google.com> resolved conflicts for merge of 50c3e9ca to dalvik-dev

Change-Id: Icbb5373de3ca5b7a6110b5ab44f83daa05051f02
c11d9b8870de5f860b13c84003ade7b3f3125a52 19-Sep-2013 Mathieu Chartier <mathieuc@google.com> Re-enable concurrent system weak sweeping.

Enabled by disallowing new system weaks during the pause and
re-allowing it after the system weaks have been swept. Reduces
GC pause by ~1ms.

Fixes pause regression caused by fix for
Bug: 10626133

Change-Id: If49d33e7ef19cb728ed3cef5187acfa53b9b05d8
ark_sweep.cc
ark_sweep.h
0f72e4136aecaf6976fdb55916bbd7b6d5c9c77b 07-Sep-2013 Mathieu Chartier <mathieuc@google.com> Improve heap verification.

Re-enabled checking the allocation stack for heap verification.

Added tracking of recent frees in DlMallocSpace if debug spaces is
enabled. This is useful when you have heap corruption caused by a
live object referencing a recently freed object.

Added various other sanity checks in the GC.

Bug: 10626133

Change-Id: I5ada11966336ae9a06615b16f4b933f05b5d0c32
ark_sweep.cc
ark_sweep.h
5712d5d04640925970db9c98938ffaf806b3962c 19-Sep-2013 Mathieu Chartier <mathieuc@google.com> resolved conflicts for merge of f466553d to dalvik-dev

Change-Id: I1205b5c5da3a02e410fff760a8b0dda41cb7249f
9e452d1d097bc0f29a24e13ced5477fa3c9463f9 19-Sep-2013 Mathieu Chartier <mathieuc@google.com> Fix system weak sweeping race.

There was a race related to sweeping the intern table with mutators
unpaused. The race occurred when an unused intern was seen as not
referenced by the GC but another thread attempted to create intern
this same string before we swept the system weaks. This caused the
thread to get a pointer to the stale string which was shortly going
to be reclaimed.

The fix moves sweeping the system weaks inside of the pause. This is
a temporary solution since it adds < 1ms of pause time.

Bug: 10626133

Change-Id: Ibf669ae5237ddb2ab44a9efd72e207bd06b53147
ark_sweep.cc
8aa003fcb06c3473a9e58d2071f2518aa53077f8 18-Sep-2013 Mathieu Chartier <mathieuc@google.com> am b5a80a1e: am ba5c7a7e: am 3c60d137: Merge "Fix soft reference clearing issue." into klp-dev

* commit 'b5a80a1eb89f621950aa9a93c0c6655a24c7cc96':
Fix soft reference clearing issue.
b4ea4de2d6b63a3855968f2748878018a27af106 18-Sep-2013 Mathieu Chartier <mathieuc@google.com> Fix soft reference clearing issue.

There was a bug where we would check that the pending next field was
non null before enqueueing up cleared references. This was causing
references to not get queued up during ProcessReferences.

Bug: 10626133

Change-Id: Ic1e00e42045092280b4abb3d41f1c58f7adbc3de
ark_sweep.cc
6aa3df965395566ed6a4fec4af37c2b7577992e9 18-Sep-2013 Mathieu Chartier <mathieuc@google.com> Refactor system weak sweeping, add support for modification.

Required for moving collectors.

Change-Id: Ib97ba4a05af1139f8d388077a15e62bcb9534855
ark_sweep.cc
ark_sweep.h
423d2a3dcbb260b020efb5da59f784c9f02accbf 13-Sep-2013 Mathieu Chartier <mathieuc@google.com> Add support for changing roots through the root visitor callback.

Needed for copying collectors.

Change-Id: Icc4a342a57e0cfb79587edb02ef8c85e08808877
ark_sweep.cc
ark_sweep.h
3f9667022788ba1effcd1e47fc9e3decc4db569d 05-Sep-2013 Mathieu Chartier <mathieuc@google.com> Add more systrace logging to GC.

There was some confusing systrace messages which made it seem like
pauses were longer than they actually were due to premption occuring
during thread_list->ResumeAll().

Bug: 10612142

Change-Id: I6eeedd1cf85ff38c5b116f15059469db52cbb73b
arbage_collector.cc
ark_sweep.cc
958291c7afe723d846a39539fd00410c102485f3 28-Aug-2013 Mathieu Chartier <mathieuc@google.com> Fix build.

Build broken due to a rare lock checking violation happening during
oat generation. Added a new lock level for the mark stack lock since
it may be acquired with the globals/pins lock.

Change-Id: I944ae0700c66ad050fe3d23aba2f2c377c5df799
ark_sweep.cc
ark_sweep.h
ba311b4385fa3f382f01312a8cc97b52011232e3 27-Aug-2013 Mathieu Chartier <mathieuc@google.com> Fix mark stack expand race.

We now guard parallel mark stack pushing with a lock. This is
only used by checkpoint root marking. I did not observe a
significant slowdown by looking at ritzperf and maps, but it may
be worth reinvestigating in the future.

Also a bit of refactoring.

Bug: 10113123

Change-Id: Ifcb12d14df437e2aea9a1165a9568054f80d91b3
ark_sweep.cc
ark_sweep.h
ee23f85dc66e651a1a220a612d3407689b8de5e8 22-Aug-2013 Mathieu Chartier <mathieuc@google.com> Merge "Add more runtime options." into dalvik-dev
2775ee4f82dff260663ca16adddc0b15327aaa42 21-Aug-2013 Mathieu Chartier <mathieuc@google.com> Add more runtime options.

Changed HeapGCThreads to be split into two different options:
-XX:ParallelGCThreads: Which specifies how many threads the GC may
use when the mutators are suspended.

-XX:ConcGCThreads: Which specifies how many threads the GC may use
when the mutators are running.

Added runtime options to specify long pause / long GC thresholds:
-XX:LongPauseThreshold (default 5ms)
-XX:LongGCThreshold (default 100ms)
These thresholds were previously constants, but are now runtime
options. If we exceed either of the thresholds, we print the GC
message.

Added a new runtime option: -XX:IgnoreMaxFootprint which makes it
that the GC only does GC when the number of bytes allocated hits
the growth limit. This causes GC to occur much less frequently and
can be useful to measure how much of an impact GC has on performance.

Changed the GC behaviour to use only one thread when we do not care
about pauses to prevent jank that can be caused by 2 simultaneous GC
on different processes fighting for CPU time.

Added thread pool functionality for changing the maximum number of
active workers.

Fixed an accounting error where we didn't count large objects in the
total freed.

Bug: 9986416

Change-Id: I86afa358d93dcd3780e18ac5d85bdb1a130cb7e7
ark_sweep.cc
ark_sweep.h
b8a0b94735f188bc739e4c55479c37699006b881 21-Aug-2013 Ian Rogers <irogers@google.com> Switch JNI globals lock to reader-writer.

Also, verifier rejected classes lock. These locks show as contended during
dex2oat, however, they're commonly only read access is required.

Change-Id: If8bb834cc69cd8c26494c87fb7f7b20d4e41cdfd
ark_sweep.cc
e53225c7b8c98f8fc3855fc70f718e7f8abab307 19-Aug-2013 Mathieu Chartier <mathieuc@google.com> Print information about large objects freed and AllocSpace objects.

When we print information about the GC, we now print how many
AllocSpace and LOS objects were freed along with their corresponding
total sizes.

Fixed a disabled timing longer which I had accidentally done in
another CL.

Change-Id: I6cf958ec553bdbb365f242b3844461acec801078
ark_sweep.cc
ark_sweep.h
720ef7680573c1afd12f99f02eee3045daee5168 17-Aug-2013 Mathieu Chartier <mathieuc@google.com> Fix non concurrent GC ergonomics.

If we dont have concurrent GC enabled, we need to force GC for alloc
when we hit the maximum allowed footprint so that our heap doesn't
keep growing until it hits the growth limit.

Refactored a bit of stuff.

Change-Id: I8eceac4ef01e969fd286ebde3a735a09d0a6dfc1
arbage_collector.cc
ark_sweep.cc
94c32c5f01c7d44781317bf23933ed0a5bc4b796 09-Aug-2013 Mathieu Chartier <mathieuc@google.com> More parallel GC, rewritten parallel mark stack processing.

Card scanning may now be done in parallel. This speeds up sticky and
reduces pause times for all GC types.

Speedup on my mako (ritz perf):
Average pause time for sticky GC (~250 samples):
Without parallel cards scanning enabled: 2.524904215ms
Parallel card scanning (num_gc_threads_): 1.552123552ms
Throughput (~250 samples):
Sticky GC throughput with parallel card scanning: 69MB/s
Sticky GC throughput without parallel card scanning: 51MB/s

Rewrote the mark stack processing to be LIFO and use a prefetch queue
like the non parallel version.

Cleaned up some of the logcat printing for the activity manager
process state listening.

Added unlikely hints to object scanning since arrays and classes are
scanned much less often than normal objects.

Fixed a bug where the number of GC threads was clamped to 1 due to a
bool instead of a size_t.

Fixed a race condition when we added references to the reference
queues. Sharded the reference queue lock into one lock for each reference
type (weak, soft, phatom, finalizer).

Changed timing splits to be different for processing gray objects with
and without mutators paused since sticky GC does both.

Mask out the class bit when visiting fields as an optimization, this is
valid since classes are held live by the class linker.

Partially completed: Parallel recursive mark + finger.

Bug: 10245302
Bug: 9969166
Bug: 9986532
Bug: 9961698

Change-Id: I142d09718c4609b7c2387cb28f517a6983c73288
arbage_collector.cc
ark_sweep-inl.h
ark_sweep.cc
ark_sweep.h
ticky_mark_sweep.cc
ticky_mark_sweep.h
02e25119b15a6f619f17db99f5d05124a5807ff3 15-Aug-2013 Mathieu Chartier <mathieuc@google.com> Fix up TODO: c++0x, update cpplint.

Needed to update cpplint to handle const auto.

Fixed a few cpplint errors that were being missed before.

Replaced most of the TODO c++0x with ranged based loops. Loops which
do not have a descriptive container name have a concrete type instead
of auto.

Change-Id: Id7cc0f27030f56057c544e94277300b3f298c9c5
arbage_collector.cc
ark_sweep.cc
ark_sweep.h
artial_mark_sweep.cc
artial_mark_sweep.h
ticky_mark_sweep.cc
ticky_mark_sweep.h
ea46f950e7a51585db293cd7f047de190a482414 30-Jul-2013 Brian Carlstrom <bdc@google.com> Refactor java.lang.reflect implementation

Cherry-picked from commit ed41d5c44299ec5d44b8514f6e17f802f48094d1.

Move to ArtMethod/Field instead of AbstractMethod/Field and have
java.lang.reflect APIs delegate to ArtMethod/ArtField.

Bug: 10014286.

Change-Id: Iafc1d8c5b62562c9af8fb9fd8c5e1d61270536e7
ark_sweep-inl.h
ark_sweep.cc
b22a451675c29ac3fc82a8761d2a385a170d6d7f 14-Aug-2013 Hiroshi Yamauchi <yamauchi@google.com> Avoid a long alloc space lock contention during sticky GC sweep.

The lock contention logging systrace support (b/9986464) indicates
that under the Ritz allocation benchmark the alloc space lock contends
rather long (up to ~10 ms on Nexus 4) in a single lump during each
sticky GC sweep. This is because all the dead objects in the
allocation stack are freed in one FreeList() call. While the lock
contention isn't a GC pause and may only be an issue with a
high-allocation application like Ritz, it'd stall any application
threads that attempt to allocate during the contention and would be
bad for jank. I think it's better to free the objects incrementally in
chunks and interleave the GC thread and application threads.

This change does not speed up or slow down the Ritz allocation
benchmark noticeably.

Bug: 9986464
Change-Id: I24ea805456c0009752991f7d3eb031f6f57145fc
ark_sweep.cc
4654322261c8e4d799acdea60a7e227f33c5c2db 12-Aug-2013 Anwar Ghuloum <anwarg@google.com> Timing logger clean-up

Catching some missing regions in GC, renamed unpause, relacing start/endsplits with scopedsplit.
More to come in further CLs.

Change-Id: I809e0c3af8f8432b22c6205aade5d0dd05ac1b38
ark_sweep.cc
a9a50926963b5093fb851ed966d201f3e95f72d3 10-Aug-2013 Anwar Ghuloum <anwarg@google.com> Fixing build breakage due to bad newsplit placement

Change-Id: Iba8c80d6f893dcf35398fb7e5c00006af18aaaa6
ark_sweep.cc
4446ab9e70dde779d97f451c4904f6b8770232bd 10-Aug-2013 Anwar Ghuloum <anwarg@google.com> Systrace fixups and refactoring for TimingLogger

Fixed systrace bounds for timing logger in GC. Refactored CumulativeLogger a bit to
allow for multiple identical entries. Added ScopedSplit, now composes with explicit
start/end/new splits. Adds some unit tests.
Bug: 10036801

Change-Id: If0afb88b48ec3a1e19462ed354babb274a9517a7
ark_sweep.cc
ticky_mark_sweep.cc
9642c96bd5a1ccc4e221de9c0af4a545af8182d2 06-Aug-2013 Mathieu Chartier <mathieuc@google.com> Move allocation stack unmarking outside of pause.

Reduces pause time by moving the allocation stack unmarking outside
of the pause. This is especially helpful for devices which have longer
GC times since these times result in having more things to be
unmarked in the allocation stack.

Bug: 9969166

Change-Id: I570f2213cbdda9d90545b64538e2cbeb0dc32d16
ark_sweep.cc
ark_sweep.h
184e322fe8ddd75c844a1eb2eb1ca32bc02f2d45 03-Aug-2013 Mathieu Chartier <mathieuc@google.com> Disable and remove finger.

Finger is useless for multithreaded GC, removing it should provide a
slight speed up due to avoided comparison.

Change-Id: I7eb7abcbab8d3307807b1086c06d68b2d4bcd2e9
ark_sweep.cc
ark_sweep.h
ticky_mark_sweep.cc
0f055d11096cb02563e9c040cd03c791fd8f69a3 26-Jul-2013 Brian Carlstrom <bdc@google.com> resolved conflicts for merge of 7934ac28 to dalvik-dev

Change-Id: I8798a6f154463d3f92f6aca62f14130aec82d273
7934ac288acfb2552bb0b06ec1f61e5820d924a4 26-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint whitespace/comments issues

Change-Id: Iae286862c85fb8fd8901eae1204cd6d271d69496
ark_sweep.cc
ark_sweep.h
6f28d91aab952e3244fbb4e707fa38f85538f374 25-Jul-2013 Anwar Ghuloum <anwarg@google.com> Add systrace support to NewTimingLogger, migrate compiler timing logging to NewTimingLogger

Rpleaced old TimingLogger by NewTimingLogger, renamed NewTimingLogger to TimingLogger, added systrace support to TimingLogger.
Tests passing, phone booting, systrace working.

Change-Id: I2aeffb8bcb7f0fd979d8a2a3a8bcfbaa02413679
arbage_collector.h
ark_sweep.cc
f1694e19322ba9a147b25347fe003482a5d4930d 19-Jul-2013 Brian Carlstrom <bdc@google.com> am 751d4ed4: Fixing cpplint legal/copyright issues

* commit '751d4ed43e8118f145edf18db4d987daf7dabbae':
Fixing cpplint legal/copyright issues
58d8b956e7d83bb1b219405be87e135c89b9cdb2 19-Jul-2013 Brian Carlstrom <bdc@google.com> am 02c8cc6d: Fixing cpplint whitespace/blank_line, whitespace/end_of_line, whitespace/labels, whitespace/semicolon issues

* commit '02c8cc6d1312a2b55533f02f6369dc7c94672f90':
Fixing cpplint whitespace/blank_line, whitespace/end_of_line, whitespace/labels, whitespace/semicolon issues
751d4ed43e8118f145edf18db4d987daf7dabbae 19-Jul-2013 Brian Carlstrom <bdc@google.com> Fixing cpplint legal/copyright issues

Change-Id: I9547b53bccb0e67079c5e9c6b26889316739dc64
c_type.cc
02c8cc6d1312a2b55533f02f6369dc7c94672f90 19-Jul-2013 Brian Carlstrom <bdc@google.com> Fixing cpplint whitespace/blank_line, whitespace/end_of_line, whitespace/labels, whitespace/semicolon issues

Change-Id: Ide4f8ea608338b3fed528de7582cfeb2011997b6
ark_sweep.cc
artial_mark_sweep.h
ticky_mark_sweep.h
0ab5ef3e1ab1642fe59fe037f67ce8238da31b35 18-Jul-2013 Brian Carlstrom <bdc@google.com> am 3e3d591f: Fix cpplint build/namespaces issues

* commit '3e3d591f781b771de89f3b989830da2b6ac6fac8':
Fix cpplint build/namespaces issues
3e3d591f781b771de89f3b989830da2b6ac6fac8 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint build/namespaces issues

Change-Id: I19c68703270c1482d6c6aad8cdf97d3d2924360a
ark_sweep.cc
b9070095218595a5d6a37ef874df2794c1761030 18-Jul-2013 Brian Carlstrom <bdc@google.com> resolved conflicts for merge of df62950e to dalvik-dev

Change-Id: I78fbcfc7a2dcbeccb7557ca27302928d7d00debd
08524597899d0bb021c9165218deff51dc88da50 18-Jul-2013 Brian Carlstrom <bdc@google.com> am 0cd7ec2d: Fix cpplint whitespace/blank_line issues

* commit '0cd7ec2dcd8d7ba30bf3ca420b40dac52849876c':
Fix cpplint whitespace/blank_line issues
df62950e7a32031b82360c407d46a37b94188fbb 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint whitespace/parens issues

Change-Id: Ifc678d59a8bed24ffddde5a0e543620b17b0aba9
ark_sweep.cc
0cd7ec2dcd8d7ba30bf3ca420b40dac52849876c 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint whitespace/blank_line issues

Change-Id: Ice937e95e23dd622c17054551d4ae4cebd0ef8a2
arbage_collector.h
b267c89acd0fcda0e8e2a9eeaeeed9ee048d1abc 18-Jul-2013 Brian Carlstrom <bdc@google.com> am 93ba893c: Fix cpplint runtime/explicit issues

* commit '93ba893c20532990a430741e0a97212900094e8c':
Fix cpplint runtime/explicit issues
93ba893c20532990a430741e0a97212900094e8c 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint runtime/explicit issues

Change-Id: I352ba0b427f1ff9b22887693952b180eae0839ba
ark_sweep.cc
42425b3a8befe092199fde69c95833e31db8fec9 18-Jul-2013 Brian Carlstrom <bdc@google.com> am fc0e3219: Fix multiple inclusion guards to match new pathnames

* commit 'fc0e3219edc9a5bf81b166e82fd5db2796eb6a0d':
Fix multiple inclusion guards to match new pathnames
fc0e3219edc9a5bf81b166e82fd5db2796eb6a0d 17-Jul-2013 Brian Carlstrom <bdc@google.com> Fix multiple inclusion guards to match new pathnames

Change-Id: Id7735be1d75bc315733b1773fba45c1deb8ace43
arbage_collector.h
c_type.h
ark_sweep-inl.h
ark_sweep.h
artial_mark_sweep.h
ticky_mark_sweep.h
4b95e8fad803ad307fa09c11c08894544e07a731 16-Jul-2013 Mathieu Chartier <mathieuc@google.com> Refactor atomic integer.

Refactored atomic integer to be similar to c++11 std::atomic<int>.

Removed jdwp serial lock and reverted lock level name change from
https://googleplex-android-review.googlesource.com/#/c/327297/

Change-Id: I2229f30c4d5762a0e8c72697d6aca4683750af35
ark_sweep.cc
4fcb8d335cc856380531a42c8c708cc789a77395 15-Jul-2013 Mathieu Chartier <mathieuc@google.com> Remove stale comments and kDebugLocking check

Bug: 9414652
Change-Id: I9cd450ceb29d7895eb9e0700e2c9acb28409a325
ark_sweep.cc
7940e44f4517de5e2634a7e07d58d0fb26160513 12-Jul-2013 Brian Carlstrom <bdc@google.com> Create separate Android.mk for main build targets

The runtime, compiler, dex2oat, and oatdump now are in seperate trees
to prevent dependency creep. They can now be individually built
without rebuilding the rest of the art projects. dalvikvm and jdwpspy
were already this way. Builds in the art directory should behave as
before, building everything including tests.

Change-Id: Ic6b1151e5ed0f823c3dd301afd2b13eb2d8feb81
arbage_collector.cc
arbage_collector.h
c_type.cc
c_type.h
ark_sweep-inl.h
ark_sweep.cc
ark_sweep.h
artial_mark_sweep.cc
artial_mark_sweep.h
ticky_mark_sweep.cc
ticky_mark_sweep.h