• Home
  • History
  • Annotate
  • only in /art/runtime/gc/accounting/
History log of /art/runtime/gc/accounting/
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
8f4b056427a9d2321e3aa4f21ca8ffb18b3e5ae6 02-Mar-2018 David Sehr <sehr@google.com> Move most of runtime/base to libartbase/base

Enforce the layering that code in runtime/base should not depend on
runtime by separating it into libartbase. Some of the code in
runtime/base depends on the Runtime class, so it cannot be moved yet.
Also, some of the tests depend on CommonRuntimeTest, which itself needs
to be factored (in a subsequent CL).

Bug: 22322814
Test: make -j 50 checkbuild
make -j 50 test-art-host

Change-Id: I8b096c1e2542f829eb456b4b057c71421b77d7e2
Merged-In: c431b9dc4b23cc950eb313695258df5d89f53b22

(cherry picked from commit c431b9dc4b23cc950eb313695258df5d89f53b22)
tomic_stack.h
itmap-inl.h
ard_table-inl.h
ard_table.cc
ard_table_test.cc
emembered_set.h
pace_bitmap-inl.h
67bf42e89592c3a1c648f927f2ce3ccb189a1161 27-Feb-2018 David Sehr <sehr@google.com> Header library to remove dependence on runtime/

Add a new header library to remove libdexfile and others' dependence on
runtime (typically runtime/base) includes in libdexfile. Also a small step
to tease dexlayout and profman away from relying on these as well.

Bug: 22322814
Test: make -j 50 checkbuild
make -j 50 test-art-host-gtest

Change-Id: I38e2fe399a75f4bc6318c77a71954c00ea73ec2b
od_union_table.h
emembered_set.h
8f7ea9ab1703ef52c0c5ca3490e2913ac67f2a16 26-Jan-2018 Roland Levillain <rpl@google.com> Various ART GC documentation updates.

Also some cosmetic and stylistic changes.

Test: mmma art
Change-Id: I411cc45c6b5cb2a4b0652eeb9c4a6f4a3a274bd6
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
pace_bitmap_test.cc
2ae376f5af8953d3524cd8ed915ebdacf505625c 30-Jan-2018 Roland Levillain <rpl@google.com> Stylistic and aesthetic changes.

Test: art/test.py
Change-Id: Ic41aa80430d16af748994c80f049c5b479fd9980
pace_bitmap-inl.h
ef012225dcec8f736fc8fd848f5d8ee01c3c280f 21-Jun-2017 Roland Levillain <rpl@google.com> Fix typos.

Test: art/test/testrunner/testrunner.py
Change-Id: I5f5e052c940650a3adb3a8717b84260e6b306500
tomic_stack.h
9e734c7ab4599d7747a05db0dc73c7b668cb6683 05-Jan-2018 David Sehr <sehr@google.com> Create dex subdirectory

Move all the DexFile related source to a common subdirectory dex/ of
runtime.

Bug: 71361973
Test: make -j 50 test-art-host
Change-Id: I59e984ed660b93e0776556308be3d653722f5223
pace_bitmap.cc
4557b3858a66aa20e42bce937e1f0620aad880a2 03-Jan-2018 Orion Hodson <oth@google.com> ART: Rename Atomic::CompareExchange methods

Renames Atomic::CompareExchange methods to Atomic::CompareAndSet
equivalents. These methods return a boolean and do not get the witness
value. This makes space for Atomic::CompareAndExchange methods in a
later commit that will return a boolean and get the witness value.

This is pre-work for VarHandle accessors which require both forms.

Bug: 65872996
Test: art/test.py --host -j32
Change-Id: I9c691250e5556cbfde7811381b06d2920247f1a1
tomic_stack.h
itmap-inl.h
ard_table-inl.h
pace_bitmap-inl.h
170331f0e44a0e07fcfe0b5932517e0500f5cd1f 08-Dec-2017 Andreas Gampe <agampe@google.com> ART: Remove base/logging from heap-inl.h

Hide the single uncommon VLOG and remove the include. Fix up
transitive includes.

Test: m
Change-Id: I917df597cb62c57040c1fb0e0079df4d95e5a658
od_union_table.cc
57943810cfc789da890d73621741729da5feaaf8 07-Dec-2017 Andreas Gampe <agampe@google.com> ART: Replace base/logging with android-base/logging

Replace wherever possible. ART's base/logging is now mainly VLOG
and initialization code that is unnecessary to pull in and makes
changes to verbose logging more painful than they have to be.

Test: m test-art-host
Change-Id: I3e3a4672ba5b621e57590a526c7d1c8b749e4f6e
tomic_stack.h
itmap-inl.h
ard_table-inl.h
ard_table.cc
eap_bitmap.h
pace_bitmap-inl.h
2ffb703bf431d74326c88266b4ddaf225eb3c6ad 08-Nov-2017 Igor Murashkin <iam@google.com> cpplint: Cleanup errors

Cleanup errors from upstream cpplint in preparation
for moving art's cpplint fork to upstream tip-of-tree cpplint.

Test: cd art && mm
Bug: 68951293
Change-Id: I15faed4594cbcb8399850f8bdee39d42c0c5b956
ard_table.h
ard_table_test.cc
eap_bitmap.h
od_union_table.h
emembered_set.h
pace_bitmap.h
2a1513bdb0a3ecac628cd5620420d66c28413f1e 08-Aug-2017 Mathieu Chartier <mathieuc@google.com> Allow non word aligned heap limit for space bitmap

For the mem map large object space, the bitmap covers the entire
range of memory. This is done by passing in max int as the space
capacity. Previously, the space bitmap heap limit was computed by
multiplying the number of words by the word stride. In 32 bit systems
this caused heap limit to be (1 << 32) == 0 for the mem map large
object space.

This CL allows storing max int as the heap limit and returning that
from the function.

Added regression test to large object test.

Test: verify lyft installs and launches with the profile that was causing problems
Test: test-art-host

Bug: 64413646

(cherry picked from commit 76bb14a18283859b3b6e44c2b71ac441d8c252cf)

Change-Id: I34cabe80420e5bbb880adaea5e1737ca8efd8886
pace_bitmap.cc
pace_bitmap.h
cc55e1dcdd2ec669c635420468b3cc4c99740f0a 28-Jul-2017 Hans Boehm <hboehm@google.com> Don't use fences to implement volatiles

Mixing the fence-based implementation with acquire/release instructions
on ARMv8 is not just ugly but incorrect. A volatile store; volatile
load sequence implemented as a release store followed by ld; dmb
does not prevent reordering.

This should remove the last places we were using fences to implement
volatiles.

The HeapReference representation is changed to be an Atomic,
thereby avoiding many casts. We no longer inherit from ObjectReference,
which was documented to be a value type. HeapReference is not, since
it contains an atomic.

Disentangle HeapReference and ObjectReference/CompressedReference
uses sufficiently to get the code to compile again. They were
previously used somewhat interchangably in a few places, in spite
of the different intended semantics (value-type vs. a concurrently-
updateable field). Further disentanglement might be useful.

Flag a strange fence use I haven't yet understood.

Test: Booted AOSP. Ran default tests. Some object code inspection.

Bug: 31023171

Test: Built AOSP
Change-Id: I7b3c3e624f480994541c8e3a79e585071c122a3d
od_union_table.cc
b5efe0f0a1d655281d87457be95803a6414607c1 25-Jul-2017 Treehugger Robot <treehugger-gerrit@google.com> Merge "ART: Add SpaceBitmap walk order test"
b4bf949bc4511acc54ee9a03e61262949f9fa155 15-Apr-2014 Andreas Gampe <agampe@google.com> ART: Add SpaceBitmap walk order test

Add tests that ensure Walk and VisitMarkedRange traverse
the heap in increasing order as specified. Refactor test
for code reuse.

Test: m test-art-host-gtest-space_bitmap_test
Change-Id: I5c4c13bdda3cf85a75151bafb137f2f11d299b73
pace_bitmap_test.cc
8cf9cb386cd9286d67e879f1ee501ec00d72a4e1 19-Jul-2017 Andreas Gampe <agampe@google.com> ART: Include cleanup

Let clang-format reorder the header includes.

Derived with:

* .clang-format:
BasedOnStyle: Google
IncludeIsMainRegex: '(_test|-inl)?$'

* Steps:
find . -name '*.cc' -o -name '*.h' | xargs sed -i.bak -e 's/^#include/ #include/' ; git commit -a -m 'ART: Include cleanup'
git-clang-format -style=file HEAD^
manual inspection
git commit -a --amend

Test: mmma art
Change-Id: Ia963a8ce3ce5f96b5e78acd587e26908c7a70d02
od_union_table.h
emembered_set.cc
pace_bitmap.cc
0c18338ebae6b3597c882887f8354b64abb5e90f 14-Jul-2017 Andreas Gampe <agampe@google.com> ART: Replace or remove some ObjectCallback Walk functions

Replace with visitor functions in RegionSpace and SpaceBitmap. Remove
old ObjectCallback version in HeapBitmap. Fix up users. Move some
thread-safety annotations.

Move ObjectCallback definition to the only remaining user (ModUnionTable).

Test: m
Change-Id: I10307aeacad0c60d21fbade2081ec040d6a6ac4c
eap_bitmap.cc
eap_bitmap.h
od_union_table.cc
od_union_table.h
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
351c44765279142d15333e2ae02b8a423d195b1b 13-Jul-2017 Andreas Gampe <agampe@google.com> ART: Add templated VisitObjects

Add templated versions of VisitObjects that accept visitors. This
allows to use more modern lambdas.

Test: m
Change-Id: I71a7f59bcae02090b9493bf8b477bb8b6ba649de
eap_bitmap-inl.h
eap_bitmap.h
pace_bitmap-inl.h
pace_bitmap.h
bdb5c8335fcce3c20b568955b14b5899bf9ff9a2 08-Jun-2017 Hans Boehm <hboehm@google.com> Merge "Document and use AtomicStack concurrency properties"
5d08fcc21a14190c4dba3028a3e35d281afdd016 06-Jun-2017 Andreas Gampe <agampe@google.com> ART: More header cleanups

Pull out more dependencies through forward declarations.

Test: m test-art-host
Change-Id: I7d86726928937f788b956ec9eac91532d66d57ae
pace_bitmap.h
57cf00bde719ddc84a6015b107b90a20169e3099 06-Jun-2017 Andreas Gampe <agampe@google.com> ART: Remove old object_callbacks.h includes

Remove unused includes.

Test: m
Change-Id: I70c227d32c6900904e186b975d1f2131b718d93d
itmap.h
eap_bitmap.h
emembered_set.h
f81e6566a4d8f6ee6160ab61e23404cf28eacb7c 02-Jun-2017 Hans Boehm <hboehm@google.com> Document and use AtomicStack concurrency properties

Document the fact that most AtomicStack operations are not atomic,
and document the one case in which they are.

I initially believed, based on its implementation, that PopBackCount
was also intended to be callable concurrently with other PopBackCount
calls. But the SequentiallyConsistent requirement still made
no sense, since there can't safely be any concurrent readers.
On closer inspection, its only client assumes that it has exclusive
access to the stack. Thus this removes the unused, and probably
unusable, thread-safety overhead.

Test: Builds. TreeHugger.

Bug: 62270718
Change-Id: I38e8413fd1851af59a30313f28864496f1d9458b
tomic_stack.h
b486a98aadc95d80548953410cf23edba62259fa 01-Jun-2017 Andreas Gampe <agampe@google.com> ART: Introduce thread-current-inl.h

Factor out Thread::Current() code into its own -inl file to remove
transitive includes.

This requires at the same time correcting mutex.h, i.e., moving
some functions into mutex-inl.h.

Test: m test-art-host
Change-Id: I88f888b604e0897368d9b483edce6ce4332dd9c9
od_union_table.cc
od_union_table_test.cc
d49012909625c3bf87bf51138fe79315ce1b1bdc 31-May-2017 Andreas Gampe <agampe@google.com> ART: Clean up heap headers

Use more forward declarations for accounting structures and spaces.
Factor out structs to reduce header surface. Remove heap include where
unnecessary. Fix up transitive users. Move some debug-only code out
of line.

Test: m test-art-host
Change-Id: I16db4aaa803f39e155ce6e1b0778b7e393dcbb17
ard_table.cc
ard_table.h
3768adec2f819cb506577618fdde158ba659ddd4 02-May-2017 Mathieu Chartier <mathieuc@google.com> Revert "Revert "Add concurrent card graying for immune spaces""

Added an additional checkpoint before the concurrent graying to set
the mark entrypoints to non-null.

Bug: 37876887

Test: test-art-host

This reverts commit c83dd7bfde2171c879efb92a31a363505385ffb9.

Change-Id: I961c4631c9595c52cd7765f61b22b8291fef2088
ard_table.h
c83dd7bfde2171c879efb92a31a363505385ffb9 02-May-2017 Mathieu Chartier <mathieuc@google.com> Revert "Add concurrent card graying for immune spaces"

Bug: 37876887

This reverts commit 88d329a698ba186aeb1f1ef8794355512ada84a9.

Test: mm

Change-Id: I93880fb7cd8c4c27c65777079d48947075f8cb64
ard_table.h
612ff540cd3329935351f05923358cf29b9c9b44 01-May-2017 Mathieu Chartier <mathieuc@google.com> Increase GC card size to 1024

With CC, having a larger GC card size does not commonly affect
pauses.

Results:
Before (128):
System wide card table PSS: 3642.4K
GrayAllNewlyDirtyImmuneObjects avg: 145us

After (1024):
System wide card table PSS: 1477.8K
GrayAllNewlyDirtyImmuneObjects avg: 75us

Test: test-art-host

Bug: 36457259
Change-Id: Icec03664985ef75859f4e99d9f76c3b9988193e6
ard_table.h
a44d67e7ff3c0d3967f4e9a2d9d1d18c5701c7e7 01-May-2017 Mathieu Chartier <mathieuc@google.com> Merge "Add concurrent card graying for immune spaces"
88d329a698ba186aeb1f1ef8794355512ada84a9 27-Apr-2017 Mathieu Chartier <mathieuc@google.com> Add concurrent card graying for immune spaces

We now age the cards and gray the objects before the GC pause. This
is done to reduce how much work is required during the pause and
allows increasing the card size without regressing the GC pause
time.

We rescan the cards in the pause and only process the cards that were
dirtied since the concurrent graying.

Pause time spent graying objects on maps (Pixel). The average is the
per GC metric.

Disabled entrypoint switching for x86 and x86_64. This is to fix a
case where the gray bit is set but the entrypoint is null, resulting
in crashes.

Also reverted to checking "is gc marking" for x86 and x86_64 codegen
to prevent performance regressions.

128 byte cards without the change:
Sum: 1.912ms 99% C.I. 125us-244us Avg: 159.333us Max: 244us

512 byte cards without the change:
Sum: 12.027ms 99% C.I. 0.940ms-1.495ms Avg: 1.202ms Max: 1.495ms

512 byte cards with concurrent graying:
Sum: 1.385ms 99% C.I. 51us-239us Avg: 86.562us Max: 239us

Bug: 36457259
Bug: 12687968
Bug: 31022084

Test: test-art-host

(cherry picked from commit a3856d0d801f066b9b09649b3a17bdbb747f012d)

Change-Id: I7e8f8a5716f96dde827377234f854482452bc9cd
ard_table.h
88307edcd5c35bd5f602b4c6b2464c2d4332753e 26-Apr-2017 Yi Kong <yikong@google.com> Fix up code warning and errors

Discovered by new LLVM rebase.

Clean up, no functionality change.

Test: build
Bug: 37752547
Change-Id: I7df6097706e0772226dcb07611e7d044a6085421
od_union_table.cc
0dfc315a096bf5678cb5182771e3fdcca8ea4433 24-Apr-2017 Andreas Gampe <agampe@google.com> ART: More header cleanup - mem_map.h

Move MemMap maps_ into cc file and move allocator include. Remove
other unused includes. Force clients to import sys/mman.h if they
need it. Try to lower dependencies on ART by using libbase helpers
more.

Test: mmma art
Change-Id: I8f31e5f371789b3d72e12af377181fee2ac9a611
itmap.cc
ead_barrier_table.h
291ce17ada5a126be99f0fc069a028d2100bcf9e 24-Apr-2017 Andreas Gampe <agampe@google.com> ART: More header cleanup - CC

Forward-declare AtomicStack in CC. Use stack_reference in atomic_stack.h.

Test: mmma art
Change-Id: I863ca8d4a8dfd5e83279fd68ea0e1a94c3c5df6d
tomic_stack.h
c6ea7d00ad069a2736f603daa3d8eaa9a1f8ea11 02-Feb-2017 Andreas Gampe <agampe@google.com> ART: Clean up art_method.h

Clean up the header. Fix up other headers including the -inl file,
in an effort to prune the include graph. Fix broken transitive
includes by making includes explicit. Introduce new -inl files
for method handles and reference visiting.

Test: source build/envsetup.sh && lunch aosp_angler-userdebug && mmma art
Test: source build/envsetup.sh && lunch aosp_mips64-userdebug && mmma art
Change-Id: I8f60f1160c2a702fdf3598149dae38f6fa6bc851
od_union_table.cc
emembered_set.cc
6711cd8bc9f9053d653a52676177f8a29c1c36eb 24-Feb-2017 Hiroshi Yamauchi <yamauchi@google.com> Change the region space region size to 256k.

Also add RegionSpace::non_free_region_index_limit_ to avoid the need
to scan the free end of the region table in SetFromSpace(), which
compensates (and more) the pause time increase due to the increasing
number of regions.

Ritz EAAC avg pause time (angler little core / -Xmx512m):
Before 186us
After 436us (without non_free_region_index_limit_)
After 103us

Partially revert aog/327342 and remove the temporary adjustment of
max/min-free. Changing the region size to 256k was enough to avoid the
GCE boot issue (b/34576638), but 154-gc-loop barely fails. Make
154-gc-loop failures less strict.

Allocation performance (angler little core / -Xmx512m)
Ritz EAAC
Before 939.6
After 937.8

BinaryTree
Before 603350
After 620200 (-3%)

Bug: 12687968
Test: test-art-host
Test: GCE boot
Change-Id: I1495ab4ced806e1c4d779d49b56cea618817a0d6
ead_barrier_table.h
057d977aed600843dd4a617dca7098555d79110b 18-Feb-2017 Hiroshi Yamauchi <yamauchi@google.com> Always mark reference referents in transaction mode.

Fix a to-space invariant check failure in EnqueueFinalizerReferences.

Reference processing can be a problem and useless during transaction
because it's not easy to roll back what reference processing does and
there's no daemon threads running (in the unstarted runtime). To avoid
issues, always mark reference referents.

Add a do_atomic_update parameter to MarkHeapReference.

Bug: 35417063
Test: test-art-host with CC/CMS/SS.
Change-Id: If32eba8fca19ef86e5d13f7925d179c8aecb9e27
od_union_table.cc
od_union_table_test.cc
emembered_set.cc
46ee31b67d7ee1bd085fbc240502053caa3cf8fa 14-Dec-2016 Andreas Gampe <agampe@google.com> ART: Move to libbase StringPrintf

Remove ART's StringPrintf implementation. Fix up clients. Add
missing includes where necessary.

Test: m test-art-host
Change-Id: I564038d5868595ac3bb88d641af1000cea940e5a
pace_bitmap.cc
6e6078ab06e16bb4032b363316360f3b76685e30 25-Oct-2016 Mathieu Chartier <mathieuc@google.com> Clear (madvise) card table for CC

Since the cards are not really used, we can madvise them to reduce
RAM. Image and zygote cards are cleared in pause to prevent races
between with mutators.

Pause time goes up by only 1.5us on N6P maps:
(Paused)ClearCards: Sum: 755us 99% C.I. 0.250us-28us Avg: 1.540us Max: 28us

AOSP N6P before (60s after boot system wide CC):
4,194K: .GC
4,172K: .GC
4,110K: .GC

After:
3,253K: .GC
3,205K: .GC
3,245K: .GC

Bug: 12687968

Test: test-art-host

Change-Id: I3194b5b8044c2dca427302c32d9974920fecb289
ard_table-inl.h
ard_table.cc
ard_table.h
od_union_table.cc
od_union_table.h
od_union_table_test.cc
pace_bitmap.cc
7ec38dcb84d61f6172bbb5a303bd5ab7139f7409 08-Oct-2016 Mathieu Chartier <mathieuc@google.com> Persistent bitmap for region space

Fix bug where region space was not safely walkable due to holes
in the unevac regions possibly having dangling class pointers.

No preformance change, RAM overhead 1.1-1.2% .Heap (non LOS).

Test: test-art-host
Test: https://android-review.googlesource.com/#/c/288907/

Bug: 31522820

Change-Id: Ic4f8b7175e117689cb1ce3e28b082cf63f1f7b5a
pace_bitmap.cc
pace_bitmap.h
pace_bitmap_test.cc
709b070044354d9f47641f273edacaeeb0240ab7 13-Oct-2016 David Sehr <sehr@google.com> Remove mirror:: and ArtMethod deps in utils.{h,cc}

The latest chapter in the ongoing saga of attempting to dump a DEX
file without having to start a whole runtime instance. This episode
finds us removing references to ArtMethod/ArtField/mirror.

One aspect of this change that I would like to call out specfically
is that the utils versions of the "Pretty*" functions all were written
to accept nullptr as an argument. I have split these functions up as
follows:
1) an instance method, such as PrettyClass that obviously requires
this != nullptr.
2) a static method, that behaves the same way as the util method, but
calls the instance method if p != nullptr.
This requires using a full class qualifier for the static methods,
which isn't exactly beautiful. I have tried to remove as many cases
as possible where it was clear p != nullptr.

Bug: 22322814
Test: test-art-host
Change-Id: I21adee3614aa697aa580cd1b86b72d9206e1cb24
od_union_table.cc
31e88225b2ef68e7f32f11186acf922c74ddabab 15-Oct-2016 Mathieu Chartier <mathieuc@google.com> Move most mirror:: args to ObjPtr

Fixed possible moving GC bugs in ClinitImageUpdate class.

Bug: 31113334

Test: test-art-host
Change-Id: I0bf6578553d58b944aaa17665f1350bdf5ed15ec
emembered_set.cc
2f8eebb10545dc08d27535a4c86f3ae089087d0d 30-Sep-2016 Mathieu Chartier <mathieuc@google.com> Delete some unused bitmap walking code

No longer required with newer versions of the image writer.

Test: mm -j32

Change-Id: I496bc49c80d84bfa3aada39f8c6e3e4e4dfb15d4
pace_bitmap.cc
pace_bitmap.h
0795f23920ee9aabf28e45c63cd592dcccf00216 28-Sep-2016 Mathieu Chartier <mathieuc@google.com> Clean up ScopedThreadStateChange to use ObjPtr

Also fixed inclusion of -inl.h files in .h files by adding
scoped_object_access-inl.h and scoped_fast_natvie_object_access-inl.h

Changed AddLocalReference / Decode to use ObjPtr.

Changed libartbenchmark to be debug to avoid linkage errors.

Bug: 31113334

Test: test-art-host

Change-Id: I4d2e160483a29d21e1e0e440585ed328b9811483
ard_table_test.cc
3fec9ac0d5af1358d216eb2fdc2000ec0205f3f0 13-Sep-2016 Andreas Gampe <agampe@google.com> ART: Use libbase logging

Move most of our logging infrastructure over to system/core/base.
Retain VLOG.

Using unified Android infrastructure has two main advantages. First,
it reduces the complexity/maintenance burden in ART. Second, it
allows to detach logging for the cases where we do not want or need
a runtime, e.g., dexdump, the disassembler, etc. As a part of the
latter, libbase is also supported for all hosts (including Windows).

From a developer viewpoint, there are minor behavior changes for the
LOG statements (see above), but otherwise usage is the same. Explicit
severity enum items are in the android::base namespace now.

Bug: 31338270
Test: m test-art-host
Change-Id: I5abcb2f45f5b03d49951874c48544f72a283a91b
od_union_table.cc
bdf7f1c3ab65ccb70f62db5ab31dba060632d458 31-Aug-2016 Andreas Gampe <agampe@google.com> ART: SHARED_REQUIRES to REQUIRES_SHARED

This coincides with the actual attribute name and upstream usage.
Preparation for deferring to libbase.

Test: m
Test: m test-art-host
Change-Id: Ia8986b5dfd926ba772bf00b0a35eaf83596d8518
tomic_stack.h
ard_table.h
eap_bitmap.h
od_union_table.cc
od_union_table.h
od_union_table_test.cc
emembered_set.cc
emembered_set.h
pace_bitmap.cc
pace_bitmap.h
c381c36aacf977f7e314e6a91e47b31b04639f62 23-Aug-2016 Mathieu Chartier <mathieuc@google.com> Avoid CAS for marking region space bitmap for baker

Only have the GC thread mark it. This occurs when popping from the
mark stack. The race where an object may be pushed to the mark
stack twice is handled by not scanning if it is already marked.

Also avoid checking is_active when marking from the GC.

EAAC: 1263 -> 1253 (average of 30 runs)
GC time: 7.21s -> 6.83s (average of 18 runs)

Timings on 960 mhz N6P.

Bug: 12687968

Change-Id: I47e98c3e258829d2ba0babd803a219c82a36168c
Test: test-art-host, debug N6P booting with baker CC.
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
962cd7adf3d9d2a1dedf0318056a29e9390f1c38 16-Aug-2016 Mathieu Chartier <mathieuc@google.com> Always mark zygote large objects for CC

Prevent needing to gray holders of zygote large objects.
System wide zygote space PSS after boot:
12644 kB -> 5571 kB for CC.

Also PSS reduction in zygote large objects themselves since their
gray bit would have been set each GC.
Overall LOS savings hard to measure, could be up to 316 * 4KB per
app since there are 316 zygote large objects.

Also clear mod-union tables for image spaces to prevent dirty
image pages if any of the image spaces point to zygote large objects.
System wide .art mmap: 37432 kB -> 34372 kB

System server before (N6P):
LOS shared dirty: 12888 kB
Zygote space shared dirty: 700 kB
Zygote space private dirty: 868 kB
.art private dirty: 1696 kB

After:
LOS shared dirty 13672 kB
Zygote space shared dirty: 1072 kB
Zygote space private dirty: 496 kB
.art private dirty: 1432 kB

Bug: 29516968

Test: test-art-host with baker CC, debug N6P phone booting

Change-Id: Ia37ce2c11217cf56885bd1d1dc084332fcbb7843
od_union_table.cc
od_union_table.h
21328a15e15005815efc843e774ac6974e94d4d8 22-Jul-2016 Mathieu Chartier <mathieuc@google.com> Improve CC handling for immune objects

Currently we reduce ram for immune objects by racing agianst the
mutators to try and finish processing them before the mutators
change many objects to gray. However there is still a window of time
where the mutator can dirty immune pages by changing the lock words
to gray. These pages remain dirty for the lifetime of the app.

This CL changes uses the FlipCallback pause to gray all of the
immune objects that have a dirty card. Once these objects are all
gray we don't to gray any more objects in the immune spaces since
these objects are the only ones that may reference non immune
objects.

Also only scan objects that are gray when scanning immune spaces to
reduce scanning time.

System wide PSS after boot on N9, before:
61668 kB: .art mmap
11249 kB: .Zygote

After:
36013 kB: .art mmap
12251 kB: .Zygote

Results are better than demonstrated since there are more apps
running after.

Maps PSS / Private Dirty, before:
.art mmap 3703 3116
.Zygote 577 480

After:
.art mmap 1655 1092
.Zygote 476 392

System server before:
.art mmap 4453 3956
.Zygote 849 780

After:
.art mmap 2326 1748
.Zygote 640 564

EAAC:
Before:
ScanImmuneSpaces takes 669.434ms GC time
Scores: 718, 761, 753 average 744
GC time: 4.2s, 4.35s, 4.3s average 4.28s

After:
ScanImmuneSpaces takes 138.328ms GC time
Scores: 731, 730, 704 average 722
GC time: 3.92s, 3.83s, 3.85s average 3.87s

Additional GC pause time is 285us on Maps on N9.
TODO: Reduce this pause time.

Test: N9 booting, test-art-host, EAAC all run with CC

Bug: 29516968
Bug: 12687968

Change-Id: I584b10d017547b321f33eb23fb5d64372af6f69c
od_union_table.cc
od_union_table.h
56f9a895b938af96ffa1f5725904afd76fc47c73 07-Apr-2016 Mathieu Chartier <mathieuc@google.com> Fix regression in ModUnionTable RAM usage

We forgot to check if a reference changed before assigning it back,
this caused extra dirty pages in the image.

BusinessCard .art PSS
Before: 1678 KB
After: 991 KB

System wide .art PSS goes down ~20MB on my shamu.

Bug: 27906566

(cherry picked from commit 014885a8561f2415cb19ebde1e4084b88d63b746)

Change-Id: I64d149f10f3ef1ed1cb61810282559b47823014c
od_union_table.cc
32ce2adefb8a3d0eda59a29f5e87c1eb43eef796 04-Mar-2016 Mathieu Chartier <mathieuc@google.com> Add more systracing everywhere

Added to:
JIT
DexFile functions
Oat file manager

Added helper ScopedTrace to prevent errors and reduce excess code.

Bug: 27502458

(cherry picked from commit dabdc0fe183d4684f3cf4d70cb09d318cff81b42)

Change-Id: Ifaeff8913d79eefc797380987d13cc00456266f8
ard_table.cc
a9d82fe8bc6960b565245b920e99107a824ca515 26-Jan-2016 Mathieu Chartier <mathieuc@google.com> Use mutator lock to guard adding and removing heap spaces

Too hard to add a new lock since dlmalloc ArtMoreCore requires
looping through the spaces while holding the allocator lock.

Bug: 22858531
Change-Id: Ieac2136da02c766b6795cd604a58798bee37ef2a
od_union_table_test.cc
fbc31087932a65e036a153afab3049dc5298656a 24-Jan-2016 Mathieu Chartier <mathieuc@google.com> Revert "Revert "Load app images""

This reverts commit 1bc977cf2f8199311a97f2ba9431a184540e3e9c.

Bug: 22858531

Change-Id: Ide00bf3a73a02cba3bb364177204ad1b13f70295
ard_table.h
pace_bitmap-inl.h
1bc977cf2f8199311a97f2ba9431a184540e3e9c 23-Jan-2016 Nicolas Geoffray <ngeoffray@google.com> Revert "Load app images"

Fails when a method is duplicated (see test 097-duplicate-method)

Bug: 22858531

This reverts commit f7fd970244f143b1abb956e29794c446e4d57f46.

Change-Id: Ib30ae5be00cc568e799290be6b3c8f29cbbe4c20
ard_table.h
pace_bitmap-inl.h
f7fd970244f143b1abb956e29794c446e4d57f46 09-Nov-2015 Mathieu Chartier <mathieuc@google.com> Load app images

Support in-place patching of the app image based on boot image
location and app oat location. Only loads for art run test so far
since we do not automatically generate app images for app installs.

N5 maps launch time (~200 runs):
Before: 930ms
After: 878.18ms
After + image class table: 864.57ms

TODO:
Oatdump support.
Store class loaders as class roots in image.

Bug: 22858531

Change-Id: I9cbc645645e62ea2ed1ad8e139e91af7d88514c1
ard_table.h
pace_bitmap-inl.h
dcdc85bbd569f0ee66c331b4219c19304a616214 04-Dec-2015 Jeff Hao <jeffhao@google.com> Dex2oat support for multiple oat file and image file outputs.

Multiple changes to dex2oat and the runtime to support a --multi-image
option. This generates a separate oat file and image file output for
each dex file input.

Change-Id: Ie1d6f0b8afa8aed5790065b8c2eb177990c60129
od_union_table.cc
pace_bitmap-inl.h
073b16c8429d302d5413e8ffc488b03b8f770780 10-Nov-2015 Mathieu Chartier <mathieuc@google.com> Image space cleanup for app images

Removed Heap::GetImageSpace, added Heap::GetBootImageSpace.

Generalized some logic in the class linker for image spaces.

Bug: 22858531

Change-Id: Ib3e12bb061a247e232d3dc93b0d6b35eb3a34d25
od_union_table.cc
0b654bce0f8d7c596d4115848b7e4c4dbeadafd5 29-Oct-2015 Hiroshi Yamauchi <yamauchi@google.com> Merge "Relax some CASes for the CC collector."
1dad3f68b7f5a4a4cb2b281413357adc2309a8fd 23-Oct-2015 Nicolas Geoffray <ngeoffray@google.com> Support garbage collection of JITted code.

Change-Id: I9afc544460ae4fb31149644b6196ac7f5182c784
itmap.cc
fed3e2fd4919b58f8e4f8cbc317ee101f3b9af49 20-Oct-2015 Hiroshi Yamauchi <yamauchi@google.com> Relax some CASes for the CC collector.

That is, removing some unnecessary memory fences.

We can use the relaxed CAS for the mark bitmap and reference field/GC
root updates because only the atomicity of the updated word matters
there.

We can use the release CAS for the read barrier bits in the lock word
because it needs to make sure the reference field updates are visible
when the object changes black from gray (the field update stores won't
be reordered after the CAS.)

The CC collector's Ritz EAAC GC time decreases from 34.7s to
29.1s (-16%) on N5.

Bug: 12687968

Change-Id: If082d5911a25fac695df66263a8f55ce8149b199
pace_bitmap-inl.h
4b8f1ecd3aa5a29ec1463ff88fee9db365f257dc 26-Aug-2015 Roland Levillain <rpl@google.com> Use ATTRIBUTE_UNUSED more.

Use it in lieu of UNUSED(), which had some incorrect uses.

Change-Id: If247dce58b72056f6eea84968e7196f0b5bef4da
emembered_set.cc
c8950822095a4f5e6e87cfe7427d1bbb2a78de43 15-Sep-2015 Mathieu Chartier <mathieuc@google.com> Clean up mod union table

Fixed some formatting and removed unused forward declarations.

Bug: 19534862
Change-Id: Icfd5143e5bb7be0570248fd0f4bbf97125a9a35b
od_union_table-inl.h
od_union_table.cc
od_union_table.h
059ef3ddb2088f926ac452889e0953fdcd646a5e 18-Aug-2015 Mathieu Chartier <mathieuc@google.com> Always visit object class from VisitReferences

We don't want to unload classes which have instances.

Slight increase in CMS GC time from ~6.5s to ~7.3s on
EvaluateAndApplyChanges.

Bug: 22720414
Change-Id: I467ff9c9d55163d2a90b999aef3bdd7b3f648bac
od_union_table.cc
emembered_set.cc
009c34cba875885d9540696f33255a9b355d6e15 18-Aug-2015 Mathieu Chartier <mathieuc@google.com> Merge "Fix and improve reference cache mod-union table"
a52f2f93eda106899e2c521e49c13e4cf9446921 16-Aug-2015 Mathieu Chartier <mathieuc@google.com> Fix and improve reference cache mod-union table

Improvements:
Remove cards that only contain null references, this can save memory
in some cases.

Fixes:
Fix a bug where the mod-union table didn't properly handle class
loaders in the boot image. This was cause by not adding the new
classes as references. The fix is to leave these cards dirty.

Bug: 23203999
Change-Id: Ib1f1f74154df976dd8abaf2430c6dabd4cae2dbe
od_union_table.cc
2a5c4681ba19411c1cb22e9a7ab446dab910af1c 14-Aug-2015 Andreas Gampe <agampe@google.com> ART: Some header cleaning around bit-utils

Try to remove dependencies where they are not necessary.

Change-Id: I5ff35cb17aea369bed3725b1610b50d7eb05b81e
tomic_stack.h
3887c468d731420e929e6ad3acf190d5431e94fc 12-Aug-2015 Roland Levillain <rpl@google.com> Remove unnecessary `explicit` qualifiers on constructors.

Change-Id: Id12e392ad50f66a6e2251a68662b7959315dc567
od_union_table.cc
54d220eb9cc51215d75b9e0fe921b94bebbb3fd6 31-Jul-2015 Mathieu Chartier <mathieuc@google.com> Move ArtFields and ArtMethods to be a length prefixed array

Fixes race conditions between changing method and fields arrays
being seen in the wrong order by the GC.

Bug: 22832610
Change-Id: Ia21d6698f73ba207a6392c3d6b9be2658933073f
pace_bitmap.cc
d82e89ece0887e32e7630400db159ed01284f210 05-Aug-2015 Mathieu Chartier <mathieuc@google.com> Fix mod-union logic for native roots

Bug: 22949217
Change-Id: I9e4631870160d5a9412df1eec2a7b79c65668542
od_union_table.cc
eb837eb7c27e789bc7b05f474be9aa119f2fd99f 30-Jul-2015 Mathieu Chartier <mathieuc@google.com> Clear temporary class arrays before linking the new class

Fixes DCHECK failure from remembered sets where two classes had the
same field array which caused the remembered set to incorrectly
remove a card with a reference to the target space.

Change-Id: If43875616fb750e20667212381bc7e359c4214a5
emembered_set.cc
39089124ceac46a45f17b8261a09b88a9606bb97 28-Jul-2015 Mathieu Chartier <mathieuc@google.com> Add remembered set functionality for native roots

Fixes GSS tests. The issue was that there could be classes which had
native roots in the target space which we need to mark and update.

Change-Id: I33e628d40bcb74f768492f740f8f084077d80edf
emembered_set.cc
da7c650022a974be10e2f00fa07d5109e3d8826f 24-Jul-2015 Mathieu Chartier <mathieuc@google.com> Visit class native roots from VisitReferences

Visit class roots when we call Class::VisitReferences instead of in
the class linker. This makes it easier to implement class unloading
since unmarked classes won't have their roots visited by the class
linker.

Bug: 22181835
Change-Id: I63f31e5ebef7b2a0b764b3ba3cb038b3f561b379
od_union_table.cc
emembered_set.cc
90443477f9a0061581c420775ce3b7eeae7468bc 17-Jul-2015 Mathieu Chartier <mathieuc@google.com> Move to newer clang annotations

Also enable -Wthread-safety-negative.

Changes:
Switch to capabilities and negative capabilities.

Future work:
Use capabilities to implement uninterruptible annotations to work
with AssertNoThreadSuspension.

Bug: 20072211

Change-Id: I42fcbe0300d98a831c89d1eff3ecd5a7e99ebf33
tomic_stack.h
ard_table.h
eap_bitmap.h
od_union_table.cc
od_union_table.h
od_union_table_test.cc
emembered_set.cc
emembered_set.h
pace_bitmap.cc
pace_bitmap.h
14d90579f013b374638b599361970557ed4b3f09 16-Jul-2015 Roland Levillain <rpl@google.com> Use (D)CHECK_ALIGNED more.

Change-Id: I9d740f6a88d01e028d4ddc3e4e62b0a73ea050af
ead_barrier_table.h
pace_bitmap.cc
97509954404d031594b2ecbda607314d169d512e 13-Jul-2015 Mathieu Chartier <mathieuc@google.com> Clean up GC callbacks to be virtual methods

Change-Id: Ia08034a4e5931c4fcb329c3bd3c4b1f301135735
od_union_table.cc
od_union_table.h
od_union_table_test.cc
emembered_set.cc
emembered_set.h
pace_bitmap.h
0b71357fb52be9bb06d35396a3042b4381b01041 17-Jun-2015 Hiroshi Yamauchi <yamauchi@google.com> Thread-local mark stacks for the CC collector.

Thread-local mark stacks are assigned to mutators where they push
references in read barriers to reduce the (CAS) synchronization cost
in a global mark stack/queue.

We step through three mark stack modes (thread-local, shared,
GC-exclusive) and use per-thread flags to disable/enable system weak
accesses (only for the CC collector) instead of the existing global
one to safely perform the marking phase. The reasons are 1)
thread-local mark stacks for mutators need to be revoked using a
checkpoint to avoid races (incorrectly leaving a reference on mark
stacks) when terminating marking, and 2) we can’t use a checkpoint
while system weak accesses are disabled (or a deadlock would
happen). More details are described in the code comments.

Performance improvements in Ritzperf EAAC: a ~2.8% improvement
(13290->12918) in run time and a ~23% improvement (51.6s->39.8s) in
the total GC time on N5.

Bug: 12687968
Change-Id: I5d234d7e48bf115cd773d38bdb62ad24ce9116c7
tomic_stack.h
d39645e22b8db1767cf64dc1200a9e4b2f939ed2 10-Jun-2015 Mathieu Chartier <mathieuc@google.com> Move image intern table into image

Previously we recreated this intern table during runtime startup.
This added 50-100ms of boot time.

Fixed bug where we didn't copy over hashcodes into the image.

Deleted some stale code.

(cherry picked from commit fac3a390a247fe33d4873773d742aad4cc100118)

Bug: 20727525
Bug: 19569780
Change-Id: I08959e9aa2a73cedb52f393033e2ffea3a26e76b
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
e401d146407d61eeb99f8d6176b2ac13c4df1e33 22-Apr-2015 Mathieu Chartier <mathieuc@google.com> Move mirror::ArtMethod to native

Optimizing + quick tests are passing, devices boot.

TODO: Test and fix bugs in mips64.

Saves 16 bytes per most ArtMethod, 7.5MB reduction in system PSS.
Some of the savings are from removal of virtual methods and direct
methods object arrays.

Bug: 19264997
Change-Id: I622469a0cfa0e7082a2119f3d6a9491eb61e3f3d
od_union_table_test.cc
pace_bitmap.cc
80afd02024d20e60b197d3adfbb43cc303cf29e0 19-May-2015 Vladimir Marko <vmarko@google.com> ART: Clean up arm64 kNumberOfXRegisters usage.

Avoid undefined behavior for arm64 stemming from 1u << 32 in
loops with upper bound kNumberOfXRegisters.

Create iterators for enumerating bits in an integer either
from high to low or from low to high and use them for
<arch>Context::FillCalleeSaves() on all architectures.

Refactor runtime/utils.{h,cc} by moving all bit-fiddling
functions to runtime/base/bit_utils.{h,cc} (together with
the new bit iterators) and all time-related functions to
runtime/base/time_utils.{h,cc}. Improve test coverage and
fix some corner cases for the bit-fiddling functions.

Bug: 13925192
Change-Id: I704884dab15b41ecf7a1c47d397ab1c3fc7ee0f7
tomic_stack.h
itmap-inl.h
itmap.cc
ard_table-inl.h
ead_barrier_table.h
pace_bitmap-inl.h
eb2baaf20d9059c0fc38141780ec05bea0486c40 14-May-2015 Hiroshi Yamauchi <yamauchi@google.com> Print field info on mark sweep invalid ref crash.

Bug: 20557050
Change-Id: I3abb7a44e3b7ed3256653824fd1705aac5f7d72c
eap_bitmap.h
3130cdf29eb203be0c38d1107a65d920ec39c106 04-May-2015 Mathieu Chartier <mathieuc@google.com> Add some more DISALLOW_COPY_AND_ASSIGN

May help prevent bugs maybe.

Change-Id: Ie73d469dfcd078492ecb3aa28682b42707221202
itmap.h
ard_table.h
2cebb24bfc3247d3e9be138a3350106737455918 22-Apr-2015 Mathieu Chartier <mathieuc@google.com> Replace NULL with nullptr

Also fixed some lines that were too long, and a few other minor
details.

Change-Id: I6efba5fb6e03eb5d0a300fddb2a75bf8e2f175cb
tomic_stack.h
ard_table.cc
ard_table.h
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap_test.cc
3481ba2c4e4f3aa80d8c6d50a9f85dacb56b508b 13-Apr-2015 Vladimir Marko <vmarko@google.com> ART: Clean up includes.

Reduce dependencies to improve incremental build times.
Break up circular dependency involving class_linker-inl.h.

Change-Id: I4be742c5c2b5cd9855beea86630fd68aab76b0db
itmap.cc
itmap.h
ard_table-inl.h
ard_table.cc
ard_table.h
od_union_table_test.cc
c785344b87221f5e4e6473e5b762e4e61fe65dcf 27-Mar-2015 Mathieu Chartier <mathieuc@google.com> Move ArtField to native

Add linear alloc. Moved ArtField to be native object. Changed image
writer to put ArtFields after the mirror section.

Savings:
2MB on low ram devices
4MB on normal devices

Total PSS measurements before (normal N5, 95s after shell start):
Image size: 7729152 bytes
23112 kB: .NonMoving
23212 kB: .NonMoving
22868 kB: .NonMoving
23072 kB: .NonMoving
22836 kB: .NonMoving
19618 kB: .Zygote
19850 kB: .Zygote
19623 kB: .Zygote
19924 kB: .Zygote
19612 kB: .Zygote
Avg: 42745.4 kB

After:
Image size: 7462912 bytes
17440 kB: .NonMoving
16776 kB: .NonMoving
16804 kB: .NonMoving
17812 kB: .NonMoving
16820 kB: .NonMoving
18788 kB: .Zygote
18856 kB: .Zygote
19064 kB: .Zygote
18841 kB: .Zygote
18629 kB: .Zygote
3499 kB: .LinearAlloc
3408 kB: .LinearAlloc
3424 kB: .LinearAlloc
3600 kB: .LinearAlloc
3436 kB: .LinearAlloc
Avg: 39439.4 kB

No reflection performance changes.

Bug: 19264997
Bug: 17643507

Change-Id: I10c73a37913332080aeb978c7c94713bdfe4fe1c
od_union_table.cc
emembered_set.cc
pace_bitmap.cc
20f85597828194c12be10d3a927999def066555e 19-Mar-2015 Vladimir Marko <vmarko@google.com> Fixed layout for dex caches in boot image.

Define a fixed layout for dex cache arrays (type, method,
string and field arrays) for dex caches in the boot image.
This gives those arrays fixed offsets from the boot image
code and allows PC-relative addressing of their elements.

Use the PC-relative load on arm64 for relevant instructions,
i.e. invoke-static, invoke-direct, const-string,
const-class, check-cast and instance-of. This reduces the
arm64 boot.oat on Nexus 9 by 1.1MiB.

This CL provides the infrastructure and shows on the arm64
the gains that we can achieve by having fixed dex cache
arrays' layout. To fully use this for the boot images, we
need to implement the PC-relative addressing for other
architectures. To achieve similar gains for apps, we need
to move the dex cache arrays to a .bss section of the oat
file. These changes will be implemented in subsequent CLs.

(Also remove some compiler_driver.h dependencies to reduce
incremental build times.)

Change-Id: Ib1859fa4452d01d983fd92ae22b611f45a85d69b
od_union_table_test.cc
4460a84be92b5a94ecfb5c650aef4945ab849c93 09-Mar-2015 Hiroshi Yamauchi <yamauchi@google.com> Rosalloc thread local allocation path without a cas.

Speedup on N4:
MemAllocTest 3044 -> 2396 (~21% reduction)
BinaryTrees 4101 -> 2929 (~26% reduction)

Bug: 9986565
Change-Id: Ia1d1a37b9e001f903c3c056e8ec68fc8c623a78b
od_union_table_test.cc
5c42c29b89286e5efa4a4613132b09051ce5945b 25-Feb-2015 Vladimir Marko <vmarko@google.com> Add support for .bss section in oat files.

Change-Id: I779b80b8139d9afdc28373f8c68edff5df7726ce
tomic_stack.h
itmap.cc
ard_table.cc
ead_barrier_table.h
pace_bitmap.cc
4858a935868162266ead90ef2f7802108711371d 23-Jan-2015 Mathieu Chartier <mathieuc@google.com> Change card cache mod-union table to use bitmaps

Previously used card sets, using bitmaps saves memory and slightly
increases performance.

Added mod union table test.

Performance EvaluateAndApplyChanges (minimal changes):

Before (card cache image mu table):
UpdateAndMarkImageModUnionTable: Avg: 524.320us
ImageModUnionClearCards: Avg: 54.580us
Native PSS: ~67500kB

After (card cache image mu table):
UpdateAndMarkImageModUnionTable: Avg: 515.600us
ImageModUnionClearCards: Avg: 53.780us
Native PSS: ~66014kB

Native PSS was higher before since the mod_union_table->SetCards()
which happens pre zygote fork was allocating a large amount of
std::nodes.

Bug: 11859910

Change-Id: I956b7e51d5572feec1393ffa618b7b7d8c147b28
itmap-inl.h
itmap.cc
itmap.h
od_union_table.cc
od_union_table.h
od_union_table_test.cc
pace_bitmap.h
04a77807a657e86495e7ececf7dc530fa5003c4c 27-Jan-2015 Hiroshi Yamauchi <yamauchi@google.com> Merge "ART: reduce UpdateModUnionTable time for partial GC"
85ed6bdd890c08f50c205d7f0604b5a35603b13e 27-Jan-2015 Hiroshi Yamauchi <yamauchi@google.com> Merge "ART: clear dirty cards of alloc space for MS/CMS partial and full GCs"
4add3b4fa38ec42bb3c71d01cf70bce8e9a9fb4e 15-Jan-2015 Lei Li <lei.l.li@intel.com> ART: clear dirty cards of alloc space for MS/CMS partial and full GCs

For MS/CMS partial and full GCs, we could clear the dirty cards of alloc
space when we process cards as we care about the dirty cards after GC
starts.

Change-Id: I1f9b32b20d75979387bc5d26b0cf9a256dcf20b6
Signed-off-by: Lei Li <lei.l.li@intel.com>
ard_table.cc
ard_table.h
cb535da36915f9d10bec3880b46f1de1f7a69f22 23-Jan-2015 Mathieu Chartier <mathieuc@google.com> Change AtomicStack to use StackReference

Previously used Object*, using StackReference saves memory on 64 bit
devices.

Bug: 12935052
Bug: 17643507
Change-Id: I035878690054eeeb24d655a900b8f26c837703ff
tomic_stack.h
2cd334ae2d4287216523882f0d298cf3901b7ab1 09-Jan-2015 Hiroshi Yamauchi <yamauchi@google.com> More of the concurrent copying collector.

Bug: 12687968
Change-Id: I62f70274d47df6d6cab714df95c518b750ce3105
eap_bitmap-inl.h
eap_bitmap.h
ead_barrier_table.h
3befba443e36251cce81cf49a974b9063b36e669 23-Jan-2015 Lei Li <lei.l.li@intel.com> ART: reduce UpdateModUnionTable time for partial GC

UpdateModUnionTable is to scan objects with dirty cards in the immune
region. The immune region of partial GC are image space and zygote
space. As image space is always immuned for any kind of GC, there is no
need to scan dirty cards of zygote space pointing to image space for
partial GC.

Change-Id: I2e1f61c2b361985c211076a441418797abf324c2
od_union_table.cc
727b294b4091cf3cc2f8137cd654552f477fe46a 15-Jan-2015 Lei Li <lei.l.li@intel.com> ART: clear dirty cards of alloc space in pause phase

Clear the dirty cards of alloc spaces when scanning them in pause phase,
the dirty cards of image or zygote space will not be cleared in order to
track the references to the other spaces.

Change-Id: I519f071d954a589aa33dbce0cdba405f2d2cef71
Signed-off-by: Lei Li <lei.l.li@intel.com>
ard_table-inl.h
ard_table.h
379d09fe3c3feb7c2a2fb5a3623689b5ace7e79b 08-Jan-2015 Mathieu Chartier <mathieuc@google.com> Add clamp growth limit

Clamp growth limit shrinks the space memmaps to the current growth
limit. This reduces virtual memory usage for apps with small heaps.

Bug: 18387825
Bug: 17131630

Change-Id: I4a8fdc335d2c40492e991708adabcc46299efb7d
pace_bitmap.h
956af0f0cb05422e38c1d22cbef309d16b8a1a12 11-Dec-2014 Elliott Hughes <enh@google.com> Remove portable.

Change-Id: I3bf3250fa866fd2265f1b115d52fa5dedc48a7fc
pace_bitmap.cc
277ccbd200ea43590dfc06a93ae184a765327ad0 04-Nov-2014 Andreas Gampe <agampe@google.com> ART: More warnings

Enable -Wno-conversion-null, -Wredundant-decls and -Wshadow in general,
and -Wunused-but-set-parameter for GCC builds.

Change-Id: I81bbdd762213444673c65d85edae594a523836e5
eap_bitmap-inl.h
575e78c41ece0dec969d31f46be563d4eb7ae43b 04-Nov-2014 Andreas Gampe <agampe@google.com> ART: Replace COMPILE_ASSERT with static_assert (runtime)

Replace all occurrences of COMPILE_ASSERT in the runtime tree.

Change-Id: I01e420899c760094fb342cc6cb9e692dd670a0b2
ard_table.cc
6a3c1fcb4ba42ad4d5d142c17a3712a6ddd3866f 31-Oct-2014 Ian Rogers <irogers@google.com> Remove -Wno-unused-parameter and -Wno-sign-promo from base cflags.

Fix associated errors about unused paramenters and implict sign conversions.
For sign conversion this was largely in the area of enums, so add ostream
operators for the effected enums and fix tools/generate-operator-out.py.
Tidy arena allocation code and arena allocated data types, rather than fixing
new and delete operators.
Remove dead code.

Change-Id: I5b433e722d2f75baacfacae4d32aef4a828bfe1b
od_union_table.cc
emembered_set.cc
pace_bitmap_test.cc
2d2621a1463d2f3f03fa73503fa42e43657cdcfc 24-Oct-2014 Mathieu Chartier <mathieuc@google.com> Optimize method linking

Added more inlining, removed imt array allocation and replaced it
with a handle scope. Removed some un-necessary handle scopes.

Added logic to base interface method tables from the superclass so
that we dont need to reconstruct for every interface (large win).

Facebook launch Dalvik KK MR2:
TotalTime: 3165
TotalTime: 3652
TotalTime: 3143
TotalTime: 3298
TotalTime: 3212
TotalTime: 3211

Facebook launch TOT before:
WaitTime: 3702
WaitTime: 3616
WaitTime: 3616
WaitTime: 3687
WaitTime: 3742
WaitTime: 3767

After optimizations:
WaitTime: 2903
WaitTime: 2953
WaitTime: 2918
WaitTime: 2940
WaitTime: 2879
WaitTime: 2792

LinkInterfaceMethods no longer one of the hottest methods, new list:
4.73% art::ClassLinker::LinkVirtualMethods(art::Thread*, art::Handle<art::mirror::Class>)
3.07% art::DexFile::FindClassDef(char const*) const
2.94% art::mirror::Class::FindDeclaredStaticField(art::mirror::DexCache const*, unsigned int)
2.90% art::DexFile::FindStringId(char const*) const

Bug: 18054905
Bug: 16828525

(cherry picked from commit 1fb463e42cf1d67595cff66d19c0f99e3046f4c4)

Change-Id: I27cc70178fd3655fbe5a3178887fcba189d21321
ard_table.h
13735955f39b3b304c37d2b2840663c131262c18 08-Oct-2014 Ian Rogers <irogers@google.com> stdint types all the way!

Change-Id: I4e4ef3a2002fc59ebd9097087f150eaf3f2a7e08
tomic_stack.h
ard_table-inl.h
ard_table.cc
ard_table.h
ard_table_test.cc
od_union_table.cc
od_union_table.h
emembered_set.cc
emembered_set.h
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
pace_bitmap_test.cc
bad0267eaab9d6a522d05469ff90501deefdb88b 25-Aug-2014 Mathieu Chartier <mathieuc@google.com> Add native memory accounting through custom allocator.

Added a custom allocator that lets you pass in a special tag which
specifices where the allocation came from. This is used when
dumping. The performance overhead is low since each allocation only
does a atomic add/sub for each allocation/free.

The measurements are dumped to traces.txt during SIGQUIT.

Example output:
I/art (27274): AllocatorTagHeap active=120 max=120 total=168
I/art (27274): AllocatorTagMonitorList active=1572 max=6240 total=11724
I/art (27274): AllocatorTagClassTable active=185208 max=185208 total=268608
I/art (27274): AllocatorTagInternTable active=430368 max=430368 total=436080
I/art (27274): AllocatorTagMaps active=5616 max=6168 total=34392
I/art (27274): AllocatorTagLOS active=1024 max=1536 total=2044
I/art (27274): AllocatorTagSafeMap active=0 max=51936 total=533688
I/art (27274): AllocatorTagLOSMaps active=144 max=1248 total=5760
I/art (27274): AllocatorTagReferenceTable active=10944 max=11840 total=19136
I/art (27274): AllocatorTagHeapBitmap active=32 max=40 total=56
I/art (27274): AllocatorTagHeapBitmapLOS active=8 max=8 total=8
I/art (27274): AllocatorTagVerifier active=0 max=18844 total=1073156
I/art (27274): AllocatorTagModUnionCardSet active=5300 max=5920 total=56020
I/art (27274): AllocatorTagModUnionReferenceArray active=24864 max=24864 total=24864
I/art (27274): AllocatorTagJNILibrarires active=320 max=320 total=320
I/art (27274): AllocatorTagOatFile active=1400 max=1400 total=5852

Change-Id: Ibb470ef2e9c9a24563bb46422d46a55799704d82

(cherry picked from commit 5369c40f75fdcb1be7a7c06db212ce965c83a164)
c_allocator.cc
c_allocator.h
eap_bitmap.h
od_union_table.h
emembered_set.h
pace_bitmap.h
a5a3ac9e0fe219f4ffa9d55dc600f766253bedc0 21-Aug-2014 Mathieu Chartier <mathieuc@google.com> Fix GC memory overhead accounting.

There was some missing null checks.

Bug: 16238192

(cherry picked from commit 2e290fb35ba1959e5a0ac85e87591ab9623808c1)

Change-Id: I4220272ac9c194e30fc307fca9918a4bb725e261
c_allocator.cc
c_allocator.h
e4cab17634dc6d5809b40f2ed80d6459734ae2fe 20-Aug-2014 Mathieu Chartier <mathieuc@google.com> Enable large object space for zygote.

We now enable the large object space before the zygote fork.
This reduces the size of the zygote and removes the need for
excessive explicit GCs during phone booting.

Changed the card set mod union table to support forgetting cards.
If a card has no non null references which are in another space
then it is removed from the set.

Added logging of the zygote size when you do a SIGQUIT.

Dalvik PSS is the same or slightly lower (1-3%).

Zygote space size:
Before: 15MB
After: 8MB (+ some large objects).

TODO: Combine remembered sets and mod union tables into a single
interface.

Bug: 16398684
Change-Id: Ie48cdf35004a0a37eedb1ccc1bf214b1fa9e0cca
od_union_table.cc
od_union_table.h
d35326fc45ef71d88b82fd5fe6875ae07f3646cb 19-Aug-2014 Mathieu Chartier <mathieuc@google.com> Fix heap corruption and verification.

There was a faulty assumption that space End() was always aligned to
card boundaries. This was true for all spaces other than the image
and resulted in heap corruption when the last object of the image
space contained a reference to an object in another space.

Also fixed an error where we called the pre GC verification post GC.

Bug: 17080623

(cherry picked from commit 6724d8649ab73e4fb86c8014bda51b13bddf2f3f)

Change-Id: I931149421c738653863eb6105dfdf7ecd750601a
ard_table-inl.h
47e2e9592da2ebadd167ea8e588f95c179c47861 07-Aug-2014 Mathieu Chartier <mathieuc@google.com> Clean up card table test.

Change-Id: Idd313dbf8a238b014741d48be091375646dccd6d
ard_table_test.cc
2c26501d24d929abe096ecce44f91410290b33c0 06-Aug-2014 Mathieu Chartier <mathieuc@google.com> Add card table test.

Tests some of the functionality supported by the card table.

Removed some logcat spam from monitor_pool.

Change-Id: I1423816a72572f78aca44552effa2b4c6aac46c8
ard_table.cc
ard_table.h
ard_table_test.cc
aa3c3e5ee83c061e7f387c75b1b29c9f248ac39c 06-Aug-2014 Mathieu Chartier <mathieuc@google.com> Fix byte_cas to use uintptr_t Atomic.

Previously we used int32_t Atomic with a uintptr_t bit shift.
This was a mismatch on 64 bit and resulted in occasionally
having the cas return without having succeeded.

Bug: 16819816

Change-Id: I10fa9c2f09ae8fe0997a4c9293fa123c6bde5920
ard_table-inl.h
479f131d4bd3829dd512312020808b05f5a591f1 16-Jul-2014 Mathieu Chartier <mathieuc@google.com> Merge "Disable adding main and non moving spaces to immune region in GSS"
4c13a3ff475f206c4d0a86ee2595c45392fd942f 14-Jul-2014 Mathieu Chartier <mathieuc@google.com> Disable adding main and non moving spaces to immune region in GSS

Disabled adding the main and non moving space to the immune region.
This will enable us to recycle bump pointer spaces for malloc space
-> malloc space compaction as well as collector transitions.

Also added logic for falling back to the non moving space, we may
copy objects there.

Refactored mod union table logic into MarkReachableObjects.

No measurable performance benefit or regression.

Bug: 14059466
Bug: 16291259

Change-Id: If663d9fdbde943b988173b7f6ac844e5f78a0327
od_union_table.cc
22d5e735f403c57525fe868304c7123f0ce66399 16-Jul-2014 Ian Rogers <irogers@google.com> Remove object_utils.h.

Break into object_lock, field_helper and method_helper.
Clean up header files following this.
Also tidy some of the Handle code in response to compiler errors when resolving
the changes in this CL.

Change-Id: I73e63015a0f02a754d0866bfaf58208aebcaa295
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
fd22d5bada15d95b5ea8ab5a4dda39077e1a54ee 14-Jul-2014 Mathieu Chartier <mathieuc@google.com> Fix infinite loop when calling SetStatus after OOM.

There was a problem where we would call SetStatus when we had an OOM
error. This results in attempting to find the ExceptionInInitializer
class which if not loaded does more allocations resulting in an
infinite loop.

Also some cleanup addressing other comments.

Bug: 16082350
Change-Id: I5c1e638a03ddf700ab4e9cad9a3077d2b1b26c43
ard_table.cc
f37a88b8e6db6c587fa449a12e40cb46be1689fc 10-Jul-2014 Zuo Wang <zuo.wang@intel.com> ART: Compacting ROS/DlMalloc spaces with semispace copy GC

Current semispace copy GC is mainly associated with bump pointer
spaces. Though it squeezes fragmentation most aggressively, an extra
copy is required to re-establish the data in the ROS/DlMalloc space to allow
CMS GCs to happen afterwards. As semispace copy GC is still stop-the-world,
this not only introduces unnecessary overheads but also longer response time.
Response time indicates the time duration between the start of transition
request and the start of transition animation, which may impact the user
experience.

Using semispace copy GC to compact the data in a ROS space to another ROS(or
DlMalloc space to another DlMalloc) space solves this problem. Although it
squeezes less fragmentation, CMS GCs can run immediately after the compaction.

We apply this algorithm in two cases:
1) Right before throwing an OOM if -XX:EnableHSpaceCompactForOOM is passed in
as true.
2) When app is switched to background if the -XX:BackgroundGC option has value
HSpaceCompact.

For case 1), OOMs are significantly delayed in the harmony GC stress test,
with compaction ratio up to 0.87. For case 2), compaction ratio around 0.5 is
observed in both built-in SMS and browser. Similar results have been obtained
on other apps as well.

Change-Id: Iad9eabc6d046659fda3535ae20f21bc31f89ded3
Signed-off-by: Wang, Zuo <zuo.wang@intel.com>
Signed-off-by: Chang, Yang <yang.chang@intel.com>
Signed-off-by: Lei Li <lei.l.li@intel.com>
Signed-off-by: Lin Zang <lin.zang@intel.com>
ard_table-inl.h
be2a1df15a31a5223ee9af3015a00c31d2ad2e10 10-Jul-2014 Ian Rogers <irogers@google.com> Fix GC to use art::Atomic rather than compiler intrinsics.

Changes to SpaceBitmap::AtomicTestAndSet and Space::end_. Space::end_ is made
atomic rather than volatile to fully capture all its uses multi-threaded or not
uses.

Change-Id: I3058964b8ad90a8c253b3d7f75585f63ca2fb5e3
pace_bitmap-inl.h
pace_bitmap.h
1461144e95f7798c33cfbada49b288f0cdf28157 09-Jul-2014 Ian Rogers <irogers@google.com> Move card table away from android_atomic_cas.

For x86 use byte CAS operations for byte CAS.
Fix bug in ModifyCardsAtomic where CAS was 32-bit instead of 64-bit.

Change-Id: Ieb3fe695b4699750abf04642b0abe94103976817
ard_table-inl.h
52e4b43d62896b56f8c2bd041e528472bb4a0d8d 10-Jun-2014 Mathieu Chartier <mathieuc@google.com> Add mark compact collector.

The mark compact collector is a 4 phase collection, doing a normal
full mark_sweep, calculating forwarding addresses of objects in the
from space, updating references of objects in the from space, and
moving the objects in the from space.

Support is diabled by default since it needs to have non movable
classes and field arrays. Performance numbers is around 50% as fast.

The main advantage that this has over semispace is that the worst
case memory usage is 50% since we only need one space isntead of two.

TODO: Make field arrays and classes movable. This causes complication
since Object::VisitReferences relies on these, so if we update the
fields of an object but another future object uses this object to
figure out what fields are reference fields it doesn't work.

Bug: 14059466

Change-Id: I661ed3b71ad4dde124ef80312c95696b4a5665a1
pace_bitmap.cc
pace_bitmap.h
c5f17732d8144491c642776b6b48c85dfadf4b52 06-Jun-2014 Ian Rogers <irogers@google.com> Remove deprecated WITH_HOST_DALVIK.

Bug: 13751317
Fix the Mac build:
- disable x86 selector removal that causes OS/X 10.9 kernel panics,
- madvise don't need does zero memory on the Mac, factor into MemMap
routine,
- switch to the elf.h in elfutils to avoid Linux kernel dependencies,
- we can't rely on exclusive_owner_ being available from other pthread
libraries so maintain our own when futexes aren't available (we
can't rely on the OS/X 10.8 hack any more),
- fix symbol naming in assembly code,
- work around C library differences,
- disable backtrace in DumpNativeStack to avoid a broken libbacktrace
dependency,
- disable main thread signal handling logic,
- align the stack in stub_test,
- use $(HOST_SHLIB_SUFFIX) rather than .so in host make file variables.

Not all host tests are passing on the Mac with this change. dex2oat
works as does running HelloWorld.
Change-Id: I5a232aedfb2028524d49daa6397a8e60f3ee40d3
tomic_stack.h
ard_table.cc
pace_bitmap.cc
4479ba35389b03ccc9eabd17fba6168f9505517a 07-Jun-2014 Mathieu Chartier <mathieuc@google.com> Merge "Change FieldHelper to use a handle."
576ca0cd692c0b6ae70e776de91015b8ff000a08 07-Jun-2014 Ian Rogers <irogers@google.com> Reduce header files including header files.

Main focus is getting heap.h out of runtime.h.

Change-Id: I8d13dce8512816db2820a27b24f5866cc871a04b
pace_bitmap.cc
pace_bitmap.h
61c5ebc6aee2cac1c363de6fbdac25ada1697fdb 06-Jun-2014 Mathieu Chartier <mathieuc@google.com> Change FieldHelper to use a handle.

Fixed compaction bugs related to FieldHelper::GetType in:
artSet32InstanceFromCode
SetFieldValueImpl
CheckReceiver
Field_set
interpreter::DoFieldPut
MethodVerifier::VerifyISGet
MethodVerifier::VerifyISPut
MethodVerifier::VerifyIGetQuick

Bug: 13077697

Change-Id: I7de9ded2893b5568d43e4daa86fd135bf5508b72
pace_bitmap.cc
262a0a36ee5f553880365fecf0be1a188526ca69 05-Jun-2014 Andreas Gampe <agampe@google.com> ART: Heap-allocate buffer in space_bitmap when on 64b arch

The stack frame size will be above our limit if we stack-allocate
a buffer.

Bug: 15278350
Change-Id: Ie1a31370538410d47c177f0a6e51879a1fc0752c
pace_bitmap.cc
c179016fe188bef09487e777aa0fd861f5cdf067 23-May-2014 Mathieu Chartier <mathieuc@google.com> Add reserve area to allocation stacks.

This fixes an issue with heap verification which was caused when
the allocation stack overflowed. This resulted in heap verification
failures since we were storing the newly allocated object in a
handle scope without having it be live either in the live bitmap
or allocation stack. We now push the object in the reserve area
before we do a GC due to allocation stack overflow.

Change-Id: I83b42c4b3250d7eaab1b49e53066e21c8656a740
tomic_stack.h
3e5cf305db800b2989ad57b7cde8fb3cc9fa1b9e 21-May-2014 Ian Rogers <irogers@google.com> Begin migration of art::Atomic to std::atomic.

Change-Id: I4858d9cbed95e5ca560956b9dabd976cebe68333
tomic_stack.h
700a402244a1a423da4f3ba8032459f4b65fa18f 20-May-2014 Ian Rogers <irogers@google.com> Now we have a proper C++ library, use std::unique_ptr.

Also remove the Android.libcxx.mk and other bits of stlport compatibility
mechanics.

Change-Id: Icdf7188ba3c79cdf5617672c1cfd0a68ae596a61
tomic_stack.h
ard_table.cc
ard_table.h
c_allocator.h
od_union_table.cc
od_union_table.h
emembered_set.cc
emembered_set.h
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
pace_bitmap_test.cc
507dfdd147c97bfbadebfd63584d094b6a4e7b47 16-May-2014 Ian Rogers <irogers@google.com> Compatibility layer to transition from UniquePtr to std::unique_ptr.

Use ART_WITH_STLPORT (enabled for the target) to cause the use of UniquePtr,
for the host switch to std::unique_ptr. For now the type remains called
UniquePtr.
Make dalvik compile with clang on the host, move its build to C++11.

Change-Id: I5ba8d2757904bc089ed62047ea03de3c0853fb12
tomic_stack.h
ard_table.h
od_union_table.cc
emembered_set.cc
pace_bitmap-inl.h
pace_bitmap.h
pace_bitmap_test.cc
8194963098247be6bca9cc4a54dbfa65c73e8ccc 02-May-2014 Vladimir Marko <vmarko@google.com> Replace CountOneBits and __builtin_popcount with POPCOUNT.

Clean up utils.h, make some functions constexpr.

Change-Id: I2399100280cbce81c3c4f5765f0680c1ddcb5883
c_allocator.h
b0fa5dc7769c1e054032f39de0a3f6d6dd06f8cf 29-Apr-2014 Ian Rogers <irogers@google.com> Force inlining on trivial accessors.

Make volatility for GetFieldObject a template parameter.
Move some trivial mirror::String routines to a -inl.h.

Bug: 14285442

Change-Id: Ie23b11d4f18cb15a62c3bbb42837a8aaf6b68f92
od_union_table.cc
6f365cc033654a5a3b45eaa1379d4b5f156b0cee 23-Apr-2014 Mathieu Chartier <mathieuc@google.com> Enable concurrent sweeping for non-concurrent GC.

Refactored the GarbageCollector to let all of the phases be run by
the collector's RunPhases virtual method. This lets the GC decide
which phases should be concurrent and reduces how much baked in GC
logic resides in GarbageCollector.

Enabled concurrent sweeping in the semi space and non concurrent
mark sweep GCs. Changed the semi-space collector to have a swap semi
spaces boolean which can be changed with a setter.

Fixed tests to pass with GSS collector, there was an error related to
the large object space limit.

Before (EvaluateAndApplyChanges):
GSS paused GC time 7.81s/7.81s, score: 3920

After (EvaluateAndApplyChanges):
GSS paused GC time 6.94s/7.71s, score: 3900

Benchmark score doesn't go up since the GC happens in the allocating
thread. There is a slight reduction in pause times experienced by
other threads (0.8s total).

Added options for pre sweeping GC heap verification and pre sweeping
rosalloc verification.

Bug: 14226004
Bug: 14250892
Bug: 14386356

Change-Id: Ib557d0590c1ed82a639d0f0281ba67cf8cae938c
pace_bitmap-inl.h
pace_bitmap.h
4db7449c0065971ec3a64ca04aeb64cfd2e802f0 23-Apr-2014 Hiroshi Yamauchi <yamauchi@google.com> Improve GSS reference processing.

Support the case where the reference object is in the free list space
and the referent object is in the bump pointer space at a bump pointer
space collection.

Bug: 11650816
Change-Id: If98b08edc9e37351c74ee07cb3f2d30c2b4d0056
emembered_set.cc
emembered_set.h
73d1e17b3afc7d5e56184f90bf819dc64956448a 12-Apr-2014 Mathieu Chartier <mathieuc@google.com> Enable reading page map without lock in RosAlloc::BulkFree

Enabling this flag greatly reduces how much time was spent in the GC.
It was not done previously since it was regressing MemAllocTest. With
these RosAlloc changes, the benchmark score no longer regresses after
we enable the flag.

Changed Run::AllocSlot to only have one mode of allocation. The new
mode is finding the first free bit in the bitmap. This was
previously the slow path but is now the fast path. Some optimizations
which enabled this include always having the alloc bitmap bits which
correspond to invalid slots be set to 1. This prevents us from needing
a bound check since we will never end up allocating there.

Changed revoking thread local buffer to point to an invalid run. The
invalid run is just a run which always has all the allocation bits set
to 1. When a thread attempts to do a thread local allocation from here
it will always fail and go slow path. This eliminates the need for a
null check for revoked runs.

Changed zeroing of memory to happen during free, AllocPages should
always return zeroed memory. Added prefetching which happens when we
allocate a run.

Some refactoring to reduce duplicated code.

Ergonomics changes: Changed kStickyGcThroughputAdjustment to 1.0,
this helps reduce GC time.

Measurements (3 samples per benchmark):
Before: MemAllocTest scores: 3463, 3445, 3431
EvaluateAndApplyChanges score | total GC time
Iter 1: 3485, 23.602436s
Iter 2: 3434, 22.499882s
Iter 3: 3483, 23.253274s

After: MemAllocTest scores: 3495, 3417, 3409
EvaluateAndApplyChanges score | total GC time:
Iter 1: 3375, 17.463462s
Iter 2: 3358, 16.185188s
Iter 3: 3367, 15.822312s

Bug: 8788501
Bug: 11790317
Bug: 9986565
Change-Id: Ifd273a054824028dabed27c07c081dde1816f93c
pace_bitmap.cc
pace_bitmap.h
bbd695c71e0bf518f582e84524e1cdeb3de3896c 16-Apr-2014 Mathieu Chartier <mathieuc@google.com> Replace ObjectSet with LargeObjectBitmap.

Speeds up large object marking since large objects no longer required
a lock. Changed the GCs to use the heap bitmap for marking objects
which aren't in the fast path. This eliminates the need for a
MarkLargeObject function.

Maps before (10 GC iterations):
Mean partial time: 180ms
Mean sticky time: 151ms

Maps after:
Mean partial time: 161ms
Mean sticky time: 101ms

Note: the GC durations are long due to recent ergonomic changes and
because the fast bulk free hasn't yet been enabled. Over 50% of the
GC time is spent in RosAllocSpace::FreeList.

Bug: 13571028

Change-Id: Id8f94718aeaa13052672ccbae1e8edf77d653f62
eap_bitmap-inl.h
eap_bitmap.cc
eap_bitmap.h
pace_bitmap.cc
pace_bitmap.h
pace_bitmap_test.cc
e9ea70bb806f7c1dcd57efb6f48f1d6329d5f103 15-Apr-2014 Mathieu Chartier <mathieuc@google.com> Fix clang build.

Change-Id: Iaf0b76cfc02bac5f49ab98d494db9a774211f453
pace_bitmap-inl.h
pace_bitmap.h
a8e8f9c0a8e259a807d7b99a148d14104c24209d 09-Apr-2014 Mathieu Chartier <mathieuc@google.com> Refactor space bitmap to support different alignments.

Required for:
Using space bitmaps instead of std::set in mod union table +
remembered set.
Using a bitmap instead of set for large object marking.

Bug: 13571028

Change-Id: Id024e9563d4ca4278f79607cdb2f81895121b113
ard_table-inl.h
ard_table.h
eap_bitmap-inl.h
eap_bitmap.cc
eap_bitmap.h
od_union_table.cc
od_union_table.h
emembered_set.cc
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
pace_bitmap_test.cc
be73e57308680382efd1e60fa03ac1eb5abcc9c7 03-Apr-2014 Andreas Gampe <agampe@google.com> Fix off-by-1 error in new SpaceBitmap

Do not visit_end in the VisitMarkedRange code.

Change-Id: Iaf02788509b21a102cd1c0e2db3cbd09d0522bfa
pace_bitmap-inl.h
pace_bitmap.h
pace_bitmap_test.cc
cb8aea4bd5e77857c713edeb62bffcb1f7f06a39 03-Apr-2014 Andreas Gampe <agampe@google.com> Make SpaceBitmap cross-compiling tolerant

Change the order of bits in SpaceBitmap to be the intuitive one:
Offset 0 is bit 0, instead of the MSB. Then compiling on 32b for
64b works as expected.

Change-Id: Iee2491eaf06d4b5f8b534b7c980d5719633cb64c
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
4aeec176eaf11fe03f342aadcbb79142230270ed 28-Mar-2014 Mathieu Chartier <mathieuc@google.com> Refactor some GC code.

Reduced amount of code in mark sweep / semi space by moving
common logic to garbage_collector.cc. Cleaned up mod union tables
and deleted an unused implementation.

Change-Id: I4bcc6ba41afd96d230cfbaf4d6636f37c52e37ea
eap_bitmap-inl.h
eap_bitmap.h
od_union_table-inl.h
od_union_table.cc
od_union_table.h
3b05e9ba874449dbff65b01b8781001f7d93eea6 25-Mar-2014 Mathieu Chartier <mathieuc@google.com> Add missing debugger root visiting.

Bug: 13634574
Change-Id: I2a76f6c43f1d0ad1922f06deb40a71ff651129fd
emembered_set.cc
407f702da4f867c074fc3c8c688b8f8c32279eff 18-Feb-2014 Mathieu Chartier <mathieuc@google.com> Refactor object reference visiting logic.

Refactored the reference visiting logic to be in mirror::Object
instead of MarkSweep.

Change-Id: I773249478dc463d83b465e85c2402320488577c0
od_union_table.cc
od_union_table.h
emembered_set.cc
emembered_set.h
0e54cd0d8fc635d3dc8bf88a465fdade151a098f 20-Mar-2014 Mathieu Chartier <mathieuc@google.com> Refactor and optimize GC code.

Fixed the reference cache mod union table, and re-enabled it by
default. Added a boolean flag to count how many null objects,
immune, fast path, slow path objects we marked.

Slight speedup in mark stack processing, large speedup in image mod
union table scanning.
EvaluateAndApplyChanges Before:
Process mark stack time for full GC only:
12.464089s, 12.357870s, 12.538028s
Time spent marking mod image union table ~240ms.
After:
Process mark stack time: 12.299375s, 12.217142s, 12.187076s
Time spent marking mod image union table ~40ms.

TODO: Refactor reference visiting logic into mirror::Object.

Change-Id: I91889ded9d3f2bf127bc0051c1b1ff77e792e94f
eap_bitmap.h
od_union_table-inl.h
od_union_table.cc
emembered_set.cc
c93c530efc175954160c3834c93961a1a946a35a 21-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> Revoke rosalloc thread-local buffers at the checkpoint.

In the mark sweep collector, rosalloc thread-local buffers were
revoked during the pause. Now, they are revoked at the thread
checkpoint, as opposed to during the pause, which appears to help
reduce the pause time.

In Ritz MemAllocTest, the average sticky pause time went down ~20%
(925 us -> 724 us).

Bug: 13394464
Bug: 9986565
Change-Id: I104992a11b46d59264c0b9aa2db82b1ccf2826bc
emembered_set.cc
d387f7ee7283155cfc3a7555acd94eb1f3c4372a 20-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> Fix a libartd.so boot DCHECK failure with the GSS collector.

Bug: 11650816
Change-Id: Ibbdf1e66d2c1afe92351ee7a0ca84702bd07035d
emembered_set.cc
38e68e9978236db87c9008bbe47db80525d2fa16 07-Mar-2014 Hiroshi Yamauchi <yamauchi@google.com> Use the card table to speed up the GSS collector.

Scan only dirty cards, as opposed to the whole space, to find
references from the non-moving spaces to the bump pointer spaces at
bump pointer space only collections.

With this change, the Ritz MemAllocTest speeds up by 8-10% on host and
2-3% on N4. The Ritz EvaluateFibonacci speeds up by 8% and its average
pause time is reduced by 43% on N4.

Bug: 11650816
Change-Id: I1eefe75776bc37e24673b301ffa65a25f9bd4cde
emembered_set.cc
emembered_set.h
719d1a33f6569864f529e5a3fff59e7bca97aad0 06-Mar-2014 Ian Rogers <irogers@google.com> Enable annotalysis on clang ART builds.

Fix clang build errors aswell as restructure locking/mutex code for correct
thread safety analysis support.
Reorder make dependencies so that host builds build first as they should
provide better compilation errors than target.
Remove host's use of -fno-omit-frame-pointer as it has no value with correct
use of CFI, which we should have.

Change-Id: I72cea8da9a3757b1a0b3acb4081feccb7c6cef90
ard_table.h
eap_bitmap.h
od_union_table.cc
pace_bitmap.h
c2f4d0240b3a9b905dff5b546924865f15241481 04-Mar-2014 Mathieu Chartier <mathieuc@google.com> Fix SafePrettyTypeOf to never call VerifyObject.

Calling VerifyObject from the segfault handler caused another
segfault.

Change-Id: I459646594d830cbb110a4ac2bac25a7b90794854
ard_table-inl.h
6b8c503c6df887ff30f478ab58bdfdb223492f45 03-Mar-2014 Mathieu Chartier <mathieuc@google.com> Make ModifyCardsAtomic 64 bit safe.

There was some logic which assumed sizeof(uintptr_t) == 4 bytes.
New method also provides a slight speedup in
FormulaEvaluationActions.EvaluateAndApplyChanges

AllocSpaceClearCards:

Before total time spent aging alloc space cards:
914ms @ 389 GC iterations
886ms @ 386 GC iterations
906ms @ 413 GC iterations

After:
865ms @ 376 GC iterations
857ms @ 386 GC iterations
826ms @ 379 GC iterations

Change-Id: Ie375b7e57c36a0257a4ffe5a1622ffb6754509c7
ard_table-inl.h
a2806550cefb7c70781d8ee6279e6ad5769804cb 27-Feb-2014 Brian Carlstrom <bdc@google.com> Remove some stray stringpiece.h includes

Bug: 13186058
Change-Id: I2cb313425864a5d4e988fcf71a99c60ad63c63fb
tomic_stack.h
a1ce1fef2d49d1d537776a5308ace7102a815fe5 25-Feb-2014 Brian Carlstrom <bdc@google.com> Split up CommonTest into CommonRuntimeTest and CommonCompilerTest

Change-Id: I8dcf6b29a5aecd445f1a3ddb06386cf81dbc9c70
pace_bitmap_test.cc
815873ecc312b1d231acce71e1a16f42cdaf09f2 14-Feb-2014 Mathieu Chartier <mathieuc@google.com> Change root visitor to use Object**.

Simplifies code and improves the performance of root visiting since
we usually don't need to check to see if the object moved.

Change-Id: Iba998f5a15ae1fa1b53ca5226dd2168a411196cf
od_union_table.cc
od_union_table.h
d2fe10a3a34af171bf1631219cd2d6ff6b7778b5 15-Jan-2014 Sebastien Hertz <shertz@google.com> Remove blacklist

Removes the class initialization blacklist and use transaction to detect and
revert class initialization attempting to invoke native method. This only
concerns class initialization happening at compilation time when generating an
image (like boot.art for the system).

In transactional mode, we log every object's field assignment and array update.
Therefore we're able to abort a transaction to restore values of fields and
array as they were before the transaction starts. We also log changes to the
intern string table so we can restore its state prior to transaction start.

Since transactional mode only happens at compilation time, we don't need to log
all these changes at runtime. In order to reduce the overhead of testing if
transactional mode is on/off, we templatize interfaces of mirror::Object and
mirror::Array, respectively responsible for setting a field and setting an
array element.

For various reasons, we skip some specific fields from transaction:
- Object's class and array's length must remain unchanged so garbage collector
can compute object's size.
- Immutable fields only set during class loading: list of fields, method,
dex caches, vtables, ... as all classes have been loaded and verified before a
transaction occurs.
- Object's monitor for performance reason.

Before generating the image, we browse the heap to collect objects that need to
be written into it. Since the heap may still holds references to unreachable
objects due to aborted transactions, we trigger one collection at the end of
the class preinitialization phase.

Since the transaction is held by the runtime and all compilation threads share
the same runtime, we need to ensure only one compilation thread has exclusive
access to the runtime. To workaround this issue, we force class initialization
phase to run with only one thread. Note this is only done when generating image
so application compilation is not impacted. This issue will be addressed in a
separate CL.

Bug: 9676614
Change-Id: I221910a9183a5ba6c2b99a277f5a5a68bc69b5f9
od_union_table.cc
f5b0e20b5b31f5f5465784adcf2a204dcd69c7fd 12-Feb-2014 Hiroshi Yamauchi <yamauchi@google.com> Thread-local allocation stack.

With this change, Ritz MemAllocTest gets ~14% faster on N4.

Bug: 9986565
Change-Id: I2fb7d6f7c5daa63dd4fc73ba739e6ae4ed820617
tomic_stack.h
83c8ee000d525017ead8753fce6bc1020249b96a 28-Jan-2014 Mathieu Chartier <mathieuc@google.com> Add root types and thread id to root visiting.

Enables us to pass the root type and thread id to hprof.

Bug: 12680863
Change-Id: I6a0f1f9e3aa8f9b4033d695818ae7ca3460d67cb
eap_bitmap.cc
eap_bitmap.h
od_union_table.cc
od_union_table.h
pace_bitmap.cc
pace_bitmap.h
ef7d42fca18c16fbaf103822ad16f23246e2905d 06-Jan-2014 Ian Rogers <irogers@google.com> Object model changes to support 64bit.

Modify mirror objects so that references between them use an ObjectReference
value type rather than an Object* so that functionality to compress larger
references can be captured in the ObjectRefererence implementation.
ObjectReferences are 32bit and all other aspects of object layout remain as
they are currently.

Expand fields in objects holding pointers so they can hold 64bit pointers. Its
expected the size of these will come down by improving where we hold compiler
meta-data.
Stub out x86_64 architecture specific runtime implementation.
Modify OutputStream so that reads and writes are of unsigned quantities.
Make the use of portable or quick code more explicit.
Templatize AtomicInteger to support more than just int32_t as a type.
Add missing, and fix issues relating to, missing annotalysis information on the
mutator lock.
Refactor and share implementations for array copy between System and uses
elsewhere in the runtime.
Fix numerous 64bit build issues.

Change-Id: I1a5694c251a42c9eff71084dfdd4b51fff716822
tomic_stack.h
ard_table.cc
od_union_table.cc
od_union_table.h
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
a1602f28c0e3127ad511712d4b08db89737ae901 14-Jan-2014 Mathieu Chartier <mathieuc@google.com> Add zygote space as its own space type.

Helps prevent errors caused from doing invalid operations on the
old alloc space.

Removed some duplicated code in mark_sweep.cc and semi_space.cc.

Change-Id: I67a772cab30d698744c918aad581053f282a4a99
pace_bitmap.cc
55b2764c23776a7eba91d6514a4ad2cbc95e0fd0 24-Jan-2014 Ian Rogers <irogers@google.com> 64bit friendly GC CAS operations.

Change-Id: I54dfa5fdaf0d057ef74c84efb10a609e01923c9e
pace_bitmap-inl.h
2ef33c3dce17dabd9342ccf15cca5700487f2045 17-Jan-2014 Mathieu Chartier <mathieuc@google.com> Fix bug in CardTable::ModifyCardsAtomic.

We were passing in the wrong card address when handling unaligned
cards near the end of the scan range. This resulted in occasional
heap corruption according to heap verification. Also added
rounding to handle non card aligned scan end.

Change-Id: I1ff13c35bacb89716dac5826fdb5919d1c00e0e3
ard_table-inl.h
db7f37d57b6ac83abe6815d0cd5c50701b6be821 10-Jan-2014 Mathieu Chartier <mathieuc@google.com> Refactor large object sweeping.

Moved basic sweeping logic into large_object_space.cc.
Renamed SpaceSetMap -> ObjectSet.

Change-Id: I938c1f29f69b0682350347da2bd5de021c0e0224
eap_bitmap.cc
eap_bitmap.h
pace_bitmap.cc
pace_bitmap.h
e6da9af8dfe0a3e3fbc2be700554f6478380e7b9 16-Dec-2013 Mathieu Chartier <mathieuc@google.com> Background compaction support.

When the process state changes to a state which does not perceives
jank, we copy from the main free-list backed allocation space to
the bump pointer space and enable the semispace allocator.

When we transition back to foreground, we copy back to a free-list
backed space.

Create a seperate non-moving space which only holds non-movable
objects. This enables us to quickly wipe the current alloc space
(DlMalloc / RosAlloc) when we transition to background.

Added multiple alloc space support to the sticky mark sweep GC.

Added a -XX:BackgroundGC option which lets you specify
which GC to use for background apps. Passing in
-XX:BackgroundGC=SS makes the heap compact the heap for apps which
do not perceive jank.

Results:
Simple background foreground test:
0. Reboot phone, unlock.
1. Open browser, click on home.
2. Open calculator, click on home.
3. Open calendar, click on home.
4. Open camera, click on home.
5. Open clock, click on home.
6. adb shell dumpsys meminfo

PSS Normal ART:
Sample 1:
88468 kB: Dalvik
3188 kB: Dalvik Other
Sample 2:
81125 kB: Dalvik
3080 kB: Dalvik Other

PSS Dalvik:
Total PSS by category:
Sample 1:
81033 kB: Dalvik
27787 kB: Dalvik Other
Sample 2:
81901 kB: Dalvik
28869 kB: Dalvik Other

PSS ART + Background Compaction:
Sample 1:
71014 kB: Dalvik
1412 kB: Dalvik Other
Sample 2:
73859 kB: Dalvik
1400 kB: Dalvik Other

Dalvik other reduction can be explained by less deep allocation
stacks / less live bitmaps / less dirty cards.

TODO improvements: Recycle mem-maps which are unused in the current
state. Not hardcode 64 MB capacity of non movable space (avoid
returning linear alloc nightmares). Figure out ways to deal with low
virtual address memory problems.

Bug: 8981901

Change-Id: Ib235d03f45548ffc08a06b8ae57bf5bada49d6f3
eap_bitmap.cc
eap_bitmap.h
od_union_table.cc
b122a4bbed34ab22b4c1541ee25e5cf22f12a926 20-Nov-2013 Ian Rogers <irogers@google.com> Tidy up memory barriers.

Change-Id: I937ea93e6df1835ecfe2d4bb7d84c24fe7fc097b
tomic_stack.h
bb11f4b4f33f06c7c6e00448443b45c888ab1338 19-Dec-2013 Brian Carlstrom <bdc@google.com> Fix byte_cas to work on non-word aligned addresses

(cherry picked from commit 1279d12eb94b7664fc839d4ef7f30c7d6b20e603)

Change-Id: I307f65f20609bca8ec9c9f9a0ea4725c7ab8ad74
ard_table-inl.h
c528dba35b5faece51ca658fc008b688f8b690ad 26-Nov-2013 Mathieu Chartier <mathieuc@google.com> Enable moving classes.

Slight reduction in Zygote size, memory savings are in the noise.
Before: Zygote size: 8739224
After: Zygote size: 8733568

Fixed a bug where we didn't set the concurrent start bytes after
switching the allocator from bump pointer to ROSAlloc in the
zygote. This caused excessive memory usage.

Added the method verifiers as roots to fix an issue caused by
RegTypes holding a Class*.

Added logic to clear card table in the SemiSpace collector, this
reduces DalvikOther from ~2400k -> ~1760k when using the SemiSpace
collector.

Added a missing lock to the timing loggers which caused a rare
one time crash in std::set.

Bug: 11771255
Bug: 8499494
Bug: 10802951

Change-Id: I99d2b528cd51c1c5ed7012e3220b3aefded680ae
ard_table.cc
od_union_table.cc
cf58d4adf461eb9b8e84baa8019054c88cd8acc6 26-Sep-2013 Hiroshi Yamauchi <yamauchi@google.com> A custom 'runs-of-slots' memory allocator.

Bug: 9986565
Change-Id: I0eb73b9458752113f519483616536d219d5f798b
od_union_table-inl.h
590fee9e8972f872301c2d16a575d579ee564bee 13-Sep-2013 Mathieu Chartier <mathieuc@google.com> Compacting collector.

The compacting collector is currently similar to semispace. It works by
copying objects back and forth between two bump pointer spaces. There
are types of objects which are "non-movable" due to current runtime
limitations. These are Classes, Methods, and Fields.

Bump pointer spaces are a new type of continuous alloc space which have
no lock in the allocation code path. When you allocate from these it uses
atomic operations to increase an index. Traversing the objects in the bump
pointer space relies on Object::SizeOf matching the allocated size exactly.

Runtime changes:
JNI::GetArrayElements returns copies objects if you attempt to get the
backing data of a movable array. For GetArrayElementsCritical, we return
direct backing storage for any types of arrays, but temporarily disable
the GC until the critical region is completed.

Added a new runtime call called VisitObjects, this is used in place of
the old pattern which was flushing the allocation stack and walking
the bitmaps.

Changed image writer to be compaction safe and use object monitor word
for forwarding addresses.

Added a bunch of added SIRTs to ClassLinker, MethodLinker, etc..

TODO: Enable switching allocators, compacting on background, etc..

Bug: 8981901

Change-Id: I3c886fd322a6eef2b99388d19a765042ec26ab99
od_union_table.cc
8d31bbd3d6536de12bc20e3d29cfe03fe848f9da 13-Oct-2013 Ian Rogers <irogers@google.com> Throw IOException at source of failing to open a dex file.

Before is:
java.lang.ClassNotFoundException: Didn't find class "GCBench" on path: DexPathList[[zip file "/disk2/dalvik-dev/out/host/linux-x86/framework/GCBench.jar"],nativeLibraryDirectories=[/disk2/dalvik-dev/out/host/linux-x86/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
Suppressed: java.lang.ClassNotFoundException: GCBench
at java.lang.Class.classForName(Native Method)
at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
... 1 more
Caused by: java.lang.NoClassDefFoundError: Class "LGCBench;" not found
... 5 more
And after is:
java.lang.ClassNotFoundException: Didn't find class "GCBench" on path: DexPathList[[zip file "/disk2/dalvik-dev/out/host/linux-x86/framework/GCBench.jar"],nativeLibraryDirectories=[/disk2/dalvik-dev/out/host/linux-x86/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
Suppressed: java.io.IOException: Zip archive '/disk2/dalvik-dev/out/host/linux-x86/framework/GCBench.jar' doesn't contain classes.dex
at dalvik.system.DexFile.openDexFile(Native Method)
at dalvik.system.DexFile.<init>(DexFile.java:80)
at dalvik.system.DexFile.<init>(DexFile.java:59)
at dalvik.system.DexPathList.loadDexFile(DexPathList.java:268)
at dalvik.system.DexPathList.makeDexElements(DexPathList.java:235)
at dalvik.system.DexPathList.<init>(DexPathList.java:113)
at dalvik.system.BaseDexClassLoader.<init>(BaseDexClassLoader.java:48)
at dalvik.system.PathClassLoader.<init>(PathClassLoader.java:38)
at java.lang.ClassLoader.createSystemClassLoader(ClassLoader.java:128)
at java.lang.ClassLoader.access$000(ClassLoader.java:65)
at java.lang.ClassLoader$SystemClassLoader.<clinit>(ClassLoader.java:81)
at java.lang.ClassLoader.getSystemClassLoader(ClassLoader.java:137)
Suppressed: java.lang.ClassNotFoundException: GCBench
at java.lang.Class.classForName(Native Method)
at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
... 1 more
Caused by: java.lang.NoClassDefFoundError: Class "LGCBench;" not found
... 5 more

Also, move dex file verifier messages out of logs.
In the process the ClassLinker::dex_lock_ needed tidying to cover a smaller
scope. Bug 11301553.

Change-Id: I80058652e11e7ea63457cc01a0cb48afe1c15543
tomic_stack.h
ard_table.cc
c_allocator.cc
c_allocator.h
eap_bitmap.h
od_union_table.h
pace_bitmap.cc
pace_bitmap.h
11409ae81a3eaf84d7fd2b3c85b8b06d2bae27f0 23-Sep-2013 Mathieu Chartier <mathieuc@google.com> Refactor and improve mod-union tables.

Allow support for adding more mod union tables, reduces the amount
of baked in logic. Adds support for updating mod union table references
from compaction (not for ReferenceCache table yet).

Change-Id: I1beeda00839ed86ef0e853beff5ce10d0ab2b9d1
od_union_table-inl.h
od_union_table.cc
od_union_table.h
pace_bitmap.h
0f72e4136aecaf6976fdb55916bbd7b6d5c9c77b 07-Sep-2013 Mathieu Chartier <mathieuc@google.com> Improve heap verification.

Re-enabled checking the allocation stack for heap verification.

Added tracking of recent frees in DlMallocSpace if debug spaces is
enabled. This is useful when you have heap corruption caused by a
live object referencing a recently freed object.

Added various other sanity checks in the GC.

Bug: 10626133

Change-Id: I5ada11966336ae9a06615b16f4b933f05b5d0c32
ard_table-inl.h
ard_table.h
c4621985bdfc2b27494087e5dee65a6d0cc5a632 17-Sep-2013 Mathieu Chartier <mathieuc@google.com> Fix race in root marking.

There was a race which caused the class linker / intern table to not
become dirty after adding a root. We now guard the is dirty flag by
the corresponding locks to prevent this from occuring. This was
causing roots to be occasionally missed.

Also fixes the bug where we occasionally scan more cards than
needed.

Bug: 10626133

Change-Id: I0f6e72d92035ff463954d66988ef610ea0df61be
ard_table-inl.h
31e8925781c2302f1d1a9b39e216ba415bfe0d7e 28-Aug-2013 Mathieu Chartier <mathieuc@google.com> Write out image bitmap inside of image file.

We now create the image bitmap when we generate the image. The image
bitmap is written after the image inside of the image file. This
speeds up dex2oat by making walking the image during heap creation
unnecessary. This should also help memory pressure by enabling the
image bitmap to be swappable.

Bug: 10432288

Change-Id: Idebf459ed15edbb41a7d9b9b353934155bce2f19
pace_bitmap.cc
pace_bitmap.h
720ef7680573c1afd12f99f02eee3045daee5168 17-Aug-2013 Mathieu Chartier <mathieuc@google.com> Fix non concurrent GC ergonomics.

If we dont have concurrent GC enabled, we need to force GC for alloc
when we hit the maximum allowed footprint so that our heap doesn't
keep growing until it hits the growth limit.

Refactored a bit of stuff.

Change-Id: I8eceac4ef01e969fd286ebde3a735a09d0a6dfc1
ard_table-inl.h
94c32c5f01c7d44781317bf23933ed0a5bc4b796 09-Aug-2013 Mathieu Chartier <mathieuc@google.com> More parallel GC, rewritten parallel mark stack processing.

Card scanning may now be done in parallel. This speeds up sticky and
reduces pause times for all GC types.

Speedup on my mako (ritz perf):
Average pause time for sticky GC (~250 samples):
Without parallel cards scanning enabled: 2.524904215ms
Parallel card scanning (num_gc_threads_): 1.552123552ms
Throughput (~250 samples):
Sticky GC throughput with parallel card scanning: 69MB/s
Sticky GC throughput without parallel card scanning: 51MB/s

Rewrote the mark stack processing to be LIFO and use a prefetch queue
like the non parallel version.

Cleaned up some of the logcat printing for the activity manager
process state listening.

Added unlikely hints to object scanning since arrays and classes are
scanned much less often than normal objects.

Fixed a bug where the number of GC threads was clamped to 1 due to a
bool instead of a size_t.

Fixed a race condition when we added references to the reference
queues. Sharded the reference queue lock into one lock for each reference
type (weak, soft, phatom, finalizer).

Changed timing splits to be different for processing gray objects with
and without mutators paused since sticky GC does both.

Mask out the class bit when visiting fields as an optimization, this is
valid since classes are held live by the class linker.

Partially completed: Parallel recursive mark + finger.

Bug: 10245302
Bug: 9969166
Bug: 9986532
Bug: 9961698

Change-Id: I142d09718c4609b7c2387cb28f517a6983c73288
tomic_stack.h
ard_table-inl.h
ard_table.h
02e25119b15a6f619f17db99f5d05124a5807ff3 15-Aug-2013 Mathieu Chartier <mathieuc@google.com> Fix up TODO: c++0x, update cpplint.

Needed to update cpplint to handle const auto.

Fixed a few cpplint errors that were being missed before.

Replaced most of the TODO c++0x with ranged based loops. Loops which
do not have a descriptive container name have a concrete type instead
of auto.

Change-Id: Id7cc0f27030f56057c544e94277300b3f298c9c5
c_allocator.cc
c_allocator.h
eap_bitmap-inl.h
eap_bitmap.cc
eap_bitmap.h
od_union_table.cc
ea46f950e7a51585db293cd7f047de190a482414 30-Jul-2013 Brian Carlstrom <bdc@google.com> Refactor java.lang.reflect implementation

Cherry-picked from commit ed41d5c44299ec5d44b8514f6e17f802f48094d1.

Move to ArtMethod/Field instead of AbstractMethod/Field and have
java.lang.reflect APIs delegate to ArtMethod/ArtField.

Bug: 10014286.

Change-Id: Iafc1d8c5b62562c9af8fb9fd8c5e1d61270536e7
od_union_table.cc
pace_bitmap.cc
9b3c3cdb62f7142384e6bf2c0cb6e3a76b16f0e3 13-Aug-2013 Mathieu Chartier <mathieuc@google.com> C++11 support for ART.

We can now use auto, ranged based loops, etc..
This compiles, the phone boots, and the tests pass.

Depends on:
https://googleplex-android-review.googlesource.com/#/c/342487/

Change-Id: I8ba8ed47d2118e4711668c9c8f973a67beb261b2
ard_table.h
184e322fe8ddd75c844a1eb2eb1ca32bc02f2d45 03-Aug-2013 Mathieu Chartier <mathieuc@google.com> Disable and remove finger.

Finger is useless for multithreaded GC, removing it should provide a
slight speed up due to avoided comparison.

Change-Id: I7eb7abcbab8d3307807b1086c06d68b2d4bcd2e9
ard_table-inl.h
ard_table.h
eap_bitmap-inl.h
od_union_table.cc
pace_bitmap-inl.h
pace_bitmap.h
f082d3ce16c520b2d039869e8eb3055eda04e591 30-Jul-2013 Mathieu Chartier <mathieuc@google.com> Remove sorted variable in allocation stacks.

Speeds up allocations, mark stack processing non parallel. This is
safe to do since the pre/post GC heap verification is the only
place where we sort the allocation and live stacks and this is
done with mutators suspended.

Change-Id: I4ae1858db7109def3e2e49ebca58b924818d71f2
tomic_stack.h
7934ac288acfb2552bb0b06ec1f61e5820d924a4 26-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint whitespace/comments issues

Change-Id: Iae286862c85fb8fd8901eae1204cd6d271d69496
ard_table.cc
pace_bitmap-inl.h
58b42b986742565cccfde4366495ad23d0ba57d1 26-Jul-2013 Mathieu Chartier <mathieuc@google.com> Fix portable build.

Move dlmalloc include inside of cc file.

Change-Id: Ic36bdbf9fa964b93a23d1a45f603bab1bf9583d7
c_allocator.cc
c_allocator.h
98b6b185837bed9e7a7d8cd9deb2fe3a05f60644 26-Jul-2013 Mathieu Chartier <mathieuc@google.com> Fix build

cpplint errors.

Change-Id: I85533a6ede486a1d9129981e6a1c85c46e592905
c_allocator.h
0a9dc05e704bfd033bac2aa38a4fc6f6b8e6cf93 25-Jul-2013 Mathieu Chartier <mathieuc@google.com> GC data structures allocation tracking

Adds a new stl compatible allocator that is used in most GC data
structures. When the data structures allocate and free memory, it
lets the heap know of how much memory was allocated or freed. Using
this info, we dump the approximated stl data structures memory usage
when a sigquit occurs.

The allocation tracking can be disabled with a compile time boolean
flag to remove performance impact.

Change-Id: Idddb6713169e07be913bceeb50f305c8573e4392
c_allocator.cc
c_allocator.h
eap_bitmap-inl.h
eap_bitmap.cc
eap_bitmap.h
od_union_table.cc
od_union_table.h
pace_bitmap.h
ebd66389dacec3118232fbd426c70dbed573cff1 19-Jul-2013 Brian Carlstrom <bdc@google.com> am 4274889d: Fixing cpplint readability/check issues

* commit '4274889d48ef82369bf2c1ca70d84689b4f9e93a':
Fixing cpplint readability/check issues
58d8b956e7d83bb1b219405be87e135c89b9cdb2 19-Jul-2013 Brian Carlstrom <bdc@google.com> am 02c8cc6d: Fixing cpplint whitespace/blank_line, whitespace/end_of_line, whitespace/labels, whitespace/semicolon issues

* commit '02c8cc6d1312a2b55533f02f6369dc7c94672f90':
Fixing cpplint whitespace/blank_line, whitespace/end_of_line, whitespace/labels, whitespace/semicolon issues
4274889d48ef82369bf2c1ca70d84689b4f9e93a 19-Jul-2013 Brian Carlstrom <bdc@google.com> Fixing cpplint readability/check issues

Change-Id: Ia81db7238b4a13ff2e585aaac9d5e3e91df1e3e0
pace_bitmap_test.cc
02c8cc6d1312a2b55533f02f6369dc7c94672f90 19-Jul-2013 Brian Carlstrom <bdc@google.com> Fixing cpplint whitespace/blank_line, whitespace/end_of_line, whitespace/labels, whitespace/semicolon issues

Change-Id: Ide4f8ea608338b3fed528de7582cfeb2011997b6
od_union_table-inl.h
od_union_table.cc
pace_bitmap_test.cc
0ab5ef3e1ab1642fe59fe037f67ce8238da31b35 18-Jul-2013 Brian Carlstrom <bdc@google.com> am 3e3d591f: Fix cpplint build/namespaces issues

* commit '3e3d591f781b771de89f3b989830da2b6ac6fac8':
Fix cpplint build/namespaces issues
3e3d591f781b771de89f3b989830da2b6ac6fac8 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint build/namespaces issues

Change-Id: I19c68703270c1482d6c6aad8cdf97d3d2924360a
od_union_table.cc
b9070095218595a5d6a37ef874df2794c1761030 18-Jul-2013 Brian Carlstrom <bdc@google.com> resolved conflicts for merge of df62950e to dalvik-dev

Change-Id: I78fbcfc7a2dcbeccb7557ca27302928d7d00debd
08524597899d0bb021c9165218deff51dc88da50 18-Jul-2013 Brian Carlstrom <bdc@google.com> am 0cd7ec2d: Fix cpplint whitespace/blank_line issues

* commit '0cd7ec2dcd8d7ba30bf3ca420b40dac52849876c':
Fix cpplint whitespace/blank_line issues
df62950e7a32031b82360c407d46a37b94188fbb 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint whitespace/parens issues

Change-Id: Ifc678d59a8bed24ffddde5a0e543620b17b0aba9
tomic_stack.h
od_union_table.cc
pace_bitmap.h
pace_bitmap_test.cc
0cd7ec2dcd8d7ba30bf3ca420b40dac52849876c 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint whitespace/blank_line issues

Change-Id: Ice937e95e23dd622c17054551d4ae4cebd0ef8a2
eap_bitmap-inl.h
eap_bitmap.h
pace_bitmap.cc
pace_bitmap.h
b267c89acd0fcda0e8e2a9eeaeeed9ee048d1abc 18-Jul-2013 Brian Carlstrom <bdc@google.com> am 93ba893c: Fix cpplint runtime/explicit issues

* commit '93ba893c20532990a430741e0a97212900094e8c':
Fix cpplint runtime/explicit issues
93ba893c20532990a430741e0a97212900094e8c 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint runtime/explicit issues

Change-Id: I352ba0b427f1ff9b22887693952b180eae0839ba
eap_bitmap.h
od_union_table-inl.h
od_union_table.cc
od_union_table.h
pace_bitmap.h
42425b3a8befe092199fde69c95833e31db8fec9 18-Jul-2013 Brian Carlstrom <bdc@google.com> am fc0e3219: Fix multiple inclusion guards to match new pathnames

* commit 'fc0e3219edc9a5bf81b166e82fd5db2796eb6a0d':
Fix multiple inclusion guards to match new pathnames
fc0e3219edc9a5bf81b166e82fd5db2796eb6a0d 17-Jul-2013 Brian Carlstrom <bdc@google.com> Fix multiple inclusion guards to match new pathnames

Change-Id: Id7735be1d75bc315733b1773fba45c1deb8ace43
tomic_stack.h
ard_table-inl.h
ard_table.h
eap_bitmap-inl.h
eap_bitmap.h
od_union_table-inl.h
od_union_table.h
pace_bitmap-inl.h
pace_bitmap.h
4b95e8fad803ad307fa09c11c08894544e07a731 16-Jul-2013 Mathieu Chartier <mathieuc@google.com> Refactor atomic integer.

Refactored atomic integer to be similar to c++11 std::atomic<int>.

Removed jdwp serial lock and reverted lock level name change from
https://googleplex-android-review.googlesource.com/#/c/327297/

Change-Id: I2229f30c4d5762a0e8c72697d6aca4683750af35
tomic_stack.h
7940e44f4517de5e2634a7e07d58d0fb26160513 12-Jul-2013 Brian Carlstrom <bdc@google.com> Create separate Android.mk for main build targets

The runtime, compiler, dex2oat, and oatdump now are in seperate trees
to prevent dependency creep. They can now be individually built
without rebuilding the rest of the art projects. dalvikvm and jdwpspy
were already this way. Builds in the art directory should behave as
before, building everything including tests.

Change-Id: Ic6b1151e5ed0f823c3dd301afd2b13eb2d8feb81
tomic_stack.h
ard_table-inl.h
ard_table.cc
ard_table.h
eap_bitmap-inl.h
eap_bitmap.cc
eap_bitmap.h
od_union_table-inl.h
od_union_table.cc
od_union_table.h
pace_bitmap-inl.h
pace_bitmap.cc
pace_bitmap.h
pace_bitmap_test.cc