History log of /art/compiler/image_writer.cc
Revision Date Author Comments
aa3eff991fc34d6434465bf6bf49ef2e2fb286b7 20-Jan-2015 Andreas Gampe <agampe@google.com> ART: Curb dex2oat verbosity

Only print dedupe collisions in a debug build or on verbose logging.

Bug: 19082662

(cherry picked from commit 8d295f8f0e88fa5c6373962d545020a06033b3fc)

Change-Id: I08bd04a453d95b6dba6cf6955b5741cd97daf480
f521f423b66e952f746885dd9f6cf8ef2788955d 25-Nov-2014 Mathieu Chartier <mathieuc@google.com> Move dex cache strings from ArtMethod -> Class

Adds one load for const strings which are not direct.

Saves >= 60KB of memory avg per app.
Image size: -350KB.

Bug: 17643507
Change-Id: I2d1a3253d9de09682be9bc6b420a29513d592cc8
c6fc909c6b42b4c365b1e7f86639ee4b360249fa 22-Nov-2014 Mathieu Chartier <mathieuc@google.com> Delete ArtMethod gc_map_ field

Moved the gc_map field from OatMethod to OatQuickMethodHeader.
Deleted the ArtMethod gc_map_ field.

Bug: 17643507

Change-Id: Ifa0470c3e4c2f8a319744464d94c6838b76b3d48
3f735bd4f9d09a0f9b2b01321e4c6917879dcae6 15-Nov-2014 Igor Murashkin <iam@google.com> dex2oat: Pack likely-dirty objects together when generating the boot image

This introduces a new algorithm into image writer which "bins" objects
by how likely they are to be dirtied at runtime. Objects in the same bin
are placed contiguously in memory (i.e. into the same page). We try to
tune the bin selection based on how clean or how dirty the object will
likely be at runtime.

As-is, this saves about 150KB per-process (private-dirty pages) and 700KB in
zygote (shared-dirty).

There is still about 800KB of objects that are clean but located in
dirty pages, so with more analysis we can tune the bin selection and get
even more memory savings.

Bug: 17611661
Change-Id: Ia1455e4c56ffd0a36ae2a723d35b7e06502980f7
4b8b4693ee19efbd2d8692ca0b47b71f3b7da663 18-Nov-2014 Mathieu Chartier <mathieuc@google.com> Fix transitive equality error in set.

SubstringComparator used to be broken since it didn't have transitive
equality (ex. ab a ac). Now we use normal string comparison with
set.lower_bound saves an additional 12 bytes.

Also up frame size to 2700 to fix build.

Change-Id: Ie8d41faef37591e7feed1f5d2ad6258430b8ffb6
9433ec60b325b708b9fa87e699ab4a6565741494 06-Nov-2014 Andreas Gampe <agampe@google.com> ART: Track Flush & Close in FdFile

Implement a check that aborts when a file hasn't been explicitly
flushed and closed when it is destructed.

Add WARN_UNUSED to FdFile methods.

Update dex2oat, patchoat, scoped_flock and some gtests to pass with
this.

Change-Id: I9ab03b1653e69f44cc98946dc89d764c3e045dd4
e832e64a7e82d7f72aedbd7d798fb929d458ee8f 10-Nov-2014 Mathieu Chartier <mathieuc@google.com> Change 64 bit ArtMethod fields to be pointer sized

Changed the 64 bit entrypoint and gc map fields in ArtMethod to be
pointer sized. This saves a large amount of memory on 32 bit systems.
Reduces ArtMethod size by 16 bytes on 32 bit.

Total number of ArtMethod on low memory mako: 169957
Image size: 49203 methods -> 787248 image size reduction.
Zygote space size: 1070 methods -> 17120 size reduction.
App methods: ~120k -> 2 MB savings.

Savings per app on low memory mako: 125K+ per app
(less active apps -> more image methods per app).

Savings depend on how often the shared methods are on dirty pages vs
shared.

TODO in another CL, delete gc map field from ArtMethod since we
should be able to get it from the Oat method header.

Bug: 17643507

Change-Id: Ie9508f05907a9f693882d4d32a564460bf273ee8
23c1d0ca7ab63f4adad88631bddefb769d0dcc2c 15-Nov-2014 Mathieu Chartier <mathieuc@google.com> Combine image string char arrays into single array

Having one giant char array shared between all the strings saves memory
since it avoids the 12 bytes of array overhead per image string. Also
added substring finding based on prefixes, strings are added into the
array in reverse sorted length.

Image size goes from 11767808 -> 11100160.

Bug: 17643507

Change-Id: I2a7f177b40d0458d5c50640643d8f16b0030bdce
bfa3ed0ad988e1da13626ddbaf6dcae0c58ea79e 10-Nov-2014 Vladimir Marko <vmarko@google.com> Keep original order of fields in Class.

The fields of a class are ordered alphabetically in the dex
file. Keep the same order in the field arrays so that we can
do binary search lookups by name. Those lookups will be
implemented in a subsequent change in libcore/.

Bug: 18211592
Change-Id: I8f979de62ffe37d1c7d5c721717d2f3501e7c9e6
48447025389cd67605041a28d4ded1528381bc4e 22-Oct-2014 Igor Murashkin <iam@google.com> ART: Add support for patching and loading OAT files compiled with PIC

* Images (.art) compiled with pic now have a new field added.
* isDexOptNeeded will now skip patch-ing for apps compiled PIC
* First-boot patching now only copies boot.art, boot.oat is linked

As a result, all system preopted dex files (with --compile-pic) no
longer take up any space in /data/dalvik-cache/<isa>.

(cherry-picked from AOSP master
46774767fcf7780d1455e755729198648d08742e)

Conflicts (from aosp master):
compiler/image_test.cc
compiler/image_writer.cc
compiler/image_writer.h
compiler/oat_test.cc
dex2oat/dex2oat.cc
oatdump/oatdump.cc
runtime/elf_file.cc
runtime/elf_file.h
runtime/elf_file_impl.h
runtime/oat_file.cc
runtime/oat_file.h

Bug: 18035729

(cherry picked from commit 90ca5c0301651101de0e363842e5d08ae65233f7)

Change-Id: I8d99f95cc3d1fa221fc530ebb1fcc4b3263c183d
e05d1d5fd86867afc7513b1c546375dba11eee50 03-Nov-2014 Mathieu Chartier <mathieuc@google.com> Add hash set

More memory efficient than libcxx since we do not box the values.

Change intern table to use new hash set. Clean up intern table by
removing const casts and deleting unnecessary code.

Changed the class linker to use a hash set, also added a pre-zygote
class table.

5 samples of:
adb shell stop && adb shell start && sleep 60 && adb shell dumpsys meminfo
Before:
165929 kB: Native
175859 kB: Native
168434 kB: Native
166559 kB: Native
169958 kB: Native

After:
160972 kB: Native
159439 kB: Native
157204 kB: Native
165093 kB: Native
163039 kB: Native

TODO: Add HashTable which is implemented by using a HashSet.
TODO: Use for DexFile::find_class_def_misses_.
TODO: Investigate using mem maps instead of native heap.

Bug: 17808975

Change-Id: I93e376cf6eb9628cf52f4aefdadb6157acfb799a
90ca5c0301651101de0e363842e5d08ae65233f7 22-Oct-2014 Igor Murashkin <iam@google.com> ART: Add support for patching and loading OAT files compiled with PIC

* Images (.art) compiled with pic now have a new field added.
* isDexOptNeeded will now skip patch-ing for apps compiled PIC
* First-boot patching now only copies boot.art, boot.oat is linked

As a result, all system preopted dex files (with --compile-pic) no
longer take up any space in /data/dalvik-cache/<isa>.

(cherry-picked from AOSP master
46774767fcf7780d1455e755729198648d08742e)

Conflicts (from aosp master):
compiler/image_test.cc
compiler/image_writer.cc
compiler/image_writer.h
compiler/oat_test.cc
dex2oat/dex2oat.cc
oatdump/oatdump.cc
runtime/elf_file.cc
runtime/elf_file.h
runtime/elf_file_impl.h
runtime/oat_file.cc
runtime/oat_file.h

Bug: 18035729
Change-Id: Ie1acad81a0fd8b2f24e1f3f07a06e6fdb548be62
1fb463e42cf1d67595cff66d19c0f99e3046f4c4 24-Oct-2014 Mathieu Chartier <mathieuc@google.com> Optimize method linking

Added more inlining, removed imt array allocation and replaced it
with a handle scope. Removed some un-necessary handle scopes.

Added logic to base interface method tables from the superclass so
that we dont need to reconstruct for every interface (large win).

Facebook launch Dalvik KK MR2:
TotalTime: 3165
TotalTime: 3652
TotalTime: 3143
TotalTime: 3298
TotalTime: 3212
TotalTime: 3211

Facebook launch TOT before:
WaitTime: 3702
WaitTime: 3616
WaitTime: 3616
WaitTime: 3687
WaitTime: 3742
WaitTime: 3767

After optimizations:
WaitTime: 2903
WaitTime: 2953
WaitTime: 2918
WaitTime: 2940
WaitTime: 2879
WaitTime: 2792

LinkInterfaceMethods no longer one of the hottest methods, new list:
4.73% art::ClassLinker::LinkVirtualMethods(art::Thread*, art::Handle<art::mirror::Class>)
3.07% art::DexFile::FindClassDef(char const*) const
2.94% art::mirror::Class::FindDeclaredStaticField(art::mirror::DexCache const*, unsigned int)
2.90% art::DexFile::FindStringId(char const*) const

Bug: 18054905
Bug: 16828525

Change-Id: I27cc70178fd3655fbe5a3178887fcba189d21321
63bc11efaac0c041e849ab401f9fc368631a00f5 18-Sep-2014 Ian Rogers <irogers@google.com> DO NOT MERGE. Only have a portable entrypoint in portable builds.

Bug: 16214885

Change-Id: Iff7b7415efdbdabd7e6020e221a540f6a774c852
cb6b0f31ede2275e79e6199ec391147585a37a2a 12-Aug-2014 Ian Rogers <irogers@google.com> Avoid use of std::string where we have const char*.

Removing the ClassHelper caused std::string creation for all calls to
Class::GetDescriptor and a significant performance regression. Make the
std::string an out argument so the caller can maintain it and its life time
while allowing GetDescriptor to return the common const char* case.

Don't generate GC maps when compilation is disabled.

Remove other uses of std::string that are occuring on critical paths.
Use the cheaper SkipClass in CompileMethod in CompilerDriver.
Specialize the utf8 as utf16 comparison code for the common shorter byte
encoding.
Force a bit of inlining, remove some UNLIKELYs (they are prone to pessimizing
code), add some LIKELYs.

x86-64 host 1-thread interpret-only of 57 apks:
Before: 29.539s
After: 23.467s

Regular compile:
Before: 1m35.347s
After: 1m20.056s

Bug: 16853450
Change-Id: Ic705ea24784bee24ab80084d06174cbf87d557ad

Conflicts:
runtime/utils.cc
a59dd80f9f48cb750d329d4d4af2d99d72b484d1 03-Jul-2014 Alex Light <allight@google.com> Runtime can now be set to require relocation

Add a pair of runtime flags -Xrelocate and -Xnorelocate that can force
the runtime to require that all files that are run are relocated, to
prevent attacks based on the known art base address.

Add support for running patchoat on oat files compiled without an image.

Change run-test to have new --prebuild and --relocate flags.

Bug: 15358152

Change-Id: I91166c62dd1ab80e5cbcb7883a2cd0d56afca32d
22d5e735f403c57525fe868304c7123f0ce66399 16-Jul-2014 Ian Rogers <irogers@google.com> Remove object_utils.h.

Break into object_lock, field_helper and method_helper.
Clean up header files following this.
Also tidy some of the Handle code in response to compiler errors when resolving
the changes in this CL.

Change-Id: I73e63015a0f02a754d0866bfaf58208aebcaa295
98d1cc8033251c93786e2fa8c59a2e555a9493be 16-May-2014 Mingyao Yang <mingyao@google.com> Improve performance of invokevirtual/invokeinterface with embedded imt/vtable

Add an embedded version of imt/vtable into class object. Both tables start at
fixed offset within class object so method/entry point can be loaded directly
from class object for invokeinterface/invokevirtual.

Bug: 8142917
Change-Id: I4240d58cfbe9250107c95c0708c036854c455968
53cb16b98acf3cf6f3a1e2204ad4958ecf1b5a3c 12-Jun-2014 Alex Light <allight@google.com> Add patchoat tool to Art.

Add a new executable called patchoat to art. This tool takes already
compiled images and oat files and changes their base address, acting as
a cheap form of relocation.

Add a --include-patch-information flag to dex2oat and code to add
required patch information to oat files created with the quick compiler.

Bug: 15358152

Change-Id: Ie0c580db45bb14ec180deb84930def6c3628d97d
eefbe39799126afdf7d315a79359b9da23d1cef5 08-Jul-2014 Alex Light <allight@google.com> Fix some style nitpicks

Change-Id: Icfdd327f4ddf129f0a8607162c09ba271c1d49d9
e9e3e697f0c426132bee10aaa6aee9107d2d7dc6 24-Jun-2014 Hiroshi Yamauchi <yamauchi@google.com> Add more read barriers to the class linker.

This change makes it possible to concurrently scan the remaining roots
in the class linker (the non-class-table roots that are visited by
ClassLinker::VisitRoots()) by adding read barriers.

Bug: 12687968
Change-Id: I66fecf7a303eee7537429e018f38da8270b18c67
e1d07816510e0f6c9969cddd7d7b683698f93b26 21-May-2014 Sebastien Hertz <shertz@google.com> Fix interpreter entrypoint initialization in the image

Sets the interpreter_to_interpreter_bridge entrypoint if the method is not
compiled. This avoids the slow path through Quick compiler's world.i

Bumps oat version to force recompilation of image.

Bug: 15098601
Change-Id: I08f7dfce26ecc5ca88bdf21187b2909c0c069596
ffddfdf6fec0b9d98a692e27242eecb15af5ead2 03-Jun-2014 Tim Murray <timmurray@google.com> DO NOT MERGE

Merge ART from AOSP to lmp-preview-dev.

Change-Id: I0f578733a4b8756fd780d4a052ad69b746f687a9
d8a737aafd2b797f8186cf92e529482e7e307aa8 30-May-2014 Mathieu Chartier <mathieuc@google.com> Revert "Revert "Clean up image writer patching.""

This reverts commit fb241b6e93f8b54122b978ad0df23a9e514eef0f.

Change-Id: I99da82fd549bcdc3c2fdfe496147897335c50b8e
7d791a4df0ac5d242c800e60d5151a7a370e9e95 30-May-2014 Mathieu Chartier <mathieuc@google.com> Revert "Revert "Add support for relative patching to quick offsets.""

This reverts commit 3e2e99484043aa25b8ed41509ac1b3ccd7d6a959.

Change-Id: Ifbe604330cd3a1dd0a8afa7c500868e2fd0fc500
3e2e99484043aa25b8ed41509ac1b3ccd7d6a959 30-May-2014 Mathieu Chartier <mathieuc@google.com> Revert "Add support for relative patching to quick offsets."

This reverts commit 57d27334bcb9ea95c969be70d8d93eb1a9c72b41.

Change-Id: I5df09ee342c337291388a48e4975a6bf5c1f9944
fb241b6e93f8b54122b978ad0df23a9e514eef0f 30-May-2014 Mathieu Chartier <mathieuc@google.com> Revert "Clean up image writer patching."

This reverts commit fc3b871ca37593561e3b67006771710bf4d174ea.

Change-Id: I4d1250f3adcd17fcd1b63496816e098db4d5be3c
fc3b871ca37593561e3b67006771710bf4d174ea 30-May-2014 Mathieu Chartier <mathieuc@google.com> Clean up image writer patching.

Also fixed an issue related to incorrect patching of non relative
patches.

Change-Id: I55b0b3814b65b7fa8749a934dfaaf07b97735304
57d27334bcb9ea95c969be70d8d93eb1a9c72b41 29-May-2014 Mathieu Chartier <mathieuc@google.com> Add support for relative patching to quick offsets.

We now have support for patching relative patches to quick offsets
(interpreter bridge, generic trampoline). This fixes an issue where
the compiler was generating relative branches to methods but then
later deciding to not compile them, we now properly patch these in
the image writer.

Credits: agampe for help.
Bug: 15283931

Change-Id: I3ea5278b39def6c442432fb14103a91c6eae6d0f
0cd81352a7c06e381951cea1b104fd73516f4341 23-May-2014 Mathieu Chartier <mathieuc@google.com> Revert "Revert "Fix an outstanding compaction bug in interpreter.""

Fixed the generic trampoline to not use ToJObject when unnecessary.

Bug: 15167269

This reverts commit 3bdb873122964da7937eb070cbcf2ef638a8e459.

Change-Id: I0525d0e0f3afb753c770e1572070a0fa22b02271
3bdb873122964da7937eb070cbcf2ef638a8e459 23-May-2014 Mathieu Chartier <mathieuc@google.com> Revert "Fix an outstanding compaction bug in interpreter."

This reverts commit e09ae0920be57760fb390b6944bce420fa0b5582.

Change-Id: I48036306130d5ccfec683d0dc3e9a642a02ee9c1
e09ae0920be57760fb390b6944bce420fa0b5582 15-May-2014 Mathieu Chartier <mathieuc@google.com> Fix an outstanding compaction bug in interpreter.

Fixed a bug in DoFieldPut where the FieldHelper GetType could cause
thread suspension which would result in a stale obj.

Added more handles in the class linker to facilitate moving fiels
and methods in the future.

Removed un-necessarly passing handle references since these are value
types and don't need to be passed by reference.

Added a special NullHandle type which allows null handles without a
handle scope.

Change-Id: I1b51723920a2e4f4f8b2907066f578a3e879fd5b
700a402244a1a423da4f3ba8032459f4b65fa18f 20-May-2014 Ian Rogers <irogers@google.com> Now we have a proper C++ library, use std::unique_ptr.

Also remove the Android.libcxx.mk and other bits of stlport compatibility
mechanics.

Change-Id: Icdf7188ba3c79cdf5617672c1cfd0a68ae596a61
f832284dd847ff077577bb5712225430bbbb3b67 16-May-2014 Mathieu Chartier <mathieuc@google.com> Delete ClassHelper and fix compaction bug in GetDirectInterface

Cleanup helps to prevent compaction bugs. Fixed a fairly serious
compaction error caused by calling ClassHelper::GetDirectInterface
without handling the case where it causes thread suspension due to
ResolveType.

Bug: 8981901

Change-Id: I82b3bb6dd48d21eb6ece7aae0733c4a23c2bc408
f0972a410a0665dbe32bd96df09a572d69f9f3a3 16-May-2014 Dmitry Petrochenko <dmitry.petrochenko@intel.com> Fix generic jni issue in ArtMethod::GetQuickFrameInfo

The 64-bit host mode fails to start due to incorrect
detection of GetQuickGenericJniTrampoline.
The quick_code is 32-bit and taken from oat file, but
GetQuickGenericJniTrampoline returnf 0x7fffxx (64-bit)
address of trampoline and execution went to incorrect way.

Some clean-up.

Original Author: Dmitry Petrochenko <dmitry.petrochenko@intel.com>
Signed-off-by: Dmitry Petrochenko <dmitry.petrochenko@intel.com>
Signed-off-by: Serguei Katkov <serguei.i.katkov@intel.com>
Change-Id: I0952443b2a9f6833ad37ec373837ae208681fad7
507dfdd147c97bfbadebfd63584d094b6a4e7b47 16-May-2014 Ian Rogers <irogers@google.com> Compatibility layer to transition from UniquePtr to std::unique_ptr.

Use ART_WITH_STLPORT (enabled for the target) to cause the use of UniquePtr,
for the host switch to std::unique_ptr. For now the type remains called
UniquePtr.
Make dalvik compile with clang on the host, move its build to C++11.

Change-Id: I5ba8d2757904bc089ed62047ea03de3c0853fb12
eb8167a4f4d27fce0530f6724ab8032610cd146b 08-May-2014 Mathieu Chartier <mathieuc@google.com> Add Handle/HandleScope and delete SirtRef.

Delete SirtRef and replaced it with Handle. Handles are value types
which wrap around StackReference*.

Renamed StackIndirectReferenceTable to HandleScope.

Added a scoped handle wrapper which wraps around an Object** and
restores it in its destructor.

Renamed Handle::get -> Get.

Bug: 8473721

Change-Id: Idbfebd4f35af629f0f43931b7c5184b334822c7a
6e83c172f385cb45dd13bbcf41d2df8e410828c6 02-May-2014 Hiroshi Yamauchi <yamauchi@google.com> Replace the bool kDoReadBarrier template parameter with an enum.

Fix one kDoReadBarrier/kIsVolatile mixup in an Object::GetFieldObject
call.

Bug: 12687968

Change-Id: I896b1137b21a20c0504abd2bf3fe6f83805f3300
b0fa5dc7769c1e054032f39de0a3f6d6dd06f8cf 29-Apr-2014 Ian Rogers <irogers@google.com> Force inlining on trivial accessors.

Make volatility for GetFieldObject a template parameter.
Move some trivial mirror::String routines to a -inl.h.

Bug: 14285442

Change-Id: Ie23b11d4f18cb15a62c3bbb42837a8aaf6b68f92
8a630577ed2d9e9571c3434c505e5de223b23c07 09-Apr-2014 Vladimir Marko <vmarko@google.com> Move mapping table and vmap table offsets to OatMethodHeader.

This change has a libcore/ companion CL
"Remove ArtMethod's quick fields mapping table and vmap table."
https://android-review.googlesource.com/91254

Bug: 11767815
Change-Id: I46ce2067e1ecd915da3890606498e31ffc332813
4d7f61d44a732cfbc8573e5d93364983fd746888 17-Apr-2014 Mathieu Chartier <mathieuc@google.com> Use non volatile lock words in semispace collector.

GSS FormulaEvaluationActions.EvaluateAndApplyChanges:
Before GC time: 9.1s
After GC time: 7.98s

Fixed timing logger errors.

Change-Id: I4193c6ccbbbe7a7220dfaabbf3472a5dcebae616
a8e8f9c0a8e259a807d7b99a148d14104c24209d 09-Apr-2014 Mathieu Chartier <mathieuc@google.com> Refactor space bitmap to support different alignments.

Required for:
Using space bitmaps instead of std::set in mod union table +
remembered set.
Using a bitmap instead of set for large object marking.

Bug: 13571028

Change-Id: Id024e9563d4ca4278f79607cdb2f81895121b113
624468cd401cc1ac0dd70c746301e0788a597759 01-Apr-2014 Hiroshi Yamauchi <yamauchi@google.com> Make the support code for read barriers a bit more general.

Add an option for Baker in addition to Brooks.

Bug: 12687968
Change-Id: I8a31db817ff6686c72951b6534f588228e270b11
b7ea3acf7b83975a9effadb350c0b62ce4ab2142 25-Mar-2014 Mathieu Chartier <mathieuc@google.com> Refactor image writer reference visiting logic.

Now uses Object::VisitReferences.

Change-Id: I5a4557e10796d6f34596f2e8796ad9382121c567
49161cef10a308aedada18e9aa742498d6e6c8c7 12-Mar-2014 Jeff Hao <jeffhao@google.com> Allow patching between dex files in the boot classpath.

Change-Id: I53f219a5382d0fcd580e96e50025fdad4fc399df
1a5706611bffa5d6ed6843ee5e320f504590e097 12-Mar-2014 Ian Rogers <irogers@google.com> A few 64bit fixes.

Change-Id: I1fe189d638b9cb5127b897da6cecdad6902db930
c645f1ddb7c40bea6a38eda4b3f83f6b6dec405b 07-Mar-2014 Mathieu Chartier <mathieuc@google.com> Add more VerifyObject calls.

Added verify object calls to SirtRef, IndirectReferenceTable,
ReferenceTable.

Removed un-needed verify object in ScopedObjectAccess / DecodeJObject
since object sources are handled.

Bug: 12934910
Change-Id: I55a46a8ea61fed2a77526eda27fd2cce97a9b125
5e754d88c73f1f1fafbbf87d564df000d5a0a12c 05-Mar-2014 Brian Carlstrom <bdc@google.com> Revert "Revert "Avoid compiling monster methods in boot image""

This reverts commit f679fab9be8905505d26a62bea257641f0bb0168.

(cherry picked from commit 9d0bdfe63bec5a43d7668d9094a52caf77b6ed53)

Change-Id: I72af1d44483edbd2a5f2e6b678e43f740d4613fd
9d0bdfe63bec5a43d7668d9094a52caf77b6ed53 05-Mar-2014 Brian Carlstrom <bdc@google.com> Revert "Revert "Avoid compiling monster methods in boot image""

This reverts commit f679fab9be8905505d26a62bea257641f0bb0168.

Change-Id: Ib92c4b9bff52e80e2e23440054fa2b9d8a639979
2ec6520d57479d393bffa05defa1479b25ca8382 04-Mar-2014 Brian Carlstrom <bdc@google.com> Support compiler filters for boot classpath

image_writer.cc
Remove assumption that all methods in the boot classpath are compiled

oat_writer.cc
Don't skip writing ArtMethod::quick_code_offset_ for methods that need resolution, leave that to ImageWriter

dex2oat.cc
Allow dex2dex compilation of image dex files by making the in memory pages writable in all cases, not just app case.

oatdump.cc
dump new OatHeader fields
use ImageSpace.GetImageFilename, not command line image filename, since location may be in dalvik-cache
remove inaccurate check about non-null GC map

quick_trampoline_entrypoints.cc
add and improve some DCHECKS that were useful while debugging

class_linker.cc
image_space.cc
fix double facepalm

parsed_options.cc
fix zygote logging to not skip values to two part options like -classpath <foo>

runtime.cc
wireup parsed compiler options to runtime

Change-Id: Iad314df0b80623c0663d61713d5098297ab9ac87
9583fbcf597eff6d0b3c5359b8e8d5f70ed82c40 28-Feb-2014 Nicolas Geoffray <ngeoffray@google.com> Remove oat file location in the image.

The oat file is now always in the same directory, and has the
same name as the image file. Only difference is the extension.

This also removes the need for host-prefix.

Change-Id: I16d1f7aeb1d58372d41921694664e9c321afc1ad
2da882315a61072664f7ce3c212307342e907207 27-Feb-2014 Andreas Gampe <agampe@google.com> Initial changes towards Generic JNI option

Some initial changes that lead to an UNIMPLEMENTED. Works
by not compiling for JNI right now and tracking native methods
which have neither quick nor portable code. Uses new trampoline.

Change-Id: I5448654044eb2717752fd7359f4ef8bd5c17be6e
9d04a20bde1b1855cefc64aebc1a44e253b1a13b 31-Jan-2014 Hiroshi Yamauchi <yamauchi@google.com> (Experimental) Add Brooks pointers.

This feature is disabled by default.

Verified that the Brooks pointers are installed correctly by using the
CMS/SS collectors.

Change-Id: Ia9be9814ab6e29169ac85edc4792ce8c81d552a9
9837939678bb5dcba178e5fb00ed59b5d14c8d9b 25-Feb-2014 Ian Rogers <irogers@google.com> Avoid std::string allocations for finding an array class.

Introduce ClassLinker::FindArrayClass which performs an array class lookup
given the element/component class. This has a 16 element cache of recently
looked up arrays.
Pass the current thread to ClassLinker Find .. Class routines to avoid calls
to Thread::Current().
Avoid some uses of FindClass in the debugger where WellKnownClasses is a
faster and more compacting GC friendly alternative.

Change-Id: I60e231820b349543a7edb3ceb9cf1ce92db3c843
4e30541a92381fb280cd0be9a1763b713ee4d64c 19-Feb-2014 Mathieu Chartier <mathieuc@google.com> Fix and optimize verify object.

VerifyObject no longer resides in heap. You can now enable
VerifyObject for non-debug builds. VerifyStack is still slow, so it
is now guarded by its own flag.

Fixed the image writer to not use verification at places where
verification fails due to invalid reads.

Fixed RosAlloc to use SizeOf which doesn't call verify object.

Added a flag paremeter to some of the mirror getters / setters to
be able to selectively disable VerifyObject on certain calls.

Optimized the GC to not verify each object multiple times during
object scanning if verify object is enabled.

Added 3 verification options: verify reads, verify this, and verify
writes so that you can select how much verification you want for
mirror getters and setters.

Removed some useless DCHECKs which would slow debug builds without
providing any benefits.

TODO: RosAlloc verification doesn't currently work with verify
objects.

Bug: 12934910
Bug: 12879358

Change-Id: Ic61033104dfc334543f89b0fc0ad8cd4f4015d69
d2fe10a3a34af171bf1631219cd2d6ff6b7778b5 15-Jan-2014 Sebastien Hertz <shertz@google.com> Remove blacklist

Removes the class initialization blacklist and use transaction to detect and
revert class initialization attempting to invoke native method. This only
concerns class initialization happening at compilation time when generating an
image (like boot.art for the system).

In transactional mode, we log every object's field assignment and array update.
Therefore we're able to abort a transaction to restore values of fields and
array as they were before the transaction starts. We also log changes to the
intern string table so we can restore its state prior to transaction start.

Since transactional mode only happens at compilation time, we don't need to log
all these changes at runtime. In order to reduce the overhead of testing if
transactional mode is on/off, we templatize interfaces of mirror::Object and
mirror::Array, respectively responsible for setting a field and setting an
array element.

For various reasons, we skip some specific fields from transaction:
- Object's class and array's length must remain unchanged so garbage collector
can compute object's size.
- Immutable fields only set during class loading: list of fields, method,
dex caches, vtables, ... as all classes have been loaded and verified before a
transaction occurs.
- Object's monitor for performance reason.

Before generating the image, we browse the heap to collect objects that need to
be written into it. Since the heap may still holds references to unreachable
objects due to aborted transactions, we trigger one collection at the end of
the class preinitialization phase.

Since the transaction is held by the runtime and all compilation threads share
the same runtime, we need to ensure only one compilation thread has exclusive
access to the runtime. To workaround this issue, we force class initialization
phase to run with only one thread. Note this is only done when generating image
so application compilation is not impacted. This issue will be addressed in a
separate CL.

Bug: 9676614
Change-Id: I221910a9183a5ba6c2b99a277f5a5a68bc69b5f9
55d0eac918321e0525f6e6491f36a80977e0d416 06-Feb-2014 Mark Mendell <mark.p.mendell@intel.com> Support Direct Method/Type access for X86

Thumb generates code to optimize calls to methods within core.oat.
Implement this for X86 as well, but take advantage of mov with 32 bit
immediate and call relative with 32 bit immediate.

Fix some incorrect return locations for long inlines.

Change-Id: I1907bdfc7574f3d0aa76c7fad13dc537acdf1ed3
Signed-off-by: Mark Mendell <mark.p.mendell@intel.com>
ef7d42fca18c16fbaf103822ad16f23246e2905d 06-Jan-2014 Ian Rogers <irogers@google.com> Object model changes to support 64bit.

Modify mirror objects so that references between them use an ObjectReference
value type rather than an Object* so that functionality to compress larger
references can be captured in the ObjectRefererence implementation.
ObjectReferences are 32bit and all other aspects of object layout remain as
they are currently.

Expand fields in objects holding pointers so they can hold 64bit pointers. Its
expected the size of these will come down by improving where we hold compiler
meta-data.
Stub out x86_64 architecture specific runtime implementation.
Modify OutputStream so that reads and writes are of unsigned quantities.
Make the use of portable or quick code more explicit.
Templatize AtomicInteger to support more than just int32_t as a type.
Add missing, and fix issues relating to, missing annotalysis information on the
mutator lock.
Refactor and share implementations for array copy between System and uses
elsewhere in the runtime.
Fix numerous 64bit build issues.

Change-Id: I1a5694c251a42c9eff71084dfdd4b51fff716822
be1ca55db3362f5b100c4c65da5342fd299520bb 15-Jan-2014 Hiroshi Yamauchi <yamauchi@google.com> Use direct class pointers at allocation sites in the compiled code.

- Rather than looking up a class from its type ID (and checking if
it's resolved/initialized, resolving/initializing if not), use
direct class pointers, if possible (boot-code-to-boot-class pointers
and app-code-to-boot-class pointers.)
- This results in a 1-2% speedup in Ritz MemAllocTest on Nexus 4.
- Embedding the object size (along with class pointers) caused a 1-2%
slowdown in MemAllocTest and isn't implemented in this change.
- TODO: do the same for array allocations.
- TODO: when/if an application gets its own image, implement
app-code-to-app-class pointers.
- Fix a -XX:gc bug.
cf. https://android-review.googlesource.com/79460/
- Add /tmp/android-data/dalvik-cache to the list of locations to
remove oat files in clean-oat-host.
cf. https://android-review.googlesource.com/79550
- Add back a dropped UNLIKELY in FindMethodFromCode().
cf. https://android-review.googlesource.com/74205

Bug: 9986565
Change-Id: I590b96bd21f7a7472f88e36752e675547559a5b1
5ddb4104ac605d66693b55b79f26f8b8a5505e63 07-Jan-2014 Ian Rogers <irogers@google.com> Remove intialized static storage from dex cache.

The initialized static storage array is used by compiled code to determine if
for a sget/sput class initialization is necessary. The compiled code typically
doesn't require this test as the class is pre-initialized or the class being
accessed is the same as the current method.

Change-Id: Icbc45e692b3d0ac61e559e69edb6c9b29439e571
460536209b741bc469f1b0857775449abb2102fb 12-Dec-2013 Bernhard Rosenkränzer <Bernhard.Rosenkranzer@linaro.org> Don't rely on gcc extensions

Make the code more compatible with different compilers.
clang doesn't allow extra static qualifiers on template
specializations, const qualifiers on function types,
or inline attributes on lambda functions, and is more
picky about casting away constness with reinterpret_cast.

These modifications are compatible with both gcc and
clang.

Change-Id: I739b10df2780bec537827a13679fd2bcc2cc7188
Signed-off-by: Bernhard Rosenkränzer <Bernhard.Rosenkranzer@linaro.org>
50482234bed852766498321f71d2ff5e46e4fec2 21-Nov-2013 Mathieu Chartier <mathieuc@google.com> Compact zygote.

We now start out using the bump pointer allocator and switch to
the free list collector (ROSAlloc) after the zygote forks.

Before compaction:
Zygote size: 9060352

After compaction
Zygote size: 8425864

The main reason the size doesn't reduce more is that most of the
zygote space is non-movable objects allocated by
VMRuntime.newNonMovableObject. The objects which are non-movable
but could be movable include around 10000 classes and some number
of fields and methods.

Bug: 8981901

Change-Id: Iea21b70fb7af27cb7e92d72070d278a5cd4026ac
24c534d740fc50d5ed03538ec725144f44b4c0f7 14-Nov-2013 Ian Rogers <irogers@google.com> Don't read characters from 0 length strings.

Caught by valgrind. Bug: 11670287

Change-Id: I3acf4855c8662b804cf0c24680fc21c50c435bdb
12aeccd09a87be7a0b995182151d778cfeb50e73 14-Nov-2013 Mathieu Chartier <mathieuc@google.com> Fix image writer bitmap size calculation.

Wasn't properly dividing by 8 inside of the image header creation.
This resulted in a bitmap size which was larger than it should be
inside of the image.

Change-Id: I344d1f3c1794a7cff3c9e22afc7fdabedf74413c
590fee9e8972f872301c2d16a575d579ee564bee 13-Sep-2013 Mathieu Chartier <mathieuc@google.com> Compacting collector.

The compacting collector is currently similar to semispace. It works by
copying objects back and forth between two bump pointer spaces. There
are types of objects which are "non-movable" due to current runtime
limitations. These are Classes, Methods, and Fields.

Bump pointer spaces are a new type of continuous alloc space which have
no lock in the allocation code path. When you allocate from these it uses
atomic operations to increase an index. Traversing the objects in the bump
pointer space relies on Object::SizeOf matching the allocated size exactly.

Runtime changes:
JNI::GetArrayElements returns copies objects if you attempt to get the
backing data of a movable array. For GetArrayElementsCritical, we return
direct backing storage for any types of arrays, but temporarily disable
the GC until the critical region is completed.

Added a new runtime call called VisitObjects, this is used in place of
the old pattern which was flushing the allocation stack and walking
the bitmaps.

Changed image writer to be compaction safe and use object monitor word
for forwarding addresses.

Added a bunch of added SIRTs to ClassLinker, MethodLinker, etc..

TODO: Enable switching allocators, compacting on background, etc..

Bug: 8981901

Change-Id: I3c886fd322a6eef2b99388d19a765042ec26ab99
4e6a31eb97f22f4480827474b30b9e64f396eace 31-Oct-2013 Mathieu Chartier <mathieuc@google.com> Lazily compute object identity hash codes.

Before, we computed identity hashcodes whenever we inflated a monitor.
This caused issues since it meant that we would have all of these
hash codes in the image, causing locks to excessively inflate during
application run time.

This change makes it so that we lazily compute hash codes. When a
thin lock gets inflated, we assign a hash code of 0 assigned to it.
This value signifies no hash code. When we try to get the identity
hash code of an object with an inflated monitor, it gets computed if
it is 0.

Change-Id: Iae6acd1960515a36e74644e5b1323ff336731806
dfb325e0ddd746cd8f7c2e3723b3a573eb7cc111 30-Oct-2013 Ian Rogers <irogers@google.com> Don't use UTF16 length as length for MUTF8.

Bug 11367555.

Change-Id: Ia0b07072a1a49d435c3b71ed9a668b316b7ff5d8
ad2541a59c00c2c69e8973088891a2b5257c9780 25-Oct-2013 Mathieu Chartier <mathieuc@google.com> Fix object identity hash.

The object identity hash is now stored in the monitor word after
being computed. Hashes are computed by a pseudo random number
generator.

When we write the image, we eagerly compute object hashes to
prevent pages getting dirtied.

Bug: 8981901

Change-Id: Ic8edacbacb0afc7055fd740a52444929f88ed564
88474b416eb257078e590bf9bc7957cee604a186 24-Oct-2013 Jeff Hao <jeffhao@google.com> Implement Interface Method Tables (IMT).

Change-Id: Idf7fe85e1293453a8ad862ff2380dcd5db4e3a39
8d31bbd3d6536de12bc20e3d29cfe03fe848f9da 13-Oct-2013 Ian Rogers <irogers@google.com> Throw IOException at source of failing to open a dex file.

Before is:
java.lang.ClassNotFoundException: Didn't find class "GCBench" on path: DexPathList[[zip file "/disk2/dalvik-dev/out/host/linux-x86/framework/GCBench.jar"],nativeLibraryDirectories=[/disk2/dalvik-dev/out/host/linux-x86/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
Suppressed: java.lang.ClassNotFoundException: GCBench
at java.lang.Class.classForName(Native Method)
at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
... 1 more
Caused by: java.lang.NoClassDefFoundError: Class "LGCBench;" not found
... 5 more
And after is:
java.lang.ClassNotFoundException: Didn't find class "GCBench" on path: DexPathList[[zip file "/disk2/dalvik-dev/out/host/linux-x86/framework/GCBench.jar"],nativeLibraryDirectories=[/disk2/dalvik-dev/out/host/linux-x86/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
Suppressed: java.io.IOException: Zip archive '/disk2/dalvik-dev/out/host/linux-x86/framework/GCBench.jar' doesn't contain classes.dex
at dalvik.system.DexFile.openDexFile(Native Method)
at dalvik.system.DexFile.<init>(DexFile.java:80)
at dalvik.system.DexFile.<init>(DexFile.java:59)
at dalvik.system.DexPathList.loadDexFile(DexPathList.java:268)
at dalvik.system.DexPathList.makeDexElements(DexPathList.java:235)
at dalvik.system.DexPathList.<init>(DexPathList.java:113)
at dalvik.system.BaseDexClassLoader.<init>(BaseDexClassLoader.java:48)
at dalvik.system.PathClassLoader.<init>(PathClassLoader.java:38)
at java.lang.ClassLoader.createSystemClassLoader(ClassLoader.java:128)
at java.lang.ClassLoader.access$000(ClassLoader.java:65)
at java.lang.ClassLoader$SystemClassLoader.<clinit>(ClassLoader.java:81)
at java.lang.ClassLoader.getSystemClassLoader(ClassLoader.java:137)
Suppressed: java.lang.ClassNotFoundException: GCBench
at java.lang.Class.classForName(Native Method)
at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
... 1 more
Caused by: java.lang.NoClassDefFoundError: Class "LGCBench;" not found
... 5 more

Also, move dex file verifier messages out of logs.
In the process the ClassLinker::dex_lock_ needed tidying to cover a smaller
scope. Bug 11301553.

Change-Id: I80058652e11e7ea63457cc01a0cb48afe1c15543
fc0e94bed3f88ed7e50854fd8dfaf5dcb345250f 24-Sep-2013 Ian Rogers <irogers@google.com> StringPiece clean up.

Profile guided clean up.
Try to avoid creating StringPieces with the contents of a dex file where
the length is known.
Try to avoid RegTypeCache::FromDescriptor when there's a class available.
Make ConstantType::ConstantValue inlinable.
Saving of about 50ms from a 2 threaded ThinkFree compile on host.

Change-Id: I47a12c3c76f46e2c9805be1c3a3e3870fe1f5d85
ee39a10e45a6a0880e8b829525c40d6055818560 19-Sep-2013 Ian Rogers <irogers@google.com> Use class def index from java.lang.Class.

Bug: 10244719
This removes the computation of the dex file index, when necessary this is
computed by searching the dex file. Its only necessary in
dalvik.system.DexFile.defineClassNative and DexFile::FindInClassPath, the
latter not showing up significantly in profiling with this change.

(cherry-picked from 8b2c0b9abc3f520495f4387ea040132ba85cae69)
Change-Id: I20c73a3b17d86286428ab0fd21bc13f51f36c85c
8b2c0b9abc3f520495f4387ea040132ba85cae69 19-Sep-2013 Ian Rogers <irogers@google.com> Use class def index from java.lang.Class.

Bug: 10244719
Depends on:
https://googleplex-android-review.git.corp.google.com/362363
This removes the computation of the dex file index, when necessary this is
computed by searching the dex file. Its only necessary in
dalvik.system.DexFile.defineClassNative and DexFile::FindInClassPath, the
latter not showing up significantly in profiling with this change.

Change-Id: I20c73a3b17d86286428ab0fd21bc13f51f36c85c
31e8925781c2302f1d1a9b39e216ba415bfe0d7e 28-Aug-2013 Mathieu Chartier <mathieuc@google.com> Write out image bitmap inside of image file.

We now create the image bitmap when we generate the image. The image
bitmap is written after the image inside of the image file. This
speeds up dex2oat by making walking the image during heap creation
unnecessary. This should also help memory pressure by enabling the
image bitmap to be swappable.

Bug: 10432288

Change-Id: Idebf459ed15edbb41a7d9b9b353934155bce2f19
02e25119b15a6f619f17db99f5d05124a5807ff3 15-Aug-2013 Mathieu Chartier <mathieuc@google.com> Fix up TODO: c++0x, update cpplint.

Needed to update cpplint to handle const auto.

Fixed a few cpplint errors that were being missed before.

Replaced most of the TODO c++0x with ranged based loops. Loops which
do not have a descriptive container name have a concrete type instead
of auto.

Change-Id: Id7cc0f27030f56057c544e94277300b3f298c9c5
414af10d719603fb4d8d972f5a022c17957b44e1 13-Aug-2013 Brian Carlstrom <bdc@google.com> Add flock(2)ing on dex-cache files to prevent races

Bug: 9071417
Change-Id: I1ee9ff281867f90fba7a8ed8bbf06b33ac29d511
7571e8b761ebc2c923525e12ea9fcf07e62cb33e 13-Aug-2013 Brian Carlstrom <bdc@google.com> Add flock(2)ing on dex-cache files to prevent races

Bug: 9071417
Change-Id: I1ee9ff281867f90fba7a8ed8bbf06b33ac29d511
ea46f950e7a51585db293cd7f047de190a482414 30-Jul-2013 Brian Carlstrom <bdc@google.com> Refactor java.lang.reflect implementation

Cherry-picked from commit ed41d5c44299ec5d44b8514f6e17f802f48094d1.

Move to ArtMethod/Field instead of AbstractMethod/Field and have
java.lang.reflect APIs delegate to ArtMethod/ArtField.

Bug: 10014286.

Change-Id: Iafc1d8c5b62562c9af8fb9fd8c5e1d61270536e7
96faf5b363d922ae91cf25404dee0e87c740c7c5 10-Aug-2013 Ian Rogers <irogers@google.com> Uleb128 compression of vmap and mapping table.

Bug 9437697.

Change-Id: I30bcb97d12cd8b46d3b2cdcbdd358f08fbb9947a
(cherry picked from commit 1809a72a66d245ae598582d658b93a24ac3bf01e)
468532ea115657709bc32ee498e701a4c71762d4 05-Aug-2013 Ian Rogers <irogers@google.com> Entry point clean up.

Create set of entry points needed for image methods to avoid fix-up at load time:
- interpreter - bridge to interpreter, bridge to compiled code
- jni - dlsym lookup
- quick - resolution and bridge to interpreter
- portable - resolution and bridge to interpreter

Fix JNI work around to use JNI work around argument rewriting code that'd been
accidentally disabled.
Remove abstact method error stub, use interpreter bridge instead.
Consolidate trampoline (previously stub) generation in generic helper.
Simplify trampolines to jump directly into assembly code, keeps stack crawlable.
Dex: replace use of int with ThreadOffset for values that are thread offsets.
Tidy entry point routines between interpreter, jni, quick and portable.

Change-Id: I52a7c2bbb1b7e0ff8a3c3100b774212309d0828e
(cherry picked from commit 848871b4d8481229c32e0d048a9856e5a9a17ef9)
1809a72a66d245ae598582d658b93a24ac3bf01e 10-Aug-2013 Ian Rogers <irogers@google.com> Uleb128 compression of vmap and mapping table.

Bug 9437697.

Change-Id: I30bcb97d12cd8b46d3b2cdcbdd358f08fbb9947a
848871b4d8481229c32e0d048a9856e5a9a17ef9 05-Aug-2013 Ian Rogers <irogers@google.com> Entry point clean up.

Create set of entry points needed for image methods to avoid fix-up at load time:
- interpreter - bridge to interpreter, bridge to compiled code
- jni - dlsym lookup
- quick - resolution and bridge to interpreter
- portable - resolution and bridge to interpreter

Fix JNI work around to use JNI work around argument rewriting code that'd been
accidentally disabled.
Remove abstact method error stub, use interpreter bridge instead.
Consolidate trampoline (previously stub) generation in generic helper.
Simplify trampolines to jump directly into assembly code, keeps stack crawlable.
Dex: replace use of int with ThreadOffset for values that are thread offsets.
Tidy entry point routines between interpreter, jni, quick and portable.

Change-Id: I52a7c2bbb1b7e0ff8a3c3100b774212309d0828e
7934ac288acfb2552bb0b06ec1f61e5820d924a4 26-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint whitespace/comments issues

Change-Id: Iae286862c85fb8fd8901eae1204cd6d271d69496
c50d8e11a098cc5c6239aa86b47d4fcf8cbb4899 24-Jul-2013 Brian Carlstrom <bdc@google.com> Remove OatWriter buffering to memory for ElfWriterQuick

This allows the oat contents to be directly written to the file.

Change-Id: Ibc7ddf57477b152f07784b52f7334be73fd22833
3e3d591f781b771de89f3b989830da2b6ac6fac8 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint build/namespaces issues

Change-Id: I19c68703270c1482d6c6aad8cdf97d3d2924360a
2ce745c06271d5223d57dbf08117b20d5b60694a 18-Jul-2013 Brian Carlstrom <bdc@google.com> Fix cpplint whitespace/braces issues

Change-Id: Ide80939faf8e8690d8842dde8133902ac725ed1a
7940e44f4517de5e2634a7e07d58d0fb26160513 12-Jul-2013 Brian Carlstrom <bdc@google.com> Create separate Android.mk for main build targets

The runtime, compiler, dex2oat, and oatdump now are in seperate trees
to prevent dependency creep. They can now be individually built
without rebuilding the rest of the art projects. dalvikvm and jdwpspy
were already this way. Builds in the art directory should behave as
before, building everything including tests.

Change-Id: Ic6b1151e5ed0f823c3dd301afd2b13eb2d8feb81