History log of /art/dexlayout/dexlayout.cc
Revision Date Author Comments
8f4b056427a9d2321e3aa4f21ca8ffb18b3e5ae6 02-Mar-2018 David Sehr <sehr@google.com> Move most of runtime/base to libartbase/base

Enforce the layering that code in runtime/base should not depend on
runtime by separating it into libartbase. Some of the code in
runtime/base depends on the Runtime class, so it cannot be moved yet.
Also, some of the tests depend on CommonRuntimeTest, which itself needs
to be factored (in a subsequent CL).

Bug: 22322814
Test: make -j 50 checkbuild
make -j 50 test-art-host

Change-Id: I8b096c1e2542f829eb456b4b057c71421b77d7e2
Merged-In: c431b9dc4b23cc950eb313695258df5d89f53b22

(cherry picked from commit c431b9dc4b23cc950eb313695258df5d89f53b22)
b2ec9f5c128673c43f776cbe12c8eeb0a6884ebb 21-Feb-2018 David Sehr <sehr@google.com> Remove duplication, split tests

The code move to libdexfile/dex/descriptors_names.cc apparently did not
remove the original code from runtime/utils.cc. Fix that duplication
and all the header mentions needed. Also, split the test files to go
along with the new locations for the code to be tested.

Bug: 22322814
Test: make -j 50 checkbuild
make -j 50 test-art-host-gtest
flash & boot marlin

Change-Id: Ie734672c4bca2c647d8016291f910b5608674545
05f90d133d288b524eaa9deacdc56e7d29ea9f99 07-Feb-2018 Mathieu Chartier <mathieuc@google.com> Avoid generating cdex for dex files with duplicate methods

For the cases where two different method have the same code item or
debug info, avoid generating cdex and write the input dex into the
oat file isntead.

Test: test-art-host-gtest
Bug: 72997076
Bug: 72100771
Bug: 72100668
Bug: 63756964

Change-Id: I5dfe2b47b2c908921c8421367536c2d950186ac6
7517555480074dcbd5c6d9472d5a394ce0e88023 25-Jan-2018 Mathieu Chartier <mathieuc@google.com> Add dexlayout class filtering

For fixed_up_dexfile, the returned dex only needs to contain a single
class. This CL adds filtering during IR creation to only contain that
class. This speeds up wifi tests from 25s to 15s on marlin.

Bug: 63756964
Bug: 72064989
Test: mm test-art-host-gtest-dexlayout_test -j64
Change-Id: I3648ac886b377d95e33e5b40a63a34c2a9dbde58
c3a22aa19bbe35ff8447460b29e07d42937a39de 20-Jan-2018 Mathieu Chartier <mathieuc@google.com> Add shared separate data section for compact dex

Added a shared dex data buffer for compact dex files, this buffer
is referenced by all compact dex files in a vdex file. Repurposed
the existing data_off / data_size fields in the header.

After the shared buffer is filled up, it is placed after the dex
files in the oat writer and the dex file headers are fixed up to have
the correct offsets / sizes to the shared buffer.

Motivation:
Make it easy to deduplicate data across dexes.

Bug: 63756964
Test: test-art-host
Change-Id: I17855a0c78b20be3d323d12dedb9c695962be3ed
e6b6ff89e3cac4e443e731fbcdd957e32c7bc189 20-Jan-2018 Mathieu Chartier <mathieuc@google.com> Revamp dexlayout writing functions

Added some stream abstractions to make it easier to split apart
the main part of the dex file compared to the data one.

Moved to using a vector based stream instead of a mem_map one.

Motivations:
Make it easy to separate data into a separate stream for multidex data
deduping.
Remove upper bounds on dex expansion that would SIGSEGV if the writer
went over the limit.

Bug: 72051652
Bug: 63756964
Test: test-art-host

Change-Id: Ic622a4142b161566d149166a1767434ff4cc7fec
221d9810aa8af1ceab4626620a81baf0fd8377d7 23-Jan-2018 Andreas Gampe <agampe@google.com> ART: Use libbase for logging in command-line tools

Use LOG for logging, and StderrLogger to redirect all logging to
the terminal.

This only applies to tools used only on the command-line.

Test: m test-art-host
Change-Id: Ia3a6363a06c6a849eb1068213962d686c4495e29
8740c66cbfbeb3b7f306279c843650c3b18f2dca 11-Jan-2018 Mathieu Chartier <mathieuc@google.com> Use efficient encoding for compact dex code item fields

Use a compact format for compact dex code items. In the common case,
there are only 4 bytes worth of fields.

This is accomplished by shrining the fields and adding an optional
preheader before each code item that describes fields that don't fit
in the fast path case. The preheader may use up to 12 extra bytes.
Experiments showed that the preheader is only non empty for ~1% of
code items.

CompactDex code item:
0-12 bytes: Optional preheader
<Code item pointer>
4 bits: register size
4 bits: ins size
4 bits: outs size
4 bits: tries size
11 bits: num code item units
5 bits: preheader flags
[Dex Instructions]

Standard dex code item:
See: https://source.android.com/devices/tech/dalvik/dex-format
<Code item pointer>
16 bits: register size
16 bits: ins size
16 bits: outs size
16 bits: tries size
32 bits: num code item units
[Dex Instructions]

Also change code item alignment to be 2 byte aligned for code items
that don't have payload instructions that require 4 byte alignments.

Results (on golem).
The CL reduces vdex size by an average of 4.3%. Interpreter performance
is 0.3-1% slower, but may be noise.

Test: test-art-host

Bug: 63756964

Change-Id: I836fd098063dd1573a3dfb6ac465a6d33d2cb2c9
1557778b03b02b589283100ee8de821692b88813 16-Jan-2018 Mathieu Chartier <mathieuc@google.com> Add more extra space for dexlayout writing

Temporary fix to prevent aborts that can happen when converting very
small dex files to compact dex. Fixes a broken build.

Test: make
Bug: 63756964

Change-Id: I8fce5ebd6d6659d740b100890c825a1899575c31
013fd8073f3ece22b0bba1853d3f3430c8a9e4bd 12-Jan-2018 David Sehr <sehr@google.com> Create an ART-independent DexFileLoader

Opening DEX files should not rely on instantiating a runtime or having a
large number of dependencies on runtime components. This CL makes
DexFileLoader a stub class that is independent of ART, and introduces a
subclass ArtDexFileLoader that contains the current implementations.

Bug: 22322814
Test: make -j 50 test-art-host
Change-Id: Ia6e92ae93c347057ea0c10455525239cbbe42c03
21cf2581844c478db3627d3e0259c205bce76ae7 09-Jan-2018 Mathieu Chartier <mathieuc@google.com> Convert to StandardDex in fixup_dex_dex_file

In fixup_dex_file, run dexlayout to convert compact dex back to
standard dex since the output is supposed to be standard dex.

Fixed a bug in dexlayout where conversion from compact dex ->
standard dex was not supported becuase the dex header was always
preserved.

Test: test/testrunner/testrunner.py --host ---redefine-stress --compact-dex-level fast
Bug: 63756964

Change-Id: Id13e8593458213e7040d053c35e8914404ae5381
9e734c7ab4599d7747a05db0dc73c7b668cb6683 05-Jan-2018 David Sehr <sehr@google.com> Create dex subdirectory

Move all the DexFile related source to a common subdirectory dex/ of
runtime.

Bug: 71361973
Test: make -j 50 test-art-host
Change-Id: I59e984ed660b93e0776556308be3d653722f5223
2c4b084bf93ddfea6b60d05ff82c44dab9de9f28 13-Dec-2017 Mathieu Chartier <mathieuc@google.com> Fix cdex bugs to enable ART_DEFAULT_COMPACT_DEX_LEVEL=fast tests passing

Compute dex checksum for compact dex generation. Handle input vdex
by not aborting in oat_writer, instead just avoid generating compact
dex for the input vdex case.

Re-enabled some compact dex tests.

Bug: 63756964
Test: ART_DEFAULT_COMPACT_DEX_LEVEL=fast test-art-host
Change-Id: Ic9b4e4e59e6cd22b66ee2fc0d32c9b4a15f13497
219cb9021fa74af7773066ffb8fc77ac85f3d0de 07-Dec-2017 Vladimir Marko <vmarko@google.com> Reduce memory allocations in dexlayout's CodeFixups.

Using heaptrack for the services.odex compilation of
aosp_taimen-userdebug:
- before: calls to allocation functions: 5032880
- after: calls to allocation functions: 4645378

Test: Rely on TreeHugger
Bug: 70331349
Change-Id: I477d5a84e0bf22994ce3de9fe56525de0b498801
57943810cfc789da890d73621741729da5feaaf8 07-Dec-2017 Andreas Gampe <agampe@google.com> ART: Replace base/logging with android-base/logging

Replace wherever possible. ART's base/logging is now mainly VLOG
and initialization code that is unnecessary to pull in and makes
changes to verbose logging more painful than they have to be.

Test: m test-art-host
Change-Id: I3e3a4672ba5b621e57590a526c7d1c8b749e4f6e
3e0c5170c87b39c88c7f3b5fbd7b07e60547bfd7 12-Nov-2017 Mathieu Chartier <mathieuc@google.com> Add logic for calculating offsets during writing

Delete all the offset related layout logic, move the logic to
writing. Layout is now controlled by changing the order of the data
vectors.

Calculate all offsets during writing instead of reading. This allows
more flexible layout optimizations without needing to be as careful
about offsets in other sections changing.

Maintained logic for using existing offsets in the case where
changing the layout is not required.

Areas to improve:
How to size the memmap, do we want 2 passes? Currently the algorithm
reserves 10% extra space in case required. This is sufficient for top
1000 apps in the play store. Will consider a two pass approach later.

Bug: 63756964
Bug: 68948395
Bug: 68867570
Bug: 68864106
Bug: 68864168

Test: test/testrunner/testrunner.py --host -j40
Test: test-art-host-gtest
Test: Dexlayout speed profile top 1000 apps

Change-Id: I7dee869da3a010559547f8cfdf93e9aa4c3f47ff
2b2bef245d5b2c6faa2d6f36da14866b2d8f5e4f 27-Oct-2017 Mathieu Chartier <mathieuc@google.com> Refactor DexInstructionIterator

Add a way to get the dex PC for the "for each" use case.

Bug: 67104794
Test: test-art-host
Change-Id: I144c459c9a2a03ec8d56842280338d1f7ce1caf0
2ef3b88203961af0022540ea5893bfe91cab14b7 21-Oct-2017 Mathieu Chartier <mathieuc@google.com> Re-generate dex IR to verify dex file

Instead of using the existing IR, re-generate it to catch possible
bugs writing the output dex.

Test: test-art-host
Bug: 63756964
Change-Id: Ib63f3137f5807ee5de1b05f8278137a3b3a3aee0
24066ec509949f777a4c6e61ef631de33bfdc28a 22-Oct-2017 Mathieu Chartier <mathieuc@google.com> Re-enable dex verifier after dexlayout

Disabled class def ordering since it violates the spec regarding
superclasses occurring before subclasses. This fixes a dex verifier
failure.

Adjust the data section size based on string data and code item
diff. This fixes dex2oat_image_test failing dex file verification.

Fixed handling of unreference annotations. Previously, these would
not get written back out. This resulted the dex file verifier
prematurely ending for annotations during CheckIntraSectionIterate
and then complaining about non zero padding (that was actually an
annotation).

Test: test-art-host
Bug: 63756964
Bug: 68208404

Change-Id: Id2b9b360640c360ac562826e9193971f7bb30eea
b4156286f889c7dea1fc3e3db85ecfcc53880ac3 20-Oct-2017 David Sehr <sehr@google.com> Handle de-duped CodeItems in dexlayout

De-duplicated (shared) CodeItems were added to the layout structures
multiple times, resulting in multiple update of offsets and
uninitialized data in the resulting dex files.

Bug: 67024225
Test: make test-art-host
Change-Id: I76ffd2cb52fe72eab3fb0dbf6ad8607951cee595
5929beb4994f55da3afdb58298225094f959976e 21-Oct-2017 Mathieu Chartier <mathieuc@google.com> Make libdexlayoutd actually have art debug flags

Previously it didn't, this caused DCHECKs to not happen. Disabled
dex verifier for dexlayout since it fails for dex2oat_image_test
now that DCHECKs are actually run.

The failure is:
Check failed: output_dex_file != nullptr Failed to re-open output
file:Failure to verify dex file 'memory mapped file for
.../core-oj-hostdex.jar': Out-of-bounds end of data subsection:
4c56d1

Test: test-art-host
Bug: 63756964
Change-Id: I05a6617ffdf24fd45024a13576c2a5ff1fcd84a8
603ccabb82511824a2a9d9531397005009a0e28a 20-Oct-2017 Mathieu Chartier <mathieuc@google.com> Add basic support for writing cdex container in dexlayout

Pass compact dex level option to dexlayout and generate a file with
a compact dex header if it is enabled.

Added test in dex2oat_test.

Added logic to check for CompactDexFile header in
DexFileLoader::OpenCommon.

Bug: 63756964
Test: test-art-host-gtest-dex2oat_test

Change-Id: Icfec03cdb64f71de900b92754ab4be3789c71cc3
095c6c96236476b605b3ac672f6d2b8c151e9479 19-Oct-2017 Nicolas Geoffray <ngeoffray@google.com> Pass the verify flag to DexFileLoader utilities.

Refactoring-only change.

Also remove unused DexFilerLoader::OpenMemory.

bug: 30972906
bug: 63920015

Test: build.

Change-Id: I7cd4b5787565ab1a3457ce2be6bb14657229c550
aa6abb081cbe14ccd693459bf016b14627f58616 12-Oct-2017 David Sehr <sehr@google.com> Fix dexlayout debug info printing

Parsing local info and position info made shared debug info print
incorrectly. Remove the dexlayout structs that represent the pre-parsed
versions and print from the byte buffer.

Bug: 67664147
Test: make test-art-host
Change-Id: Iae33ae8ff486914d8d7a5973f81faaf3c95615a6
79c87da9d4698ec58ece65af0065eebd55a1cfe0 10-Oct-2017 Mathieu Chartier <mathieuc@google.com> Add DexFileLoader class

Added DexFileLoader class, moved functionality from DexFile there:
- Multidex loading logic
- DexFile opening logic for Zip and etc
- Some other helpers

Bug: 63756964
Test: test-art-host

Change-Id: Ic3dfa458947d4b69912dea5cdd836e7e8f55061c
1d2d4ff8570bb88d9d2d4633706fd7f6fb18d75e 24-Sep-2017 Mathieu Chartier <mathieuc@google.com> Add DexInstructionIterator and use it a few places

Motivation:
Want to start abstracting away dex specific functionality to enable
CompactDex. Adding an iterator will enable CompactDex iteration to
work differently than normal dex iteration.

Will eventually replace CodeItemIterator.

Bug: 63756964
Test: test-art-host

Change-Id: I90e67c1a994b7698aaac0523a82816b0a003fbdc
e2abbc604ce003c776c00ecf1293796bb4c4ac5a 15-Sep-2017 Andreas Gampe <agampe@google.com> ART: Move kDexNoIndex to dex_file_types.h

Define the constant with the types to allow lowering the dependency
on DexFile.

Test: m
Change-Id: I3c61421db45be96d2057e01b1a7825883d8bd178
120aa286ab6adf3e76a31bc61fb4e583e5158d71 06-Aug-2017 Mathieu Chartier <mathieuc@google.com> Store layout info in dex files

Store layout info for code sections inside of the oat file. This will
be used to advise the kernel when dex files are loaded in
a follow up CL.

Added unit test in dex2oat_test.

Bug: 63178181
Test: test-art-host

(cherry-picked from commit 75c5ed6e75f70002db5fa7c609137c04dd2bdf40)

Change-Id: I4777506886bde42ff0affdac412a8395e8013a40
75c5ed6e75f70002db5fa7c609137c04dd2bdf40 06-Aug-2017 Mathieu Chartier <mathieuc@google.com> Store layout info in dex files

Store layout info for code sections inside of the oat file. This will
be used to advise the kernel when dex files are loaded in
a follow up CL.

Added unit test in dex2oat_test.

Bug: 63178181
Test: test-art-host

Change-Id: I4777506886bde42ff0affdac412a8395e8013a40
8cf9cb386cd9286d67e879f1ee501ec00d72a4e1 19-Jul-2017 Andreas Gampe <agampe@google.com> ART: Include cleanup

Let clang-format reorder the header includes.

Derived with:

* .clang-format:
BasedOnStyle: Google
IncludeIsMainRegex: '(_test|-inl)?$'

* Steps:
find . -name '*.cc' -o -name '*.h' | xargs sed -i.bak -e 's/^#include/ #include/' ; git commit -a -m 'ART: Include cleanup'
git-clang-format -style=file HEAD^
manual inspection
git commit -a --amend

Test: mmma art
Change-Id: Ia963a8ce3ce5f96b5e78acd587e26908c7a70d02
acc83d77cc99e6280391604ebd3e2e51d9a60b99 07-Jul-2017 Jeff Hao <jeffhao@google.com> Improve string layout in dexlayout.

Adds more strings to what dexlayout considers hot:
- A profiled class's superclass and interface class names
- The class names of accessed fields
- The class name, name, and shorty of methods accessed by clinits

TODO: Gather memory numbers

Bug: 35800981
Test: mm test-art-host
Change-Id: I1a7de5bd5b1b73559624af939de05a2111f78195
e46f3a8399fec0f2cbb6d730f4f8316c37d7f3c5 20-Jun-2017 Mathieu Chartier <mathieuc@google.com> Address some review comments for aog/415919

Test: test-art-host
Bug: 62040831
Change-Id: I9d8c359ab3a1ff090fe664bbb798a215a3d9eb52
7c1be8ba84b81ad2edfb6461bb4e2f3073bbc553 15-Jun-2017 Mathieu Chartier <mathieuc@google.com> Layout code items of post startup methods

Layout code items in the following manner:
<hot>
<startup only>
<other clinit>
<executed>
<not executed>

For basic use cases (write email, start navigation)
Maps vdex PSS: 17464k -> 16856k
Gmail vdex PSS: 4004k -> 3820k
Numbers are from marlin

Test: test-art-host
Bug: 36457259

Change-Id: Id7ae676b45d5ba4f5b4dabfdc954875756fd2bcb
bbe3a5efcdfb2fb15a4e5f724cc323ae0ada5111 14-Jun-2017 Mathieu Chartier <mathieuc@google.com> Refactor some profiling info functions

Deleted profile_compilation_info-inl.h

Added a new helper class MethodHotness to reduce the number of
required getters for reading method hotness, startup, and post
startup booleans.

Test: test-art-host
Bug: 62040831

Change-Id: I799a38e2bea6177ba5816c54524f4ccacedba772
206cbaabf42430347f8d83e7b9e8fd9f0d29d010 08-Jun-2017 Jeff Hao <jeffhao@google.com> Layout clinit code items of profiled methods together.

Maps startup dex file pss drops from 9.88MB -> 9.12MB.

Bug: 35800981
Test: mm test-art-host
Change-Id: I5c0167c553cd21c23bed05e824f14ab19f505d59
ea650f3bc4f54eb2c647cf0f7134398ef5038792 24-May-2017 Mathieu Chartier <mathieuc@google.com> Move profiles to use bitmaps for startup methods

This CL enables changes like compiling only hot methods while still
doing layout for hot and startup methods. The bitmaps are also a
bit smaller for post-launch use cases.

No change in compilation strategy yet.

Fixed some bugs in dexlayout test like using a profile with the wrong
dex location. This meant the second invocation of dexlayout didn't
have any profile data.

Added profman support for dump-classes-and-methods,
create-profile-from, and related test.

Profile sizes (bytes) post launch:
Gmail: 7290 -> 6136
Maps: 22896 -> 18984
Music: 8582 -> 7050
YouTube: 16733 -> 14592

Test: test-art-host

Bug: 62040831

Change-Id: I9915b81a2ff2c47464acbbdeb55ce30a33d5483f
a07394a0f356513a08266e2e198a0f4064f6e694 17-May-2017 Jeff Hao <jeffhao@google.com> Enable dex diff check on debug builds.

Dex diff checks that the input and output dex files are equivalent at
the IR level. Normally enabled by passing -v to dexlayout, dex diff is
now enabled for debug builds as well.

Bug: 36107940
Test: mm test-art-host
Change-Id: I8c5ebb6a7cc0f699299a7200e8bc019762601feb
e1f6cece72bf6e6ff2328100d919c187f39129c7 03-Apr-2017 Shubham Ajmera <shubhamajmera@google.com> Layout codeitems based on profile data

Bug: 35800981
Bug: 37287151 (TL triaged bug dependent on this)
Test: ./art/test/testrunner/testrunner.py --host -t 638-c
Change-Id: I7beef8aae2d085333e0c2bfc0706c7f42f8e5f5f

(cherry-picked from commit 36a282b8714071d9f16c98c316efa4304fd8d36a)
58f4dcdb421603ea83f3c81ddc4af84ee458410c 24-Apr-2017 Andreas Gampe <agampe@google.com> ART: More header cleanup - mem_map.h

Move MemMap maps_ into cc file and move allocator include. Remove
other unused includes. Force clients to import sys/mman.h if they
need it. Try to lower dependencies on ART by using libbase helpers
more.

Test: mmma art
Change-Id: I8f31e5f371789b3d72e12af377181fee2ac9a611
(cherry picked from commit 0dfc315a096bf5678cb5182771e3fdcca8ea4433)
36a282b8714071d9f16c98c316efa4304fd8d36a 03-Apr-2017 Shubham Ajmera <shubhamajmera@google.com> Layout codeitems based on profile data

Test: ./art/test/testrunner/testrunner.py --host -t 638-c

Change-Id: I15cef5d8bd25cbb74f844bd4c8241700b0f7dc13
0dfc315a096bf5678cb5182771e3fdcca8ea4433 24-Apr-2017 Andreas Gampe <agampe@google.com> ART: More header cleanup - mem_map.h

Move MemMap maps_ into cc file and move allocator include. Remove
other unused includes. Force clients to import sys/mman.h if they
need it. Try to lower dependencies on ART by using libbase helpers
more.

Test: mmma art
Change-Id: I8f31e5f371789b3d72e12af377181fee2ac9a611
7639cdcf4dbbbb4ef63cc8c3ae2f2b4e170a5c83 15-Apr-2017 David Sehr <sehr@google.com> Add art_defaults to dexlayout

The build rule defaults for dexlayout targets didn't include defaults
from the ART project. Because of this ART_TARGET_* weren't defined,
among other problems

Bug: 35800981
Test: make -j 40 test-art-host
Change-Id: Ic528bfc6b622fa9b4f5c0ddf883f15912641b720
3ba51e854d6ee3287641d37ca5e108dd78de697b 13-Apr-2017 Jeff Hao <jeffhao@google.com> Add check that dexlayout can open output file.

Test: mm test-art-host
Change-Id: I49b37b3e3696c8c002e22533c2e86e47a0e13e51
49e9632340eec3758e0a356b0e5ad9e7c44a0d4f 28-Mar-2017 Mathieu Chartier <mathieuc@google.com> Layout string data

Inspect dex code in class initializers and hot methods to find out
what const strings are likely to be resolved. Along with these,
look at static field accesses and group the field name and type with
other hot strings.

Group method shorties together.

Maps vdex PSS: 7478k -> 6900k
No change in odex PSS.

Changed dexlayout_test to use -v instead of comparing to a hex
output.

Bug: 36457259
Bug: 35800981

Test: test-art-host

(cherry picked from commit fa0aa0953875427d7381ac44a6427bd5b1d5020a)

Change-Id: Iaa5de649c5b2b1d9f178525d2f42fe00e3fb879f
fa0aa0953875427d7381ac44a6427bd5b1d5020a 28-Mar-2017 Mathieu Chartier <mathieuc@google.com> Layout string data

Inspect dex code in class initializers and hot methods to find out
what const strings are likely to be resolved. Along with these,
look at static field accesses and group the field name and type with
other hot strings.

Group method shorties together.

Maps vdex PSS: 7478k -> 6900k
No change in odex PSS.

Changed dexlayout_test to use -v instead of comparing to a hex
output.

Bug: 36457259
Bug: 35800981

Test: test-art-host

Change-Id: Iaa5de649c5b2b1d9f178525d2f42fe00e3fb879f
37c5846be19f28bd629ffe7d6cfde779abbccb89 28-Mar-2017 Andreas Gampe <agampe@google.com> ART: Fix some tidy warnings

Fix some tidy warnings for macros and references.

Test: m test-art-host
Change-Id: I2f8ed2174634263d4784c08e637f60ed70977b73
cc8295946caf3b965bff778fd7ae3adbcba47de3 15-Mar-2017 Jeff Hao <jeffhao@google.com> Finish adding checks to dex ir verification.

Adds missing checks for class defs, annotations, class data, code, etc.

Bug: 36107940
Test: mm test-art-host
Change-Id: Ifaac07299cf237827160c29b4230b0dc0845dc41
ec7f1a96683836d9375adc48c6bf802c0ce2160e 14-Mar-2017 Jeff Hao <jeffhao@google.com> Added new -v option to dexlayout to verify output dex file.

Passing -v will cause dexlayout to do an IR level comparison of the
output with the original input dex file. This checks that the data in
the dex files are the same, but allows for different offsets since the
output dex file may have a different layout.

Test: mm test-art-host
Bug: 36107940
Change-Id: If75a93973ffdd2d91111727f089713c800d8cee8
4a436ac4b4daa4e322d2a7cc2395462fbb16cd54 11-Mar-2017 Jeff Hao <jeffhao@google.com> Fix dex file verification check for dexlayout output file.

For debug builds, we want to run the output dex file through the dex
file verifier as a sanity check. The initial check was actually checking
the input dex file again...

Bug: 36107940
Test: mm test-art-host
Change-Id: Ibd434d67f661c325a443d4ca9cced481e7a9b02c
9335749d25f8826c72ea7ea4d470ee8bc171fa87 09-Mar-2017 David Sehr <sehr@google.com> Add section size statistics

Add an option to display the size of dex file sections for easier
analysis.

Bug: none
Test: test-art-host

Change-Id: I779f776657ad79e7a50c18c2b9486e6778c70f13
b756815e581e2bcc3db7ee0cb3ae6c031fb22ae8 10-Mar-2017 Jeff Hao <jeffhao@google.com> Add dex file verification check for dexlayout output file.

For debug builds, run the output dex file of dexlayout through the dex
file verifier as a sanity check.

Bug: 36107940
Test: mm test-art-host
Change-Id: I2b556db32cfd360cc4fe8d1d080508deebd27aa0
863f1d79a70403443c089a91ca893db6cbadd3d8 01-Mar-2017 Jeff Hao <jeffhao@google.com> Prevent dexlayout from moving code items if preceded by class data.

Class data has uleb encoded offsets to the code item, and moving code
items will change the size of the class data, which would in turn affect
the code item offsets if they precede the code items.

For now, make it so that dexlayout does not move the code items at all
in this case. A better fix would be to swap the order of the sections in
dexlayout, but that can be done in a future CL.

Unit test to follow.

Bug: 35855748
Test: mm test-art-host
Change-Id: Ica6da1e5e951cf0003fc9793f13ad10b74004eb9
e17f589d8fa46d0d0179499354b7967801f9582c 24-Feb-2017 Jeff Hao <jeffhao@google.com> Dexlayout fix for duplicate class data and preserving last code item.

This change fixes a dexlayout issue when multiple class defs reference
the same class data. A set is used to tell what class data has already
been visited.

This change also makes dexlayout preserve the offset of the last code
item in the dex file if the following section is not 4-byte aligned.
Due to limitations in dexlayout, it is difficult to adjust the offsets
of sections if more space is needed. An overhaul of dexlayout should
be done to properly fix this.

Bug: 35451910
Test: mm test-art-host-gtest-dexlayout_test
Change-Id: I8d066c2151a1a57c382ce35f12bf53a519da89f6
33083d626dbf2c8b06badfd73f50e98114483059 19-Jan-2017 Calin Juravle <calin@google.com> Rename offline_profiling_info to profile_compilation_info

This solves a long standing TODO to align the names of the files and the
class.

Bug: 32434870
Test: test-art-host

Change-Id: I2707da8fef8736a7223189d894fc00ff11bfab12
46ee31b67d7ee1bd085fbc240502053caa3cf8fa 14-Dec-2016 Andreas Gampe <agampe@google.com> ART: Move to libbase StringPrintf

Remove ART's StringPrintf implementation. Fix up clients. Add
missing includes where necessary.

Test: m test-art-host
Change-Id: I564038d5868595ac3bb88d641af1000cea940e5a
042e898f18767ed326980e3b07d39adbf98f6222 19-Oct-2016 Jeff Hao <jeffhao@google.com> Add layout scheme to dexlayout for dex file reordering.

Currently takes startup classes from profile and groups them together
with the lowest class indexes. Moves the ClassDef, ClassData, and
CodeItem sections.

Results gathered from 5 second systrace of app startup (no other input):
Maps
base.vdex - 24.70MB -> 19.35MB (30.50MB file size)
base.odex - 13.22MB -> 12.82MB (13.22MB file size)
Running time of dex2oat: 27.5s -> 37.1s

Photos
base.vdex - 12.98MB -> 9.77MB (15.10MB file size)
base.odex - 6.84MB -> 6.24MB ( 6.84MB file size)
Running time of dex2oat: 15.6s -> 19.9s

Plus
base.vdex - 14.09MB -> 13.29MB (16.72MB file size)
base.odex - 2.47MB -> 2.47MB ( 2.47MB file size)
Running time of dex2oat: 14.1s -> 18.5s

Performance and memory tracking bug: b/33017139

Test: mm test-art-host-gtest-dexlayout_test
Bug: 29921113
Change-Id: Ib500ed353d71a606e2db0dd80378750c5f7b8e33
ea7c62983beec6a5a2a6676cc910a436b20ae92c 15-Nov-2016 Jeff Hao <jeffhao@google.com> Dexlayout cleanup and refactoring.

Created option to output to a mem map in preparation of hooking
dexlayout into dex2oat.

Test: mm test-art-host-gtest-dexlayout_test
Bug: 29921113
Change-Id: Id42ef15cb8f83cc8d05b025b7647a4338e9b96b0
ca620d7bc03b23a0bcf0ef58df58603ee000dca0 08-Nov-2016 Andreas Gampe <agampe@google.com> ART: Fix tidy warnings

Switch to char versions of find variants.

Add "explicit" constructor variants or refactor and
remove defaults.

Use const references.

Bug: 32619234
Test: m test-art-host
Change-Id: I970cc2f47d6cf8f0c74104b994b075b2fafb3d45
caacd11864383aac65e61be837fb1bb5f91e3878 21-Oct-2016 David Sehr <sehr@google.com> Move dex CFG dumping out of utils.cc

Move CFG dumping to dexdump, the only client.

Bug: 22322814
Test: test-art-host
Change-Id: I0f39f1d5dfc446419d26d709b78d04e45616f42c
b34bb193dd2d45af196592bb4bae26f30d529b13 18-Oct-2016 Orion Hodson <oth@google.com> Dex disassembly suppport for invoke-polymorphic.

Bug: 30550796
Test: art/test/dexdump/run-all-tests
Change-Id: I013ce2ebbcf9555e01170dc47fc38036c276b1b4
a86210038524cecd0d96d6ba6f8f116da348a25c 04-Oct-2016 Jeff Hao <jeffhao@google.com> Revert "Revert "Add dex file writer to dexlayout tool.""

This reverts commit fd1a6c2a08ca3e2476b7424b9b0fa58e73b29e87.

Fixed output being clobbered during DexLayoutTest.DexFileOutput.
Option added to put dex output file in scratch directory.

Bug: 29921113
Test: mm test-art-host-gtest-dexlayout_test

Change-Id: I9e6b139cf06aaa39c83ad1e74329db266464a8e4
cdcfde74ccd72c60dc9456013e32d8b00fefdee6 26-Sep-2016 David Sehr <sehr@google.com> Option to gnuplot dex layout

Display the portions of the dex file that are accessed by the classes in
the dex file. Optionally limit the display to only those classes that
are present in a runtime profile.

Bug: 29921113
Change-Id: I2a998fba448fec6c23941d3b4358531b1336e1b3
Test: dexlayout test
fd1a6c2a08ca3e2476b7424b9b0fa58e73b29e87 04-Oct-2016 Nicolas Geoffray <ngeoffray@google.com> Revert "Add dex file writer to dexlayout tool."

Bug: 29921113

This reverts commit 69b58cf399c3507ca3debf9ec9b2db3c5a0fe7f2.

Change-Id: I8bf9b1222d99489059e45f47d220396a853170a4
69b58cf399c3507ca3debf9ec9b2db3c5a0fe7f2 23-Sep-2016 Jeff Hao <jeffhao@google.com> Add dex file writer to dexlayout tool.

All sections are being written out properly.
There is one small difference:
- unindexed string data is not written out

Bug: 29921113
Test: mm test-art-host-gtest-dexlayout_test
Change-Id: I3872327526a350f33f0f851516a389622c1a54ae
3ab96b460684781af58b3f9c5b9fb0e370de6b50 10-Sep-2016 Jeff Hao <jeffhao@google.com> Dexlayout changes to support offset/size output of items.

- Created offset & size fields for items, initially populated with
values from original dex file.
- Created index field for indexed items in header.
- Created Item objects for StringData and TypeList.
- Created Collections container to hold unique_ptrs for all objects
- Reorganized EncodedArrayItems to make it easier to collect them
- Added relevant annotations-related items to Collections
- Some size calculations and section offsets left as TODO

Bug: 29921113
Test: mm test-art-host-gtest-dexlayout_test
Change-Id: Id812f104c1930a3ee381e885f353a9a49f29ac9b
72359225691975281f0bd5a0a19b07a0b2e6b498 07-Sep-2016 David Sehr <sehr@google.com> Fix some memory leaks found by valgrind.

Bug: 29921113
Change-Id: If70d475a3317751d206658c5794a32d78bc33e47
Test: valgrind-test-art-host-gtest (with --trace-children=yes)
853a8e10844b9b9e791c6d40e5d332cd38af97d6 01-Sep-2016 David Sehr <sehr@google.com> Separate DexIr building from constructors.

Move all the construction from DexFile out of the constructors of the
basic IR.

Bug: 29921113
Change-Id: I3f79c104ce7183ddde73f143c047061416009a54
Test: test-art-host-gtest
c3acfc5bed5d0cb22bee05e72cdbdec49a182224 29-Aug-2016 Jeff Hao <jeffhao@google.com> Finish dexlayout dumping of dex files and added test.

The test runs dexdump and dexlayout over the first core library jar file
and diffs the output, checking that they're the same. More dex files
will be added later for better coverage.

Bug: 29921113
Test: mm test-art-host-gtest-dexlayout_test
(cherry-picked from commit a378e88db648cf19e771714dd1059cf3d41d4753)

Change-Id: Idfd8ac9f069885b675a4a2f0c70aef2cbef5a9b8
7629f60ffa7b8533ea0b50263d52cc1df72b2904 08-Aug-2016 David Sehr <sehr@google.com> Add dex file intermediate representation

Add an intermediate representation, constructed from the DexFile
utilities, that can be used to optimize the layout and re-emit dex
files.

BUG: 29921113
TEST: TODO: Add dex files to compare output with dexdump

(cherry-picked from commit af14d9f551e6169c201dc7981d9cfbcb53c7d2c7)

Change-Id: I3dacb78fad5228e43d77f20b110bd40e0e116cdb
feeb2bcd0b806ac820033c5347905f5b1f79c04f 29-Aug-2016 Jeff Hao <jeffhao@google.com> Finish dexlayout dumping of dex files and added test.

The test runs dexdump and dexlayout over the first core library jar file
and diffs the output, checking that they're the same. More dex files
will be added later for better coverage.

Bug: 29921113
Test: mm test-art-host-gtest-dexlayout_test
Change-Id: I5528aa09ef2df7acd4b35b321a555acc3952574d
af14d9f551e6169c201dc7981d9cfbcb53c7d2c7 08-Aug-2016 David Sehr <sehr@google.com> Add dex file intermediate representation

Add an intermediate representation, constructed from the DexFile
utilities, that can be used to optimize the layout and re-emit dex
files.

BUG: b/29921113
Change-Id: I62c7a2bf98320eb73e6c7f8ffa0fff0f392bf32d
TEST: TODO: Add dex files to compare output with dexdump