History log of /build/tools/releasetools/ota_from_target_files
Revision Date Author Comments
e8269481084df086a04c6bdcd3477d4209d4a7cb 29-Oct-2014 Michael Runge <mrunge@google.com> Add post-install verification phase

Verify the SHA sum of all patched and extracted files after
a file system remount.

Bug: 18145574

Change-Id: I2f053d085543e10e39153a774542d37ee0a238bd
f9d181979097af3f4a62dad9fff318a916759597 23-Oct-2014 Michael Runge <mrunge@google.com> get_stage in updater takes one param, not two.

Change-Id: I09a77eda5a1691080b1ae074d2e94d233eca7150
65f3282099a16e266bc61e0ed3016e15f6bd8959 23-Oct-2014 Michael Runge <mrunge@google.com> Allow info_dict from target_files to specify mount options

This will allow safer mount options to be added per mount FS
type, to better ensure data is written during an OTA.
Bug: 18079773, 18092222

Change-Id: I1e3e4fd4639c6fd263e550b770cc3c858ef1e03b
560569a617c86be7985bd699e20fd19c6b2715c8 19-Sep-2014 Michael Runge <mrunge@google.com> If oem_fingerprint_props is empty, ignore.

There may be cases where there is an OEM partition
but it has no effect on the OTA itself. In these
cases, ignore an empty value from the misc_info.txt

Change-Id: I5f467e873030765af12810a07ddd5f302ca8cc0b
b34fcce08cc9ab358481e672d83ff16513c4ac37 11-Sep-2014 Doug Zongker <dougz@google.com> explicitly check the superblock for differences

When generating incrementals for the system and vendor partitions,
check the first block (which contains the superblock) of the partition
to see if it's what we expect. If this check fails, give an explicit
log message about the partition having been remounted R/W (the most
likely explanation) and the need to flash to get OTAs working again.

Bug: 17393999
Change-Id: Ifd2132b428dbc4907527291712690204a3664ac0
ab7ca1d2861e70d8ef8673d350b634111414039e 26-Aug-2014 Doug Zongker <dougz@google.com> refactor BlockDifference into common

Move BlockDifference into common and make its script generation code
more complete, so that it can be use by releasetools.py to do diffs on
baseband images.

Bug: 16984795
Change-Id: Iba9afc1c7755458ce47468b5170672612b2cb4b3
fc44a515d46e6f4d5eaa0d32659b1cf3b9492305 26-Aug-2014 Doug Zongker <dougz@google.com> new block OTA system tools

Replace the xdelta/xz-based block OTA generation with a new system
based on the existing bsdiff/imgdiff tools.

Bug: 16984795
Change-Id: Ia9732516ffdfc12be86260b2cc4b1dd2d210e886
f83400896de5e8bc2b21a0dce98382d9ed0ab537 05-Aug-2014 Doug Zongker <dougz@google.com> add 5 minute timeout on binary patch construction

When making bsdiff/imgdiff patches, give up after 5 minutes. (On
certain large files it can take hours to build a patch, if it ever
even completes.)

Change-Id: I123c06f8194f85f6f4e640f7eb31c7746f76ba4d
62d4f18a30aeade677ca814cf6f2aa329cf5066d 05-Aug-2014 Doug Zongker <dougz@google.com> fall back to generating full OTA if incremental fails

Block incremental OTA generation can currently fail on some
target-files pairs. Fall back to generating a full OTA so that the
script succeeds rather than failing.

Change-Id: Ide70395d1f3759aa2076bd173836f6a5e5b397c0
3c84f569487c4e59baa332be33b5430fdefb76b3 31-Jul-2014 Doug Zongker <dougz@google.com> store images in target-files

Store sparse images in the target-files, and use those (when they're
available) for building block OTAs.

- New script add_img_to_target_files is added to make the images and
add them to the IMAGES/ subdir in the target-files. It gets run
from the Makefile when building a target-files.

- img_from_target_files becomes mostly vestigial: it creates the
img.zip by just copying the images out of the target-files. (It
still knows how to build images for use on older target-files.)

- ota_from_target_files uses images from the target-files in
preference to rebuilding images from the source files.

- sign_apk_target_files builds images and includes them in its output
target files (even if the input target-files didn't have them).

Bug: 16488065
Change-Id: I444e0d722d636978209467ffc01750a585c6db75
c8b4e849f10f3a382694b00453b3f49608c83b48 17-Jun-2014 Doug Zongker <dougz@google.com> full support for OTA of vendor partitions

Make vendor partition a first-class member of the OTA system (for
target_files that contain a VENDOR/ subdirectory).

Build vendor images in a way that is compatible with block-based OTA.
Support updating the vendor partition in both full and incremental,
block and file OTAs. In most cases this is handled by refactoring the
existing code to handle the system partition to handle either, and
then calling it twice.

Currently we don't support incremental OTAs from a target-files
without a VENDOR subdirectory to one with one, or vice versa. To add
or remove a vendor partition a full OTA will need to be done.

Bug: 15544685
Change-Id: I9cb9a1267060bd9683a9bea19b43a26b5a43800d
1113e38195da957301e3077151a90c113c86289a 13-Jun-2014 Doug Zongker <dougz@google.com> test block patch more realistically

Read and write the same file when testing block patches, which can
turn up errors that don't show up otherwise. (And will appear on the
device.)

Change-Id: Ic9b8d93ec980d13163b135f619af589f41433d7f
374e114d1691a47918a3972dc287615f25650725 31-May-2014 Martin Blumenstingl <martin.blumenstingl@googlemail.com> Document the worker_threads option.

Change-Id: I44775493bedc0c7224c2c4ef06330bdb1430a1b6
4b9596fe00e3012c8d4b44844ac53b10ee27b579 09-Jun-2014 Doug Zongker <dougz@google.com> configure progress bar during block OTA script generation

Emit script commands to produce a more accurate progress bar (for full
block OTAs) and a working progress bar (for incremental block OTAs) --
ones that are driven by the progress callback from the thing actually
writing the system image.

Change-Id: Ifca10be68cfdaab7135d23515bd0ae5be2f98a16
91a99c28e0d15a753bc303982c970c96d6dfe0f5 09-May-2014 Doug Zongker <dougz@android.com> fix ota script to generate non-block commands correctly

If you had a target_files without a recovery patch and specified
--block (which should have no effect without a patch), it would have
omitted some necessary commands from the OTA script.

Change-Id: I96e79cd561ebf09cfe53792d1cc558cc71479869
c6e3afd26db72bedcb1d89a71769138a731646a5 05-May-2014 Michael Runge <mrunge@google.com> Support block OTA + OEM partition

Change-Id: I9e662098569a43b05279908e6833e9552a7abe3a
6e836116f764cf5cebf1654df2f17d8222554f6e 16-Apr-2014 Michael Runge <mrunge@google.com> Add support for verifying OEM properties.

A separate OEM file must be specified to provide the expected
values for these properties. The list of properties comes from
the "oem_fingerprint_properties" list in misc_info.txt

Bug: b/13367676

Change-Id: I1a3eaf108492132cf6f595a5d1c9f7e0c3cb3142
c73e461537678af2c29fcc38857e26fa57103710 16-Apr-2014 Ying Wang <wangying@google.com> Fix variable name.

Change-Id: Ie7439f71cf7fd2eeacd45a5d6625e610dad8bfbe
(cherry picked from commit 1a1dfcb5fcb6eda11d8dbf5df02f0083853dd5c1)
922206ec2aeea98491e3edfc297a28b0a39df698 04-Mar-2014 Doug Zongker <dougz@android.com> move data wipe to end of OTA package

Now that OTA packages can be downloaded to /data, if they include a
data wipe we should do that last.

Change-Id: I75102fb2ff85d0f0110d55dfca06ec5f38104850
32b527d6cb951335f06c37c4d9a7a8216974a86a 04-Mar-2014 Doug Zongker <dougz@android.com> test block system image patch at build time

After building a patch for the system image (for incremental block
OTAs), apply it to a local copy of the file and test that it succeeds.
This is an imperfect test as it's using the local client's
syspatch_host, which may differ from the syspatch library actually
used in the target build, but it's somewhat better than nothing.

Change-Id: Ic0001b0145881e2ebd4b5b36ce9b5bcebd76deb4
5fad2039bbd4e55d671106c8b39a6e451b85ac23 24-Feb-2014 Doug Zongker <dougz@android.com> handle don't care regions in the system image

The system partitions has regions that we shouldn't write and can't
depend on the contents of. Adds a new script to generate a map of
these regions (using the sparse image as input), and include the map
in the package zip so it can be used when writing or patching the
system partition.

Also fixes a bug where the wrong SELinux file contexts are used when
generating incrementals.

Change-Id: Iaca5b967a3b7d1df843c7c21becc19b3f1633dad
25568486e5777f416d2fcb6cc7aa96caafc66880 03-Mar-2014 Doug Zongker <dougz@android.com> add option to specify updater binary, for development

Change-Id: I5f239afff70c87fb16ddc4b8abefa7bbcda6040d
26e6619c37e294fe2ee63aaa759e0ac861775ce8 20-Feb-2014 Doug Zongker <dougz@android.com> add --block flag to ota_from_target_files

Add the --block flag to this script to control whether block-based OTA
packages are generated (defaults to off). Make the full OTA package
produced by "make otapackage" continue to produce a block-based OTA.

Also fix a problem where block incremental OTAs didn't ever succeed,
and the --no_signing option never worked.

Change-Id: I610d0b4abed4b8b65fbe8ce0abaeec6cf52e14a1
36bd365625e6beba77698a93795a603180a5b476 07-Feb-2014 Geremy Condra <gcondra@google.com> Add support for block incremental OTAs

Change-Id: Ie72015e34ed8d7595a5c74c8df41cba73275afab
cf6d5a90740e50e03d9f16c6fd449d90d396f924 18-Feb-2014 Doug Zongker <dougz@android.com> bump releasetools python requirement to 2.7

These scripts already use some post-2.4 features, so let's make it
official: Python 2.7 is needed to run them.

Change-Id: I256e9ed99b0b62abe4e22a7b1f811acb7419e88e
01ce19c95f358ac8ef9bb939d2637ac976320401 04-Feb-2014 Doug Zongker <dougz@android.com> make full OTAs block based

Instead of writing individual files and fixing up their metadata, make
full OTAs contain a system image and simply write it to the block
device.

This is only done for target-files that already contain the recovery
flashing information, older target-files still get a file-based full
OTA.

Bug: 12893978
Change-Id: If7586083c8f275e24fec49d260af5b5aff4a0a88
c9253822ea31c1d35d3fc2b495b45b476c240a1d 04-Feb-2014 Doug Zongker <dougz@android.com> add recovery update code to system images

Currently, the "img" zip files generated by the build system lack the
script and data needed to rewrite the recovery partition, while the
"ota" zip files do (when installed).

In order to move towards block-based OTAs, we want the result of
flashing an image and the result of installing the corresponding OTA
package to be identical.

Generate the recovery-from-boot patch and install script as part of
the process of building the target-files. This requires breaking the
code to generate that out of ota_from_target_files into its own tool
that we can run from the Makefile. (ota_from_target_files can still
do this, so it continues to work with older target-files.)

Bug: 12893978
Change-Id: I80e62268840780b81216e548be89b47baf81b4ac
eb0a78afc00265479c002364fa62c9e09c3f613d 27-Jan-2014 Doug Zongker <dougz@android.com> prefer releasetools.py from target_files zip

If the target_files zip for the target build contains a
META/releasetools.py (which it has since Nov 2013), prefer that over
using a releasetools.py from the local client.

Explicitly specifying the device-specific extensions path via
command-line options takes priority over both of the above mechanisms.

Change-Id: Ia068b0e2e06ede7da89ebe4315cdec592eb8995e
e153b3464374155d03bfe47092faaab555b89e81 14-Nov-2013 Takeshi Kanemoto <takeshi.kanemoto@sonymobile.com> ota_from_target_files: Add an option to not sign OTA packages

Sometimes it is useful to be able to tell ota_from_target_files
to not sign the output zip file. For instance, the private
release key may not be available when ota_from_target_files
is executed; similarly the release tools may not be available
or executable where the private key is stored.

This change adds an option, '--no_signing', to simply output the
unsigned OTA zip file, instead of spuriously signing it with the
test key even though the zip file would need to be re-signed later
with a different key.

Change-Id: I1f3c4dc8ffa35ce85478f848b147aff3d40fe283
4038aa8fff8c68733bff8b55d8c8d1b59713ca37 14-Dec-2013 Michael Runge <mrunge@google.com> Enabled incrementals to patch + rename moved files

Change-Id: I551fc5291847e3ace15361c203d86f566c26da97
9b23f2cd786b46991b7c0198e69264b17875288d 25-Nov-2013 Doug Zongker <dougz@google.com> add option to generate two-step recovery files

When run with the -2 option, ota_from_target_files will generate a
package (full or incremental) that does some extra reboots in order to
install the new recovery first, so that the rest of the installation
is done with the new recovery. This can be useful if (say) the
package installation needs some features from the newer kernel.

For incremental packages, the verification phase is still done with
the old recovery.

This is only supported on devices where the misc partition is EMMC
(not MTD).

Two-step packages are slower to install and possibly confusing to
users (they will see their device reboot four times instead of twice),
so only use this option if necessary.

Change-Id: I3267d905e5e8eb1a1eb61bf48255b8b24ffc4ad1
90c60d3b922924cf65d26b478d8baef142a25023 22-Nov-2013 Michael Runge <mrunge@google.com> Revert "Enable incremental builder to find files that moved, and try to process them via patch + rename, instead of delete + add."

This reverts commit 37335b42385970d957d48056145a3d4c4dbc6087.

Change-Id: I61cc125d3b08eaa300a7774b6607dbb43f0e7148
37335b42385970d957d48056145a3d4c4dbc6087 07-Nov-2013 Michael Runge <mrunge@google.com> Enable incremental builder to find files that moved, and
try to process them via patch + rename, instead of
delete + add.

b/11437930

Change-Id: Ie70632a2fa0a13d4bb259f61c620bb01812494e5
0eb17d944704b3eb140bb9dded299d3be3aed77e 08-Sep-2013 Nick Kralevich <nnk@google.com> Update OTA to understand SELinux labels and capabilities

Update the OTA generation script to understand SELinux file
labels and file capabilities.

Make fs_config aware of SELinux labels and file capabilities, and
optionally output those elements whenever we output the
UID / GID / file perms. The information is emitted as a key=value pair
to allow for future extensibility.

Pass the SELinux file label and capabilities to the newly created
set_metadata() and set_metadata_recursive() calls. When the OTA
script fixes up filesystem permissions, it will also fix up the SELinux
labels and file capabilities.

If no SELinux label and capabilities are available for the file, use
the old set_perm and set_perm_recursive calls.

Bug: 8985290
Bug: 10183961
Bug: 10186213
Change-Id: I4fcfb2c234dbfb965cee9e62f060092a4274d22d
850de1995cf026a4c9a8809303ddba5fe4cd3e13 22-Jul-2013 Michael Runge <mrunge@google.com> Revert "Update OTA to understand SELinux filesystem labels"

This reverts commit fbbd79530adc6ddd6bbfb3c5fc60ba5ec0ce5f2d.

All incremental auto OTAs broken b/9964074

Change-Id: I500d2ac194804abd20a0e01d9862fd42e8f5d2de
fb9bb205fcda6544cf5ec5d84cc1e5935ae96935 22-Jul-2013 Michael Runge <mrunge@google.com> Revert "Update OTA to understand SELinux filesystem labels"

This reverts commit fbbd79530adc6ddd6bbfb3c5fc60ba5ec0ce5f2d.

All incremental auto OTAs broken b/9964074

Change-Id: I500d2ac194804abd20a0e01d9862fd42e8f5d2de
fbbd79530adc6ddd6bbfb3c5fc60ba5ec0ce5f2d 18-Jul-2013 Nick Kralevich <nnk@google.com> Update OTA to understand SELinux filesystem labels

Make fs_config aware of SELinux contexts, and output the context
whenever we output the UID / GID / file perms.

Pass the selinux context to the set_perm2() and set_perm2_recursive()
calls. When the OTA script fixes up filesystem permissions, it will
also fix up the SELinux context on the files.

Bug: 8985290
Change-Id: I6419b64c06309a93ac6b2f2cf9fc7f8815adeaf3
0d92f1f13ad89bf8ffbb75764bbe83452612792a 03-Jun-2013 Doug Zongker <dougz@android.com> improve OTA failure messages

Replace OTA script constructs of the form:

assert(foo);

with

foo || abort("sensible message");

so that the log and the on-screen display is somewhat more accessible
to non-experts. (assert() displays the source code of the false
expression 'foo'.)

Change-Id: Ic99448e4466561d305b167cd4d5c1f0f2dbadcce
e2e9f613b5259313c5216d4dad719998a2fbf014 29-May-2013 Kenny Root <kroot@google.com> Fix signing process for SELinux file_contexts

When not building locally and just using a target-files.zip from some
other build, it still tries to access the file_contexts from the out/
directory. This change instead looks at the unzipped target-files.zip
hierarchy to grab that information.

Bug: 9191141
Change-Id: I6ea12e82d6c6376fcada412314c5eefc97ff4853
b32161a2a54706efc0d3493123a971aae93e5261 21-Aug-2012 Doug Zongker <dougz@google.com> change recovery partition construction to use resource .dat

When /system/etc/recovery-resource.dat is available, use it to
construct the recovery-from-boot patch.

Change-Id: I1575f7d284711323186ca6823842eb2a866fd890
1eb74dd9a01ec14a2e41309986ef7efba790be8f 17-Aug-2012 Doug Zongker <dougz@google.com> load SYSTEM/build.prop into the info_dict

Change-Id: Ic747106ba11773ffc9c2eebc163cfd22b1d7a4d5
4c97191bde16a22a50ef51eed6cf65bad1cd1602 02-Aug-2012 Doug Zongker <dougz@google.com> add extra args to mkbootimg

Arrange to take $(BOARD_MKBOOTIMG_ARGS) and pass it to all invocations
of mkbootimg from within make, and to store it in the target_files so
it can be used by future invocations of img_from_target_files and
ota_from_target_files.

Bug: 6918260
Change-Id: I7130ac52e96bd51d4d8b80ca036635e1626f01f1
d513160b76a189899ba01f87a3987b4c6f428cae 02-Aug-2012 Doug Zongker <dougz@google.com> add extra args to mkbootimg

Arrange to take $(BOARD_MKBOOTIMG_ARGS) and pass it to all invocations
of mkbootimg from within make, and to store it in the target_files so
it can be used by future invocations of img_from_target_files and
ota_from_target_files.

Bug: 6918260
Change-Id: I7130ac52e96bd51d4d8b80ca036635e1626f01f1
56882bf9b41dc7f8b98f1dea82633144546450b2 09-Feb-2012 Stephen Smalley <sds@tycho.nsa.gov> Support the setting of file security contexts in OTA and update packages.

Pass the file_contexts configuration to the releasetools scripts
so that the security contexts of files can be properly set for OTA
and update packages.

Requires Ica5fb73d6f2ffb981b74d1896538988dbc4d9b24

Change-Id: I5a63fd61a7e74d386d0803946d06bcf2fa8a857e
1807e700a568d57901b835370d1b1ae05c3dbb0f 28-Feb-2012 Doug Zongker <dougz@android.com> don't generate retouch commands in OTA scripts

Doing ASLR at OTA time is now obsolete; we can stop emitting this code
in OTA scripts.

Change-Id: I2bcf8ef0697ea5590120f89dcd302f273daf531e
c547a4d50b92c8f8173bf4222169f87200a76dba 06-Feb-2012 Ed Heyl <ed@google.com> Please do not merge (already submitted downstream); check entire recovery partition on startup

Checksum the entire recovery partition at boot time to see if we need
to rewrite it, rather than just the first 2kb.

Bug: 5668350
Change-Id: I777754f92e8da630ae3c09bb0d4c41884ff62f39
e5ff5907bef42fa9f3eff2e90ad941423dcc91dc 17-Jan-2012 Doug Zongker <dougz@android.com> add device-specific hooks at start of OTA phases

Change-Id: I47534e6d40e7918ee8cb9e87d76d9030b9273708
bd2dadc21a5db967972a7b8f81aad716609edacb 02-Dec-2011 Doug Zongker <dougz@android.com> check entire recovery partition on startup

Checksum the entire recovery partition at boot time to see if we need
to rewrite it, rather than just the first 2kb.

Bug: 5668350
Change-Id: I777754f92e8da630ae3c09bb0d4c41884ff62f39
0276d1887374fe5a09f57d7f1c8bbdbc9f3e6c3f 02-Dec-2011 Doug Zongker <dougz@android.com> check entire recovery partition on startup

Checksum the entire recovery partition at boot time to see if we need
to rewrite it, rather than just the first 2kb.

Bug: 5668350
Change-Id: I777754f92e8da630ae3c09bb0d4c41884ff62f39
8b9a1b135edbf484d7590411e1254f8097cbc1d5 22-Sep-2011 Doug Zongker <dougz@android.com> change OTA tools to handle variable dev keys

The signing and OTA-building tools now understand the
default_sys_dev_certificate value which may be present in the
META/misc_info.txt file of the target-files packages.

Change-Id: I64f09ec0b77a5184b6ddb74019255518776ee773
831840e51b0403671f746ebc32090b7d6e548c08 22-Sep-2011 Doug Zongker <dougz@android.com> change OTA tools to handle variable dev keys

The signing and OTA-building tools now understand the
default_sys_dev_certificate value which may be present in the
META/misc_info.txt file of the target-files packages.

Change-Id: I64f09ec0b77a5184b6ddb74019255518776ee773
91b4f8a8504711153b5a0126407d3683a9ab75be 23-Sep-2011 Doug Zongker <dougz@android.com> fix syntax error in OTA script

Change-Id: I3a9535acb47fc3960020a390dec8d0294a404c2e
afb32eaca0b0f424f7d7fc5f5979d1b8acf042e4 22-Sep-2011 Doug Zongker <dougz@android.com> change OTA tools to handle variable dev keys

The signing and OTA-building tools now understand the
default_sys_dev_certificate value which may be present in the
META/misc_info.txt file of the target-files packages.

Change-Id: I64f09ec0b77a5184b6ddb74019255518776ee773
e92f15a85885de8db24b6f2ab536d25cf20a5ca5 26-Aug-2011 Doug Zongker <dougz@android.com> patch the /system/build.prop file last

Works around a problem observed on a retail device: incremental update
from build 1 to build 2 partially completed, leaving a mix of files
from the two builds. (Why it booted into the regular system instead
of recovery to restart update installation is still a mystery.)
build.prop was one of the files updated, so the device reported itself
as having build 2. The device hobbled along for months in this state,
until build 3 was released and the 2-to-3 incremental package
repeatedly failed (because some of the files it was trying to patch
were build 1).

This change makes updating build.prop the very last thing does by an
incremental update script, so if installation is aborted and the
regular system starts (and works at all), it will continue reporting
itself as build 1 and be sent the 1-to-2 OTA package again.

Change-Id: I1edc1dcef2bd2495b6fd96517c2f4c574b994f27
cad0bb9f621ff1ccfb584e18249b09768c30a0c0 15-Mar-2011 david <zathrasorama@gmail.com> sha module is deprecated

This patch removes the deprecated warnings from the python
scripts.

Change-Id: I052a0aab3fb28dd1d78de1307edafda6b6c35e5f
7e6d4e45d92fd51f42812ae63ac6e532887bfe0a 14-Dec-2010 Ying Wang <wangying@google.com> Close inherited PIPE before doing work

Gmake in Darwin has file descriptor leak.
In a full build, ota_from_target_files will inherits
more than 2000 open PIPEs from gmake and fails in a call to select.select().
This change fixes the build by closing the PIPEs before doing real work.

Change-Id: Ie7035d7add0b1da3afb6bf9c2009d40f8c7d29b3
55d932840f1a5b412f2961f79368ecec2d28f647 26-Jan-2011 Doug Zongker <dougz@android.com> support use of prebuilt bootable images

img_from_target_files now, with the -z flag, will produce an output
zip with only the bootable partitions (boot and recovery).

img_ and ota_from_target_files can take, instead of a simple
"target_files.zip", a name of the form
"target_files.zip+bootable_images.zip", where the second zip contains
bootable images that should be used instead of building them from the
target_files.zip. (This should be the zip produced by the above -z
flag, perhaps with the images messed with in some way, such as by an
unnamed OEM's extra signature wrapper for their "secure boot"
process.)

Bug: 3391371
Change-Id: Iaf96dfc8f30e806ae342dcf3241566e76ae372d4
f9bbfb577233bce7bb9a0ab7ec842074ce59e6e7 14-Dec-2010 Ying Wang <wangying@google.com> Close inherited PIPE before doing work

Gmake in Darwin has file descriptor leak.
In a full build, ota_from_target_files will inherits
more than 2000 open PIPEs from gmake and fails in a call to select.select().
This change fixes the build by closing the PIPEs before doing real work.

Change-Id: Ife021382198642a97bbbf0b623e4f24f3d86b2b2
96a57e737707d05333dced5b657c4ef21c44088a 26-Sep-2010 Doug Zongker <dougz@android.com> make info_dict and GetTypeAndDevice available to device extensions

Change-Id: I3aa04cb6d7988fc1fdd7f179634b09ceab5749fb
9edabd6958bd0bb420256e8db6f4eeed4fad12a2 22-Sep-2010 Doug Zongker <dougz@android.com> accommodate both new and old target-files when creating incrementals

(cherry-picked from gingerbread.)

Change-Id: I925bf122b0012302a85c0b6f04cca48eb694b0c4
f2ab290550d6f1b1ea9bab91eb4c3c77ceeb5df4 22-Sep-2010 Doug Zongker <dougz@android.com> accommodate both new and old target-files when creating incrementals

(cherry-picked from gingerbread.)

Change-Id: I925bf122b0012302a85c0b6f04cca48eb694b0c4
780c2372706c5250079cb7d759f6530691dd9143 22-Sep-2010 Doug Zongker <dougz@android.com> accommodate both new and old target-files when creating incrementals (do not merge)

Change-Id: Ibc211164bf1dee8275e15d10cbe81b63d666069f
9ce0fb6e59415669074896cfa01e1f0cf97979b7 21-Sep-2010 Doug Zongker <dougz@android.com> support for per-partition fs_type

Include the recovery.fstab file in the recovery image. Remove the
global fs_type and partition_type values from the target-files
key/value dict, and parse the recovery.fstab file instead to find
types for each partition.

(Cherrypicked from gingerbread w/some edits to resolve conflicts.)

Change-Id: Ic3ed85ac5672d8fe20280dacf43d5b82053311bb
258bf46ea6bb4f25d01fab1b783238589e5bbec4 21-Sep-2010 Doug Zongker <dougz@android.com> support for per-partition fs_type (do not merge)

Include the recovery.fstab file in the recovery image. Remove the
global fs_type and partition_type values from the target-files
key/value dict, and parse the recovery.fstab file instead to find
types for each partition.

Change-Id: I35ee2dd0989441dc2a704b63c1b32e598049acb5
37974731fcb4e32b1de5f213d34bd832ca889869 17-Sep-2010 Doug Zongker <dougz@android.com> consolidate target_files metadata into one key-value file

Instead of separate files for recovery api version, tool extensions,
and mkyaffs2 options, put those all in the generic key-value file.

Change-Id: Ib642311632844d52e4895fd4747093fc7e86232d
486de126e398938b4f38d3e8065ad520244fe5e3 16-Sep-2010 Doug Zongker <dougz@android.com> only do size check for yaffs images; fix incremental info dict

Change-Id: Ic862000a23b93289604b0737bfcaf8d461f36ff8
c77a9ad444d49e2ad777678cf5671f0a94f44ffb 16-Sep-2010 Doug Zongker <dougz@android.com> store user-visible image sizes in target-files

Do the yaffs-specific adjustments to image sizes in common.CheckSize,
instead of baking it into the image size stored in the target-files
package. Remove the special fs_type flag and fold it into the
"info_dict" we have for saving key-value pairs from the build system.

Change-Id: I6e63f3330f6277d9a946b22e66cadeb51203ba14
c19a8d5590a4ffd42b37ceaca2d779b48e481f99 02-Jul-2010 Doug Zongker <dougz@android.com> support for ext4/EMMC in target_files and OTA generation

Move the image sizes into a more generic key-value file. Make them
optional. Add additional key/value pairs describing what kind of
filesystem the device uses. Pass new fs-type-related arguments in
edify scripts when mounting and reformatting partitions.

Don't include all the init.*.rc files from the regular system in
recovery -- they aren't needed, and break recovery on some devices.

Change-Id: Ic1c651f754ed00ba1cffe8cf56c43f7f3b0ebfd7
ea5d7a9de7660bef5b9c68f6372a92d4b2f2f1f6 13-Sep-2010 Doug Zongker <dougz@android.com> move File and Difference classes into common script

This makes them accessible from device-specific extensions (so they
can be used to send radio images as binary patches, for instance).

Change-Id: I2f2174b93b4265abf9400f9e5a0982caca0771e9
c637db16d83b2c248b1cf0122e2ba558ed95762c 21-Apr-2010 Doug Zongker <dougz@android.com> remove remaining amend support

Remove the remaining (unused and untested) support for generating
amend scripts. This means that you won't be able to OTA directly from
cupcake to gingerbread.

Change-Id: Iaf5295db92a42b336960d05295f48b67cb729337
41d6ca8674c0322a3b5deea45fcd84abe66b75d0 22-Jun-2010 Doug Zongker <dougz@android.com> include build timestamp in OTA metadata

Change-Id: I44292c6346db3356dfafbd044027e37a9b467cfe
dafb04275588fff8248b6a5360ca047cdffd14a5 26-Aug-2010 Hristo Bojinov <hristo@google.com> Turn on ASLR by default. The "-a" option now takes an argument (on/off).

Change-Id: I4ddc70f2087c23586fc6aaaff3bb862f83d9779d
96be7205dce97977909e93b73f48779fcce3bc65 02-Aug-2010 Hristo Bojinov <hristo@google.com> Working ASLR implementation.

ASLR for shared libraries is controlled by "-a" in ota_from_target_files.
Binary files are self-contained (supported by apriori/soslim).

Signed-off-by: Hristo Bojinov <hristo@google.com>
Change-Id: I500e325bf4a70a8d69a2ab9b2938e83dadb4e65d
67369983cf23e12724c135c3850c98326558256b 07-Jul-2010 Doug Zongker <dougz@android.com> generate pseudofilenames for EMMC partitions

Change-Id: Ibe77f4338fed5745c7c650270dcef24988ebfdda
b4c7d32cbac0ef36062c4e2f348b47e027610eef 02-Jul-2010 Doug Zongker <dougz@android.com> support for ext4/EMMC in target_files and OTA generation

Move the image sizes into a more generic key-value file. Make them
optional. Add additional key/value pairs describing what kind of
filesystem the device uses. Pass new fs-type-related arguments in
edify scripts when mounting and reformatting partitions.

Don't include all the init.*.rc files from the regular system in
recovery -- they aren't needed, and break recovery on some devices.

Change-Id: I40e49e65f0d76f735259e4b4fef882322cd739da
3b85269cfe10ec15e9b544dd6964b8c9f5f761f9 22-Jun-2010 Doug Zongker <dougz@android.com> include build timestamp in OTA metadata

Change-Id: I44292c6346db3356dfafbd044027e37a9b467cfe
2ea21065b66da9819df92b37a79f0f87552ee331 29-Apr-2010 Doug Zongker <dougz@android.com> add metadata to OTA package in easily-parsed form

This is only for use by the tools that manipulate OTA packages; it
doesn't touch device code.

Change-Id: I1e21b24b86340d8f86c9d15b2aeb79c461f75d18
9ce2ebf5d300eba5f6086583b0941ef68a3e4b42 21-Apr-2010 Doug Zongker <dougz@android.com> remove remaining amend support

Remove the remaining (unused and untested) support for generating
amend scripts. This means that you won't be able to OTA directly from
cupcake to gingerbread.

Change-Id: Ie7179a5f2e5ca0d2c0305000344ac39b1dd7385f
3475d362a771bcbbca23db527c399c0e5a67f576 18-Mar-2010 Doug Zongker <dougz@android.com> fix typo (leftover from some debugging code)

Change-Id: I29a42820ccf8567508f1aeefcc89c2462e9ad43c
283e2a1e1bae4e21824969a15da6420204633ddd 16-Mar-2010 Doug Zongker <dougz@android.com> store filesystem config info in target files at build time

Store a dump of the desired uid/gid/mode for every system file in the
target_files zip. Modify ota_from_target_files to use this stored
information when it is available, instead of running fs_config from
the current client (which might be out of sync from the one where the
target_files zip was built).

b/2516887 - New android_filesystem_config.h needed

Change-Id: I8409a0265d1d50daad9c2bc033c99b74b8931b20
c8d446bcde877ec94f8e68dd5af68fe34eb1b1f9 23-Feb-2010 Doug Zongker <dougz@android.com> pass blobs to applypatch in incremental OTAs

applypatch now takes patches as blob-valued arguments instead of just
filenames, eliminating the need to unpack all patches to /tmp before
starting to apply them.

Revert the last change I made where sha1_check(read_file(...)) was
substituted for apply_patch_check(...). apply_patch_check() knows to
check /cache/saved.file if the original source file is missing or has
a bad checksum, which is important if the device loses power or
otherwise restarts during patching.

Change-Id: Ia5b761474b0f809a5a5eed29455b1b145145699e
5a48209541d5eed602bfb8e2c4ff51e31443daf2 18-Feb-2010 Doug Zongker <dougz@android.com> relocate applypatch; check patch integrity

- Delete the applypatch code (it's being moved to bootable/recovery).

- Change the OTA script generator to verify the sha1sum of all the
patch files after they are unpacked into /tmp.

b/2361316 - VZW Issue PP628: Continuous reset to Droid logo:
framework-res.apk update failed (CR LIBtt59130)

Change-Id: I5858d48f10127f72e708d50d3de844569d1aff27
42265390d993664e7797abc12d7e6bd1c2a6dc6b 12-Feb-2010 Doug Zongker <dougz@android.com> change incremental OTA file install order

(Cherry-pick from donut.)

Delete the old recovery-from-boot patch before patching system files,
then install the new one at the end of the process. Also fix an issue
with applypatch returning the wrong status.

Change-Id: I614fce39f307eded696ba792912b863f9fb0580b
14833605d26bf970cd5335c02af4354b68d93348 02-Feb-2010 Doug Zongker <dougz@android.com> add UnmountAll function to OTA script; support radio images w/path

Three unrelated changes:

- change the add-radio-file makefile function to support adding files
with paths. (The path part of the pathname is stripped off.)

- add an UnmountAll function to the OTA script generation code, so
that we can explicitly unmount everything we've mounted (in addition
to doing it automatically at the end of the script).

- add the updater API version to the info object passed to
device-specific code.

Change-Id: Ia62b15403c1cc8fce8d9910f291450c8077e49f4
e6b4e5b20ac8a3b01f1a99dd877a98c92036ce5e 13-Nov-2009 Jean-Baptiste Queru <jbq@google.com> eclair snapshot
cfd7db6d8494c7d3169a4eac0dc63737a24ff1d1 07-Oct-2009 Doug Zongker <dougz@android.com> move recovery image to the top of the OTA package

Make the recovery image be the first thing installed in system/ in the
OTA package, so it will get installed even if the system partition
fills during OTA installation.
c18736b1a777c386dc3c5e3f878249770b5edd78 30-Sep-2009 Doug Zongker <dougz@android.com> remember device-specific releasetools extensions in target-files

Store the location of the releasetools extensions in the target-files
zip, and make ota_from_target_files use that stored location by
default (though it can still be overridden with -s if desired).
481c4e650fe6cdb9c63ad04b35d221a2856ca48d 28-Sep-2009 Doug Zongker <dougz@android.com> fix python syntax error
761e642d54eec743699c6c2ce1ea587853d08f33 25-Sep-2009 Doug Zongker <dougz@android.com> parallellize computation of binary patches

Change ota_from_target_files to compute binary patches for all the
changed files in the OTA package in parallel. On a recent test build
this cuts incremental construction time roughly in half (6 min to 2
min 40 sec).
881dd40ffb683fed465df955f3fd21812fae59aa 20-Sep-2009 Doug Zongker <dougz@android.com> improve the OTA progress bar

Make the OTA progress bar advance a bit more smoothly by reallocating
how much is dedicated to various phases of incremental OTA
installation, based on some crude measurements of how long each phase
takes.
15604b84e246514da6c9721266919003f734380b 02-Sep-2009 Doug Zongker <dougz@android.com> use imgdiff -z to diff zipfiles

imgdiff now understands the zip file format and can produce smaller
patches by doing the patching in uncompressed space. Start taking
advantage of this for .zip, .apk, and .jar files.
f76b9c87b16956aebfe5af497913d698dd03cc01 28-Aug-2009 Doug Zongker <dougz@android.com> remove bootloader assertions

These should be done by the device-specific module where necessary;
they aren't appropriate for all devices.
3b949f07259ee8d67b4454627aceab5e6f44bd39 24-Aug-2009 Doug Zongker <dougz@android.com> incremental OTAs should remove /system/recovery.img

This file has been replaced by the recovery_from_boot.p patch file,
but needs to be removed from existing devices if it exists.
951495fc4802a3603f654c02c7acceda4859f5e1 14-Aug-2009 Doug Zongker <dougz@android.com> update OTA package maker to do whole-file signature

Use the new -w flag to SignApk when signing OTA packages.
fdd8e69c42e66fb70384bcaca1747f504f2c021c 04-Aug-2009 Doug Zongker <dougz@android.com> use the max image sizes from the target files zip

For some time now the build system has included all the max image
sizes in a file in the META directory. Use these instead of needing
to parse the BoardConfig.mk file for the device at the time of
building an image or OTA package.
73ef8257ce54ce1ddc2d4cfc3b3814ca734c0e11 24-Jul-2009 Doug Zongker <dougz@android.com> use a binary patch to install recovery from system

Instead of storing the whole recovery image in system in order to
flash it on first boot, we instead use an imgdiff patch from the boot
image to create the recovery image. This is substantially smaller
since it effectively only stores the recovery binary and UI images
(the kernel and the init binary are identical to that of the boot
image).

This change modifies the OTA-building script to create and install
these patches, and changes the calculation of the system image size in
the Makefile to reflect the new scheme.
0493e240fd66fcd164bb28c6f08ab901a6d2179b 22-Jul-2009 Doug Zongker <dougz@android.com> make bootloader requirements optional

Interpret a "*" in the "require version-bootloader" line as "don't
require any specific bootloader". Sholes OTAs include the bootloader
and the main system in the same package.
030614740c1a22e51c6513058852f9ab368fdf5d 14-Jul-2009 Doug Zongker <dougz@android.com> in auto mode, generate both edify and amend scripts for full OTAs

Generate packages that can be installed by either amend or edify, so
we can remove amend support from donut.
0f3298a497e32f6c2325a0071124a62d031fae6f 30-Jun-2009 Doug Zongker <dougz@android.com> delete source files not in target

Incremental OTAs were not removing files from the old build that don't
exist in the new build. Fix.
e1c31bacae7be6da140066966e6d2bbadd9331e9 24-Jun-2009 Doug Zongker <dougz@android.com> make building recovery and boot images optional

If the source target-files zip omits files needed to build the
recovery and/or boot images, leave them out instead of dying with an
error. This lets build like "generic-userdebug" work.
9fc74c7823182c4121f32114a7f83ae3fa7e4346 24-Jun-2009 Doug Zongker <dougz@android.com> don't fail if no required bootloaders are defined

Generic targets don't have a list of defined bootloaders. Instead of
failing to build an OTA package, just omit the constraint.

Fix bad references to ExternalError.
05d3dea519688b61d86e30c2d4b99ff494aeca73 22-Jun-2009 Doug Zongker <dougz@android.com> support hooks for device-specific code in OTA package generation

Replace the installation of the "radio image", which is an
HTC-specific notion, with calls to device-specific python modules that
can add whatever additional OTA script commands are necessary. Add
the -s flag to specify the location of the device-specific script
(replacing the unused -s flag in sign_target_files_apks).
c494d7cee85d980647ca915ea64355b71fe817eb 18-Jun-2009 Doug Zongker <dougz@android.com> generate OTA packages that use edify

Split the details of generating script syntax into a generator class:
one for amend (whose output should be equivalent to the current
output), and one for edify.

Fix 'otatools' build rule to build imgdiff.
171f1cde104891840b0c3c271935fae5433f1b25 16-Jun-2009 Doug Zongker <dougz@android.com> don't require some OTA features

Make the following things optional:
- kernel command lines for bootable images
- radio images
- bootloader assertions
These are not all (yet?) defined for some new devices.
048e7ca15f6391681490ce564bc71194adf146aa 15-Jun-2009 Doug Zongker <dougz@android.com> fix archive files being created with perms 000

In python 2.5 and earlier, ZipFile.writestr(filename, data) results in
the file being added to the archive with permissions 000. (See
http://svn.python.org/view?view=rev&revision=65235.) Work around this
by creating a ZipInfo object and setting the permissions explicitly.
5da317e51d1832cb1ec67dd20fbcff7708bbadb5 02-Jun-2009 Doug Zongker <dougz@android.com> support incremental updates of boot image

Modify applypatch to be able to write MTD partitions as well as read
them. Make applypatch save a backup copy of the contents of an MTD
partition it reads in cache, to be used in case an update is
interrupted while writing back to MTD. Modify OTA package creation
script to send boot image updates in patch form.
f6a8bada5f0966762eadaec96de6430d0cd577e3 29-May-2009 Doug Zongker <dougz@android.com> add support for reading MTD partitions to applypatch

Allow an MTD partition so serve as a source "file" in applypatch,
using a magically-formatted 'filename' that specifies the partition
name, size of data to read, and expected hash. Build incremental OTAs
that update the recovery image via a patch.
1c390a2aa97127ef8af8b0df1d4028f501fdce64 15-May-2009 Doug Zongker <dougz@android.com> add -e option to add extra commands to OTA script
ef85ea6086107537d07720283ca9b1af01de247c 08-May-2009 Doug Zongker <dougz@android.com> allow separate source and target files for applypatch

Allow applypatch to use different filenames for the source and target.
(Using the same filename is still fine; in fact the target filename
can be specified as "-" to mean "same as the source file.) This will
allow us to still use diffs in the case of files being renamed, and
will allow us to use diffs for the boot and recovery images.
962069ce59c85949d147874df2728a5ffd9193be 23-Apr-2009 Doug Zongker <dougz@android.com> add -n option to turn off prereq check when build OTA packages

Developers might legitimately want to install older builds on their
phones, so add option to build OTA packages that will install over
newer builds.
dbfaae5567dbbfcc17e15b31e65178ad367e6d13 22-Apr-2009 Doug Zongker <dougz@android.com> add -w option to ota_from_target_files

Add option to make OTA packages that wipe user data.
dbfb20ccd895d6ed84b95670fe2b9a0a34b17b7c 03-Apr-2009 Doug Zongker <> AI 144399: am: CL 144270 am: CL 144269 Relocate the new (google-indepedent) tools for signing and
building images & OTA packages out of vendor/google.
No device code is touched by this change.
Original author: dougz
Merged from: //branches/cupcake/...
Original author: android-build

Automated import of CL 144399
eef3944eb3673329b5e89cf188ac592805a0b08d 02-Apr-2009 Doug Zongker <> AI 144270: am: CL 144269 Relocate the new (google-indepedent) tools for signing and
building images & OTA packages out of vendor/google.
No device code is touched by this change.
Original author: dougz
Merged from: //branches/cupcake/...

Automated import of CL 144270
ad44c07ac7c505a7b6b5ef14c7234c58e3b919d0 02-Apr-2009 Doug Zongker <> AI 144269: Relocate the new (google-indepedent) tools for signing and
building images & OTA packages out of vendor/google.
No device code is touched by this change.
BUG=1753409

Automated import of CL 144269