20bdc70e540e021c4a26553dfdbfd2fabe8ea432 |
|
08-Dec-2016 |
Alex Deymo <deymo@google.com> |
Parse Android .map files for unknown filesystem. When generating a filesystem during the Android build, we also generate a text file with .map extension with the list of files and the blocks in the filesystem they are located. For filesystems unsupported in delta_generator (like squashfs) we use this text file to produce efficient delta payloads. Bug: 28150981 Test: Added unittest for parsing. Generated a delta payload of a squashfs image. Change-Id: I154e72ac785c6f508290daa901fa7958b446c010
/system/update_engine/payload_generator/payload_generation_config.h
|
5b91c6b141970c2b0095775a61e3f941417aa1ff |
|
05-Aug-2016 |
Alex Deymo <deymo@google.com> |
New setting to mark postinstall as optional. This setting allows to mark a postinstall script as optional. This allows the postinstall program to fail when it is not strictly required to run for the update to succeed. Bug: 27178350 TEST=Added unittest. Sideloaded an update with an optional postinstall. Change-Id: I41956d3308f3458b6bf94b835c9b5e470c84ca41
/system/update_engine/payload_generator/payload_generation_config.h
|
a4073ef63482fd08c3678982f7d153360b088094 |
|
23-Mar-2016 |
Alex Deymo <deymo@google.com> |
Introduce PayloadVersion struct for version information. This new little struct encapsulates the version information (major and minor version numbers) and the zlib fingerprint information. Now, instead of querying throughout if the version number is certain value, we ask the PayloadVersion struct whether certain operation is allowed in that version or not. This moves the logic of what's supported and what's not to a single place and eliminates the need to pass several booleans to the helper functions. Bug: None TEST=Unittest still pass. Change-Id: Iaa6e7bc032db6479fdeab72255d7062fca1c07e5
/system/update_engine/payload_generator/payload_generation_config.h
|
55c4f9ba7f5c59e3345f2c1869464433ffa8dc2b |
|
10-Feb-2016 |
Sen Jiang <senj@google.com> |
Include IMGDIFF operation in minor version 4 or up. Try imgdiff if both source and target contains gzip. Test: unit test & generated a payload with boot.img Bug: 26675118 Change-Id: I2861d9b953ffbdec44fdfb42cff5687698ea205e
/system/update_engine/payload_generator/payload_generation_config.h
|
05feee0fc8f4f10b425f04e1f5cdd4831bbfb969 |
|
12-Nov-2015 |
Sen Jiang <senj@google.com> |
delta_generator: Include postinstall calls in the payload major version 2. Added a new flag --new_postinstall_config_file which takes a path to a key value store config file and saves this information to the PartitionUpdate field. The config file looks like this: RUN_POSTINSTALL_root=true POSTINSTALL_PATH_root=postinstall FILESYSTEM_TYPE_root=ext4 "root" can be changed to any partition name. Bug: 24537566 TEST=Generated a payload v2 with postinstall. TEST=Added unittest. Change-Id: Ied3c7bc2cfb54f4bcc69207f1e5bd473f72024fe
/system/update_engine/payload_generator/payload_generation_config.h
|
39910dcd1d68987ccee7c3031dc269233a8490bb |
|
10-Nov-2015 |
Alex Deymo <deymo@google.com> |
Split payload application code into a subdirectory. This patch splits from the main libupdate_engine code the part that is strictly used to download and apply a payload into a new static library, moving the code to subdirectories. The new library is divided in two subdirectories: common/ and payload_consumer/, and should not depend on other update_engine files outside those two subdirectories. The main difference between those two is that the common/ tools are more generic and not tied to the payload consumer process, but otherwise they are both compiled together. There are still dependencies from the new libpayload_consumer library into the main directory files and DBus generated files. Those will be addressed in follow up CLs. Bug: 25197634 Test: FEATURES=test emerge-link update_engine; `mm` on Brillo. Change-Id: Id8d0204ea573627e6e26ca9ea17b9592ca95bc23
/system/update_engine/payload_generator/payload_generation_config.h
|
981eb1151f42480ea10cb5283ab2108454a16c92 |
|
26-Aug-2015 |
Sen Jiang <senj@google.com> |
Support multiple partitions in ImageConfig ImageConfig now have a vector of partitions instead of hardcoded rootfs and kernel. Bug: 23420126 TEST=cros_workon_make update_engine --test Change-Id: Id226cc04628b44f1fcbd58f03038809867bf9e40
/system/update_engine/payload_generator/payload_generation_config.h
|
625406cee9a90ac2ed895f480286b7f0e8497f38 |
|
17-Sep-2015 |
Sen Jiang <senj@google.com> |
payload_generator: Remove PartitionName enum. payload_generator should be able to generate payload for any partition name in major version 2, so we should use a string for partition name. Bug: 23620828 TEST=cros_workon_make update_engine --test Change-Id: Iee92af5d82e74b9be6ae793850a4ae6c31e008c5
/system/update_engine/payload_generator/payload_generation_config.h
|
262c7be2e5ca3ea6d0a814200392895fd3faa931 |
|
09-Sep-2015 |
Alex Deymo <deymo@google.com> |
Remove Chrome OS specific verity_utils module. The size of the updated region in the partition should match the size the verified boot chain requires, or at least be bigger. In Chrome OS, the kernel partition includes a command line with the size of the rootfs verity will verify, but the rootfs partition is significantly bigger (about 800 MB more) so we only update what is required. In the more general Brillo case, the size of the updated area depends on how verity is used. This patch removes this logic from the delta_generator and uses the size of the filesystem, which should match the size of the verity region in Chrome OS devices using ext2. Bug: 23084776 Test: sudo emerge_update; `mma` on AOSP Change-Id: I37f341263d86f127d937b4afc24e9d085131cb08
/system/update_engine/payload_generator/payload_generation_config.h
|
46e9b1735a16518d563b3c7e7227408f8fc3e891 |
|
31-Aug-2015 |
Sen Jiang <senj@google.com> |
Add major_version flag in delta_generator. Only affects major version number in the payload for now, actual implementation of major version 2 will follow. TEST=Call delta_generator manually with --major_version=2 then cros payload show. Bug: 23694580 Change-Id: I5baca5a868ec00dc2f6ea63d89e295d43283fd32
/system/update_engine/payload_generator/payload_generation_config.h
|
aea4c1cea20dda7ae7e85fc8924a2d784f70d806 |
|
20-Aug-2015 |
Alex Deymo <deymo@google.com> |
Re-license update_engine to Apache2 This patch automatically replaced the license on all text files from Chromium OS (BSD style) to AOSP (Apache2), keeping the original year as a reference. The license header was added to .gyp and .gypi files, the NOTICE was replaced with a copy of the Apache2 license and MODULE_LICENSE_* file was updated. BUG=b/23084294 TEST=grep 'Chromium OS Authors' doesn't find anything. Change-Id: Ie5083750755f5180a8a785b24fe67dbf9195cd10
/system/update_engine/payload_generator/payload_generation_config.h
|
2d3b2d635e50c6886e285afb86c3187d9e0bd360 |
|
18-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Implement soft/hard chunk size limits. The current approach uses a single chunk_size value, that defaults to -1 (or "whole files") for delta payloads. The "whole files" default is the preferred value since BSDIFF and SOURCE_BSDIFF operations (the most common operations in real delta payloads) will benefit from having access to the whole file in the old and new partition, in particular in cases like the chrome binary (~120 MiB in size). On the other hand, very big chunks have a big memory footprint in most cases. Current implementations of BSDIFF, REPLACE_BZ and REPLACE will require as much private memory as the destination chunk_size or more. Because of this and due to the lack of old data, a small chunk_size (1 MiB) is used for full payloads. To break this tension between having a big chunk_size for operations that will benefit from it versus having a small chunk_size for cases where it doesn't change anything, this patch introduces two chunk size values: a hard and soft limit. The hard_chunk_size has the same meaning as the old chunk_size: no operation should have a destination bigger than the provided hard limit. The soft_chunk_size is the preferred chunk size for an operation when a bigger chunk will not benefit significantly the final payload size. Having a small chunk size for operations like REPLACE_BZ, REPLACE, SOURCE_COPY and MOVE is important to keep the memory footprint low when it the extra memory is not required. The new soft_chunk_size limit is used when merging operations (previously hard-coded to 1 MiB) and when generating new operations for zeroed and moved blocks. BUG=chromium:485397 TEST=Run cros_generate_update_payload for full and delta payloads. Inspected the chunk size in the resulting payloads. Change-Id: I370048a81913ad23a151cfef6690627b7fff7277 Reviewed-on: https://chromium-review.googlesource.com/286568 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Trybot-Ready: Alex Deymo <deymo@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/payload_generation_config.h
|
896fdbaa1cc9a422ac36ef455d02c3cef1f27816 |
|
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Prevent the InplaceGenerator to add MOVEs to block 0. While we normally don't generate MOVE operations to/from the block 0, the cycle breaker logic can use the block 0 as scratch space, creating MOVE operations to and from the block 0. This patch prevents it from picking the block 0 as scratch space and logs a message when that would happend. BUG=chromium:480751,chromium:500423 TEST=Added unittest. Change-Id: I91f1b3c426a9d06aae5b685e2e901c7e448d8677 Reviewed-on: https://chromium-review.googlesource.com/286623 Reviewed-by: Gilad Arnold <garnold@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/payload_generation_config.h
|
b42b98db059a12c44110588fc0b3d5f82d32a2f8 |
|
07-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Load minor_version from the FilesystemInterface. In the normal use case, cros_generate_update_payload doesn't pass the requested minor_version to delta_generator which means we need to detect it from the old image. To detect it, we where looking at the update_engine.conf file in the mounted source directory. This patch uses the recently added FilesystemInterface::LoadSettings() method to read the update_engine.conf settings from the source rootfs partition. In order to make this information available when autodetecting the minor_version and to help future FilesystemInterface mocking for testing, we move the FilesystemInterface instance from inside the OperationsGenerator class to the PartitionConfig, so the filesystems are opened before calling GenerateUpdatePayloadFile. This patch then deprecates the --old_dir flag removing the requirement to run delta_generator and cros_generate_update_payload as root when generating deltas. BUG=chromium:305832 TEST=Ran cros_generate_update_payload to generate deltas from old (link FSI) and new (ToT link) images. Change-Id: I915679d62aae2d1a2acee68cc8c1ec691cc363ad Reviewed-on: https://chromium-review.googlesource.com/283643 Trybot-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/payload_generation_config.h
|
14158570d3995008dc93a628004118b87a6bca01 |
|
13-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Split delta_diff_generator file. The DeltaDiffGenerator class includes both an OperationsGenerator using the A-to-B operations and a set of common methods used also by the inplace generator. The delta_diff_generator.{h,cc} files also include a single function to generate the payload (GenerateUpdatePayloadFile) that centralizes the logic of generating the operations and writing the payload. This patch splits these three parts in different files. The common delta diff function are moved to the delta_diff_utils.{h,cc} files. The operations generator class that uses A-to-B operations is now in a new ab_generator.{h,cc} pair of files that implement the ABGenerator() class. Finally, the payload file writing methods are now in a single PayloadFile class. This allow us to create payload files without the need to generate images and their deltas. This will be used in a follow up CL to remove the image generation logic from the unittests. BUG=chromium:351589 TEST=Ran unittests. Regenerate a payload with and without this patch; got the same results. Change-Id: I6816d2c805ba8c0c5c9423c720131a100a15ebaa Reviewed-on: https://chromium-review.googlesource.com/280838 Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org> Trybot-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/payload_generation_config.h
|
35589c2b9a0e20b42661b132890128d8025c1954 |
|
07-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Remove rootfs_mountpt from ImageConfig. The mounted rootfs is not used anymore during opeartions generation so this patch removes the setting from the ImageConfig. The --new_dir flag is now deprecated because of this. The --old_dir is still being used when no --minor_version is passed, to autodetect the minor version of the old image. With this patch, the rootfs and kernel partitions have a more generic description in the config. BUG=None TEST=Unittest still pass. Change-Id: I0398493a44f8c09159a93a716f2168f45e8f8df9 Reviewed-on: https://chromium-review.googlesource.com/275805 Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/payload_generation_config.h
|
a26432a8fe5981ca1bb41cb6aacddfc3765666ac |
|
13-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Use the rootfs size specified by verity. When using rootfs verification and generating a payload, it is important to generate a payload that writes all the blocks verity expects to hash in the hash tree, even if those blocks are never used by the filesystem. When using squashfs, the filesystem is padded with zeros up to the fs_size provided in the disk_layout.json, which is used as the size of the dm-verity device. Because of this, it is important to always generate a payload that writes all that. This patch parses the kernel using vboot_host tools and updates the rootfs_size value with the one specified to verity in the kernel command line. When no verity options are found or the kernel is not provided (as in the case of a full kernel update in a delta payload) only the filesystem part is considered for the rootfs_size. This means that the extra zeros in the source rootfs won't be used for the delta payload when generating a full kernel payload, and also means that the zeros after the squashfs won't be written when generating the new rootfs with rootfs verification disabled. BUG=chromium:463783 TEST=FEATURES=test emerge-link update_engine TEST=Ran delta_generator with invalind kernels (in full and delta mode). TEST=Ran cros_generate_update_payload with real images with both squashfs and ext2. Change-Id: Id151063722a20d27c50724f6b27f774a3436e3ea Reviewed-on: https://chromium-review.googlesource.com/259839 Reviewed-by: Alex Vakulenko <avakulenko@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org> Trybot-Ready: Alex Deymo <deymo@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org>
/system/update_engine/payload_generator/payload_generation_config.h
|
9b244df41f1bdaddd87b7dbd8e1559556059ed1b |
|
12-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Refactor operation generation methods. We have so far three methods to generate the list of operations to update the kernel and the rootfs partitions, which are different strategies: full (using only REPLACE and REPLACE_BZ), delta "in-place" (using also MOVE and BSDIFF) and "source" delta, or delta minor version 2 (using SOURCE_COPY and SOURCE_BSDIFF instead). As we consider ways to support updates for squashfs, the two delta methods don't support filesystems other than ext2, so we need to use yet another different strategy for those. This patch splits the code that generates the list of operations for the kernel and the rootfs in three methods with the same interface. No functional changes were made. Some local functions were exposed as public static methods. The new graph_types.cc is added to avoid a link-time error when the kInvalidIndex member is not inlined. BUG=chromium:430950 TEST=FEATURES=test emerge-link update_engine Change-Id: Ib25ff7a6f17d0990637596dcd6b59568b9e15d26 Reviewed-on: https://chromium-review.googlesource.com/259462 Reviewed-by: Alex Vakulenko <avakulenko@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/payload_generation_config.h
|
f1cbe1783dd025bd7243a8df12f20548196cc023 |
|
06-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Refactor payload generation request args in a class. This patch introduces a new struct PayloadGenerationConfig encapsulating the payload generation request performed by the user in the command line. Since we have several payload generation strategies that are triggered based on the source image version and if there is an image version at all, encapsulating the request in a single struct helps splitting the different strategies in independent classes and pass the payload generation configuration struct with the request to those classes. BUG=chromium:430950 TEST=FEATURES=test emerge-link update_engine Change-Id: Ie960c6510bf02d8e10f5e1adaa847a2048c59598 Reviewed-on: https://chromium-review.googlesource.com/258520 Reviewed-by: Alex Vakulenko <avakulenko@chromium.org> Tested-by: Alex Deymo <deymo@chromium.org> Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/payload_generation_config.h
|