| 929461ab11769339108658f2022d068446ca90f7 |
|
22-Jun-2017 |
Sen Jiang <senj@google.com> |
Only split operation if it has multiple dst extents.
FragmentOperations() will split operations so that every operation will
have only one dst extent, but if the operation already meet this
requirement, it should be a no-op and we shouldn't waste time recompressing
it. This can save a lot of time if there's huge replace operation that
only has one dst extent.
As a side effect of not splitting those operations, if they are SOURCE_COPY
operations, they will not have src_length and dst_length set anymore,
because those were set in SplitSourceCopy() before this change, but these
lengths are never actually used in delta performer since the very first
implementation of PerformSourceCopyOperation():
https://chromium-review.googlesource.com/c/263747/
So it's safe to remove it from the manifest. We might as well just remove
src_length and dst_length in all SOURCE_COPY operations (even for those
are splitted) to reduce payload size a little bit.
Bug: 62470452
Test: generated a payload, all operations are still the same except
SOURCE_COPY don't have src_length and dst_length any more
Change-Id: I019ca9056479165e76cc25d7a4ba453088398928
/system/update_engine/payload_generator/ab_generator.cc
|
| 5c6bb1d8f89b87e782e26fe74789b6f510704394 |
|
27-Jul-2016 |
Chih-Hung Hsieh <chh@google.com> |
Fix clang-tidy performance warnings in update_engine.
* Use const reference type for parameters, local variables,
and for-loop index variables to avoid unnecessary copy.
* Convert some for-loops to for-rang loops.
Bug: 30407689
Bug: 30413223
Bug: 30413862
Change-Id: I78996b3f799639fc57ced45e110807625be7dcce
Test: build with WITH_TIDY=1
/system/update_engine/payload_generator/ab_generator.cc
|
| 726aeca1c665910593761a1990ee50f22d98f53f |
|
06-Apr-2016 |
Alex Deymo <deymo@google.com> |
delta_generator: Use REPLACE_XZ in deltas when allowed.
This patch moves the generation of the best allowed full operation to a
new function which now includes ZERO, REPLACE, REPLACE_BZ and REPLACE_XZ
when allowed. This function is used to produce a full operation
whenever it is needed in the "full" payload generator and the AB "delta"
payload generator.
Bug: 24578399
TEST=brillo_update_payload generated REPLACE_XZ operations for a delta payload that was using REPLACE_BZ operations and merged them.
TEST=brillo_update_payload generated a full payload with REPLACE, REPLACE_BZ and REPLACE_XZ operations.
TEST=Updated unittests.
Change-Id: I29618094a84ef8050f8dd7ede0fd70056dc9d650
/system/update_engine/payload_generator/ab_generator.cc
|
| a4073ef63482fd08c3678982f7d153360b088094 |
|
23-Mar-2016 |
Alex Deymo <deymo@google.com> |
Introduce PayloadVersion struct for version information.
This new little struct encapsulates the version information (major and
minor version numbers) and the zlib fingerprint information. Now,
instead of querying throughout if the version number is certain value,
we ask the PayloadVersion struct whether certain operation is allowed in
that version or not. This moves the logic of what's supported and
what's not to a single place and eliminates the need to pass several
booleans to the helper functions.
Bug: None
TEST=Unittest still pass.
Change-Id: Iaa6e7bc032db6479fdeab72255d7062fca1c07e5
/system/update_engine/payload_generator/ab_generator.cc
|
| 55c4f9ba7f5c59e3345f2c1869464433ffa8dc2b |
|
10-Feb-2016 |
Sen Jiang <senj@google.com> |
Include IMGDIFF operation in minor version 4 or up.
Try imgdiff if both source and target contains gzip.
Test: unit test & generated a payload with boot.img
Bug: 26675118
Change-Id: I2861d9b953ffbdec44fdfb42cff5687698ea205e
/system/update_engine/payload_generator/ab_generator.cc
|
| 39910dcd1d68987ccee7c3031dc269233a8490bb |
|
10-Nov-2015 |
Alex Deymo <deymo@google.com> |
Split payload application code into a subdirectory.
This patch splits from the main libupdate_engine code the part that
is strictly used to download and apply a payload into a new static
library, moving the code to subdirectories. The new library is divided
in two subdirectories: common/ and payload_consumer/, and should not
depend on other update_engine files outside those two subdirectories.
The main difference between those two is that the common/ tools are more
generic and not tied to the payload consumer process, but otherwise they
are both compiled together.
There are still dependencies from the new libpayload_consumer library
into the main directory files and DBus generated files. Those will be
addressed in follow up CLs.
Bug: 25197634
Test: FEATURES=test emerge-link update_engine; `mm` on Brillo.
Change-Id: Id8d0204ea573627e6e26ca9ea17b9592ca95bc23
/system/update_engine/payload_generator/ab_generator.cc
|
| 82352f97b4621dbf6af8308ff0b0b17b0968b53a |
|
10-Nov-2015 |
Sen Jiang <senj@google.com> |
delta_generator: Include per-operation source hash.
This adds a src_sha256_hash field in the operation if minor version is 3.
Bug: 23182225
TEST=cros_workon_make update_engine --test
TEST=Generated a minor version 3 payload, the manifest is about 50% larger.
Change-Id: I0e363fca4de24bae64ac83930afdd42a6ddc21c9
/system/update_engine/payload_generator/ab_generator.cc
|
| abdc0115548985d8ca3f3f2aeb1283741ce409a6 |
|
22-Oct-2015 |
Alex Deymo <deymo@google.com> |
Fix building delta_generator.
Previous CL had a few include renames missing.
Bug: None
Test: sudo emerge update_engine
Change-Id: Ifcdf5f11d52561cf3f976f18c0d66b95dd3c0d66
/system/update_engine/payload_generator/ab_generator.cc
|
| 3f39d5cc753905874d8d93bef94f857b8808f19e |
|
13-Oct-2015 |
Alex Vakulenko <avakulenko@google.com> |
update_engine: Rename "chromeos" -> "brillo" in include paths and namespaces
libchromeos is transitioning to libbrillo and chromeos namespaces
and include directory is changing to brillo.
Bug: 24872993
Change-Id: I770659a95be380a50fe3b2ba9f91d65818f40945
/system/update_engine/payload_generator/ab_generator.cc
|
| 625406cee9a90ac2ed895f480286b7f0e8497f38 |
|
17-Sep-2015 |
Sen Jiang <senj@google.com> |
payload_generator: Remove PartitionName enum.
payload_generator should be able to generate payload for any partition name
in major version 2, so we should use a string for partition name.
Bug: 23620828
TEST=cros_workon_make update_engine --test
Change-Id: Iee92af5d82e74b9be6ae793850a4ae6c31e008c5
/system/update_engine/payload_generator/ab_generator.cc
|
| ebdf17d4202c67933764135bfc1cece629829201 |
|
19-Aug-2015 |
Sen Jiang <senj@google.com> |
Generate update operations per partition
In order to support multiple partitions, OperationsGenerator should take one
partition at a time, and be called multiple times. This also reduce code
repetition.
TEST=FEATURES=test emerge-peppy update_engine; cros_generate_update_payload
generate same payload as before.
BUG: 23037923
Change-Id: Iefbe87401217873e6dbdc616791667cfd1ec76a3
/system/update_engine/payload_generator/ab_generator.cc
|
| aea4c1cea20dda7ae7e85fc8924a2d784f70d806 |
|
20-Aug-2015 |
Alex Deymo <deymo@google.com> |
Re-license update_engine to Apache2
This patch automatically replaced the license on all text files from
Chromium OS (BSD style) to AOSP (Apache2), keeping the original year as
a reference.
The license header was added to .gyp and .gypi files, the NOTICE was
replaced with a copy of the Apache2 license and MODULE_LICENSE_* file
was updated.
BUG=b/23084294
TEST=grep 'Chromium OS Authors' doesn't find anything.
Change-Id: Ie5083750755f5180a8a785b24fe67dbf9195cd10
/system/update_engine/payload_generator/ab_generator.cc
|
| a12ee11c78ac6d7c2605921a4006b6a7416e0c35 |
|
13-Aug-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Move InstallOperation to the top level.
The InstallOperation message in the protobuf is a nested message
inside the DeltaArchiveManifest message, making all references to
operation types be very long names like
DeltaArchiveManifest_InstallOperation_Type_REPLACE_BZ while most other
messages are not nested in the DeltaArchiveManifest message.
To improve readability and to prepare for future update metadata
changes, this patch moves the InstallOperation message to the top level
and replaces all references to operation types with the new shorter
version like InstallOperation::REPLACE_BZ.
This change only impacts the scope of the generated classes and the
serialized format of the protobuf. This exact same question was
addressed by protobuf maintainers here:
https://groups.google.com/forum/#!topic/protobuf/azWAPa6hP4A
Finally coding style and indentation was automatically updated due to
the shorter names.
BUG=b:23179128
TEST=Unittest still pass.
Change-Id: I55add54265934cd1fd3e9cb786c5d3f784902d17
Reviewed-on: https://chromium-review.googlesource.com/293504
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/ab_generator.cc
|
| 8cc502dacbccdab96824d42287f230ce04004784 |
|
10-Aug-2015 |
Sen Jiang <senj@chromium.org> |
update_engine: Change OperationsGenerator to use BlobFileWriter
BUG=chromium:517280
TEST=Unit test for BlobFileWriter
Change-Id: Ib49925676331acee97ff6b4cec38a81ca8b157a1
Reviewed-on: https://chromium-review.googlesource.com/291441
Tested-by: Sen Jiang <senj@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Sen Jiang <senj@chromium.org>
/system/update_engine/payload_generator/ab_generator.cc
|
| 2d3b2d635e50c6886e285afb86c3187d9e0bd360 |
|
18-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Implement soft/hard chunk size limits.
The current approach uses a single chunk_size value, that defaults to
-1 (or "whole files") for delta payloads. The "whole files" default is
the preferred value since BSDIFF and SOURCE_BSDIFF operations (the most
common operations in real delta payloads) will benefit from having
access to the whole file in the old and new partition, in particular in
cases like the chrome binary (~120 MiB in size).
On the other hand, very big chunks have a big memory footprint in most
cases. Current implementations of BSDIFF, REPLACE_BZ and REPLACE will
require as much private memory as the destination chunk_size or more.
Because of this and due to the lack of old data, a small chunk_size
(1 MiB) is used for full payloads.
To break this tension between having a big chunk_size for operations
that will benefit from it versus having a small chunk_size for cases
where it doesn't change anything, this patch introduces two chunk
size values: a hard and soft limit.
The hard_chunk_size has the same meaning as the old chunk_size: no
operation should have a destination bigger than the provided hard
limit. The soft_chunk_size is the preferred chunk size for an
operation when a bigger chunk will not benefit significantly the
final payload size. Having a small chunk size for operations like
REPLACE_BZ, REPLACE, SOURCE_COPY and MOVE is important to keep the
memory footprint low when it the extra memory is not required.
The new soft_chunk_size limit is used when merging operations
(previously hard-coded to 1 MiB) and when generating new
operations for zeroed and moved blocks.
BUG=chromium:485397
TEST=Run cros_generate_update_payload for full and delta payloads.
Inspected the chunk size in the resulting payloads.
Change-Id: I370048a81913ad23a151cfef6690627b7fff7277
Reviewed-on: https://chromium-review.googlesource.com/286568
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/ab_generator.cc
|
| 896fdbaa1cc9a422ac36ef455d02c3cef1f27816 |
|
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Prevent the InplaceGenerator to add MOVEs to block 0.
While we normally don't generate MOVE operations to/from the block 0,
the cycle breaker logic can use the block 0 as scratch space, creating
MOVE operations to and from the block 0.
This patch prevents it from picking the block 0 as scratch space and
logs a message when that would happend.
BUG=chromium:480751,chromium:500423
TEST=Added unittest.
Change-Id: I91f1b3c426a9d06aae5b685e2e901c7e448d8677
Reviewed-on: https://chromium-review.googlesource.com/286623
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/ab_generator.cc
|
| 3b2c7d0e6d859e6fc75c82b3417f87cf5968a49d |
|
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Sort full operations by destination.
The inplace generator moves all the full operations to the end of the
list. This patch sorts those operations by the destination.
This patch also cleans up the Vertex class by using the new
AnnotatedOperation class instead of the operation and file_name
separated.
BUG=None
TEST=Unittest still pass. minor_version=1 full operations appear in
order according to "cros payload".
Change-Id: Ia0c15939086cec52dc855cbc3afa913f8cbebf6b
Reviewed-on: https://chromium-review.googlesource.com/286213
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/ab_generator.cc
|
| b42b98db059a12c44110588fc0b3d5f82d32a2f8 |
|
07-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Load minor_version from the FilesystemInterface.
In the normal use case, cros_generate_update_payload doesn't pass the
requested minor_version to delta_generator which means we need to detect
it from the old image. To detect it, we where looking at the
update_engine.conf file in the mounted source directory. This patch uses
the recently added FilesystemInterface::LoadSettings() method to read
the update_engine.conf settings from the source rootfs partition.
In order to make this information available when autodetecting the
minor_version and to help future FilesystemInterface mocking for
testing, we move the FilesystemInterface instance from inside the
OperationsGenerator class to the PartitionConfig, so the filesystems are
opened before calling GenerateUpdatePayloadFile.
This patch then deprecates the --old_dir flag removing the requirement
to run delta_generator and cros_generate_update_payload as root when
generating deltas.
BUG=chromium:305832
TEST=Ran cros_generate_update_payload to generate deltas from old (link
FSI) and new (ToT link) images.
Change-Id: I915679d62aae2d1a2acee68cc8c1ec691cc363ad
Reviewed-on: https://chromium-review.googlesource.com/283643
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/ab_generator.cc
|
| 14158570d3995008dc93a628004118b87a6bca01 |
|
13-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Split delta_diff_generator file.
The DeltaDiffGenerator class includes both an OperationsGenerator using the
A-to-B operations and a set of common methods used also by the inplace generator.
The delta_diff_generator.{h,cc} files also include a single function to generate
the payload (GenerateUpdatePayloadFile) that centralizes the logic of generating
the operations and writing the payload.
This patch splits these three parts in different files. The common delta diff
function are moved to the delta_diff_utils.{h,cc} files. The operations generator
class that uses A-to-B operations is now in a new ab_generator.{h,cc} pair of files
that implement the ABGenerator() class. Finally, the payload file writing methods
are now in a single PayloadFile class.
This allow us to create payload files without the need to generate images and
their deltas. This will be used in a follow up CL to remove the image generation
logic from the unittests.
BUG=chromium:351589
TEST=Ran unittests. Regenerate a payload with and without this patch; got the same results.
Change-Id: I6816d2c805ba8c0c5c9423c720131a100a15ebaa
Reviewed-on: https://chromium-review.googlesource.com/280838
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/ab_generator.cc
|