| 5c6bb1d8f89b87e782e26fe74789b6f510704394 |
|
27-Jul-2016 |
Chih-Hung Hsieh <chh@google.com> |
Fix clang-tidy performance warnings in update_engine.
* Use const reference type for parameters, local variables,
and for-loop index variables to avoid unnecessary copy.
* Convert some for-loops to for-rang loops.
Bug: 30407689
Bug: 30413223
Bug: 30413862
Change-Id: I78996b3f799639fc57ced45e110807625be7dcce
Test: build with WITH_TIDY=1
/system/update_engine/payload_generator/inplace_generator.h
|
| 05871fab8b32938ad66ce230250ff6db5563bd7a |
|
01-Jun-2016 |
Alex Deymo <deymo@google.com> |
Allow to shrink ext2 filesystems.
There was a function a long time ago that didn't handle shrinking an
ext2 filesystem when computing the diff of the blocks that were not
part of a file nor metadata (mostly empty space blocks). This is now
handled in a different way, which is not related to how the filesystems
are parsed in the ABGenerator.
This patch removes the check that prevents the delta generator from
generating those payloads and fixes the inplace generator for partitions
that shrink in size.
Bug: 28846535
TEST=Generated a payload to a smaller ext2 system image, both for
minor version 1 and 2+.
Change-Id: Ide408c48b0032dbe478c8e348c40e1a0b5665ea8
/system/update_engine/payload_generator/inplace_generator.h
|
| ebdf17d4202c67933764135bfc1cece629829201 |
|
19-Aug-2015 |
Sen Jiang <senj@google.com> |
Generate update operations per partition
In order to support multiple partitions, OperationsGenerator should take one
partition at a time, and be called multiple times. This also reduce code
repetition.
TEST=FEATURES=test emerge-peppy update_engine; cros_generate_update_payload
generate same payload as before.
BUG: 23037923
Change-Id: Iefbe87401217873e6dbdc616791667cfd1ec76a3
/system/update_engine/payload_generator/inplace_generator.h
|
| aea4c1cea20dda7ae7e85fc8924a2d784f70d806 |
|
20-Aug-2015 |
Alex Deymo <deymo@google.com> |
Re-license update_engine to Apache2
This patch automatically replaced the license on all text files from
Chromium OS (BSD style) to AOSP (Apache2), keeping the original year as
a reference.
The license header was added to .gyp and .gypi files, the NOTICE was
replaced with a copy of the Apache2 license and MODULE_LICENSE_* file
was updated.
BUG=b/23084294
TEST=grep 'Chromium OS Authors' doesn't find anything.
Change-Id: Ie5083750755f5180a8a785b24fe67dbf9195cd10
/system/update_engine/payload_generator/inplace_generator.h
|
| a12ee11c78ac6d7c2605921a4006b6a7416e0c35 |
|
13-Aug-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Move InstallOperation to the top level.
The InstallOperation message in the protobuf is a nested message
inside the DeltaArchiveManifest message, making all references to
operation types be very long names like
DeltaArchiveManifest_InstallOperation_Type_REPLACE_BZ while most other
messages are not nested in the DeltaArchiveManifest message.
To improve readability and to prepare for future update metadata
changes, this patch moves the InstallOperation message to the top level
and replaces all references to operation types with the new shorter
version like InstallOperation::REPLACE_BZ.
This change only impacts the scope of the generated classes and the
serialized format of the protobuf. This exact same question was
addressed by protobuf maintainers here:
https://groups.google.com/forum/#!topic/protobuf/azWAPa6hP4A
Finally coding style and indentation was automatically updated due to
the shorter names.
BUG=b:23179128
TEST=Unittest still pass.
Change-Id: I55add54265934cd1fd3e9cb786c5d3f784902d17
Reviewed-on: https://chromium-review.googlesource.com/293504
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 8cc502dacbccdab96824d42287f230ce04004784 |
|
10-Aug-2015 |
Sen Jiang <senj@chromium.org> |
update_engine: Change OperationsGenerator to use BlobFileWriter
BUG=chromium:517280
TEST=Unit test for BlobFileWriter
Change-Id: Ib49925676331acee97ff6b4cec38a81ca8b157a1
Reviewed-on: https://chromium-review.googlesource.com/291441
Tested-by: Sen Jiang <senj@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Sen Jiang <senj@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 2d3b2d635e50c6886e285afb86c3187d9e0bd360 |
|
18-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Implement soft/hard chunk size limits.
The current approach uses a single chunk_size value, that defaults to
-1 (or "whole files") for delta payloads. The "whole files" default is
the preferred value since BSDIFF and SOURCE_BSDIFF operations (the most
common operations in real delta payloads) will benefit from having
access to the whole file in the old and new partition, in particular in
cases like the chrome binary (~120 MiB in size).
On the other hand, very big chunks have a big memory footprint in most
cases. Current implementations of BSDIFF, REPLACE_BZ and REPLACE will
require as much private memory as the destination chunk_size or more.
Because of this and due to the lack of old data, a small chunk_size
(1 MiB) is used for full payloads.
To break this tension between having a big chunk_size for operations
that will benefit from it versus having a small chunk_size for cases
where it doesn't change anything, this patch introduces two chunk
size values: a hard and soft limit.
The hard_chunk_size has the same meaning as the old chunk_size: no
operation should have a destination bigger than the provided hard
limit. The soft_chunk_size is the preferred chunk size for an
operation when a bigger chunk will not benefit significantly the
final payload size. Having a small chunk size for operations like
REPLACE_BZ, REPLACE, SOURCE_COPY and MOVE is important to keep the
memory footprint low when it the extra memory is not required.
The new soft_chunk_size limit is used when merging operations
(previously hard-coded to 1 MiB) and when generating new
operations for zeroed and moved blocks.
BUG=chromium:485397
TEST=Run cros_generate_update_payload for full and delta payloads.
Inspected the chunk size in the resulting payloads.
Change-Id: I370048a81913ad23a151cfef6690627b7fff7277
Reviewed-on: https://chromium-review.googlesource.com/286568
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 896fdbaa1cc9a422ac36ef455d02c3cef1f27816 |
|
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Prevent the InplaceGenerator to add MOVEs to block 0.
While we normally don't generate MOVE operations to/from the block 0,
the cycle breaker logic can use the block 0 as scratch space, creating
MOVE operations to and from the block 0.
This patch prevents it from picking the block 0 as scratch space and
logs a message when that would happend.
BUG=chromium:480751,chromium:500423
TEST=Added unittest.
Change-Id: I91f1b3c426a9d06aae5b685e2e901c7e448d8677
Reviewed-on: https://chromium-review.googlesource.com/286623
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 3b2c7d0e6d859e6fc75c82b3417f87cf5968a49d |
|
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Sort full operations by destination.
The inplace generator moves all the full operations to the end of the
list. This patch sorts those operations by the destination.
This patch also cleans up the Vertex class by using the new
AnnotatedOperation class instead of the operation and file_name
separated.
BUG=None
TEST=Unittest still pass. minor_version=1 full operations appear in
order according to "cros payload".
Change-Id: Ia0c15939086cec52dc855cbc3afa913f8cbebf6b
Reviewed-on: https://chromium-review.googlesource.com/286213
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| f6165357bfbb74b22a4da5780a474f439611e167 |
|
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Resolve read-after-write dependencies in the kernel.
BUG=chromium:510909
TEST=Ran delta_generator on rambi from 6415.1.0 to 7272.0.0 dev-channel.
Resulting kernel operations are in order.
Change-Id: I8843c1497b095af7520404d9845db9037bb5e262
Reviewed-on: https://chromium-review.googlesource.com/286212
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 14158570d3995008dc93a628004118b87a6bca01 |
|
13-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Split delta_diff_generator file.
The DeltaDiffGenerator class includes both an OperationsGenerator using the
A-to-B operations and a set of common methods used also by the inplace generator.
The delta_diff_generator.{h,cc} files also include a single function to generate
the payload (GenerateUpdatePayloadFile) that centralizes the logic of generating
the operations and writing the payload.
This patch splits these three parts in different files. The common delta diff
function are moved to the delta_diff_utils.{h,cc} files. The operations generator
class that uses A-to-B operations is now in a new ab_generator.{h,cc} pair of files
that implement the ABGenerator() class. Finally, the payload file writing methods
are now in a single PayloadFile class.
This allow us to create payload files without the need to generate images and
their deltas. This will be used in a follow up CL to remove the image generation
logic from the unittests.
BUG=chromium:351589
TEST=Ran unittests. Regenerate a payload with and without this patch; got the same results.
Change-Id: I6816d2c805ba8c0c5c9423c720131a100a15ebaa
Reviewed-on: https://chromium-review.googlesource.com/280838
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 6b9e38ef1180efe55e4a82bb18536d1b53fe493d |
|
05-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Use the FilesystemInterface to generate deltas.
This patch removes the FilesystemIterator used to iterate the file data
blocks, the metadata.{cc,h} files used to iterate the metadata blocks and
the ReadUnwrittenBlocks() method used to process the rest of the blocks.
Instead, these three cases are handled by the same DeltaReadFilesystem()
method using the FilesystemInterface abstraction for ext2.
The sub-file block handling was changed from (filename, offset, size) to
just the list of blocks (or Extents) and the parsing of the filesystem
was removed from the methods that produce operations. The filename is
kept just as a label for logging purposes.
This patch implies that the filesystem doesn't need to be mounted in
order to generate the payload, as all the data is accessed as blocks
in a partition. The mountpoint was removed from all the operations
generators, but it is still required to mount the old filesystem to
detect the minor version supported by the old update_engine. Also,
since the list of blocks is never accessed using the FIGETBSZ ioctl,
no payload_generator/ test require root privileges, nor the
delta_generator binary.
BUG=chromium:305832,chromium:331965
TEST=Updated unittest to use extents.
Change-Id: Ia2ea9433190258f70e3bce09f896b18326a7abf9
Reviewed-on: https://chromium-review.googlesource.com/275804
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 568734533c25a5783ea004aeb0da38244dcd3e5b |
|
28-Mar-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Remove sparse hole extents.
Whenever there is an extent beginning with a sparse hole, that extent
will be filtered out. Extents are now read in from the partition rather
than the file itself in ReadFileToDiff. src/dst_length are set to the
number of blocks in src/dst_extents * block_size.
This was tested manually with _GenerateSinglePayload to generate a delta
payload between test images.
BUG=chromium:469792,chromium:474497
TEST=`FEATURES=test emerge-link update_engine` and manual testing.
Change-Id: I384ec3f16f7fd9087159817358308f22d29e9edf
Reviewed-on: https://chromium-review.googlesource.com/264442
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 703022b71fc6a89796f2f97448b1a419007a52ca |
|
07-Apr-2015 |
Shawn N <shawnn@chromium.org> |
Revert "update_engine: Remove sparse hole extents."
Speculatively reverting commit 96b659d794be39762e8e7fd9f72fe6f723e21ec8 due to crbug.com/474497.
Change-Id: I17fd91c8568b30eafea9e70c9f2255ac2dc459a6
Reviewed-on: https://chromium-review.googlesource.com/264306
Reviewed-by: Shawn N <shawnn@chromium.org>
Commit-Queue: Shawn N <shawnn@chromium.org>
Tested-by: Shawn N <shawnn@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 8e2c4ef89b1c2c8779d003b40c70c75db1839bfb |
|
28-Mar-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Remove sparse hole extents.
Whenever there is an extent beginning with a sparse hole, that extent
will be filtered out. Extents are now read in from the partition rather
than the file itself in ReadFileToDiff.
BUG=chromium:469792
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: I615df120f6e46f66d500128561e0ddfcd407c13a
Reviewed-on: https://chromium-review.googlesource.com/262965
Reviewed-by: Allie Wood <alliewood@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 477aec2166a571cbe28081d806c3226e8b31b6e9 |
|
25-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Refactor OperationsGenerator into a base class.
This refactor cleans up the interface of the algorithms that generate
the list of rootfs and kernel operations removing the mention of a
Graph from it. The Graph class is only used by the in-place generator
because it requires to keep track of dependencies between operations
reading or writting the same block. The full update generator, using
only REPLACE or REPLACE_BZ doesn't need to use a graph to do that, but
in order to reuse some code, the interface was hacked that way.
This patch now uses two vectors of "AnnotatedOperations", which are
a mere InstallOperation as defined by the .proto file plus a name
used for logging purposes only. Both rootfs and kernel operations
have now the same type on the interface, allowing to share common
functions handling those.
BUG=chromium:331965
TEST=FEATURES=test emerge-link update_engine
Change-Id: I78566bbecb948634b7ecc8d086766ce67a79b43e
Reviewed-on: https://chromium-review.googlesource.com/262281
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| 9b244df41f1bdaddd87b7dbd8e1559556059ed1b |
|
12-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Refactor operation generation methods.
We have so far three methods to generate the list of operations to
update the kernel and the rootfs partitions, which are different
strategies: full (using only REPLACE and REPLACE_BZ), delta "in-place"
(using also MOVE and BSDIFF) and "source" delta, or delta minor
version 2 (using SOURCE_COPY and SOURCE_BSDIFF instead).
As we consider ways to support updates for squashfs, the two delta
methods don't support filesystems other than ext2, so we need to use
yet another different strategy for those.
This patch splits the code that generates the list of operations for
the kernel and the rootfs in three methods with the same interface.
No functional changes were made. Some local functions were exposed
as public static methods. The new graph_types.cc is added to avoid
a link-time error when the kInvalidIndex member is not inlined.
BUG=chromium:430950
TEST=FEATURES=test emerge-link update_engine
Change-Id: Ib25ff7a6f17d0990637596dcd6b59568b9e15d26
Reviewed-on: https://chromium-review.googlesource.com/259462
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| f1cbe1783dd025bd7243a8df12f20548196cc023 |
|
06-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Refactor payload generation request args in a class.
This patch introduces a new struct PayloadGenerationConfig encapsulating
the payload generation request performed by the user in the command
line.
Since we have several payload generation strategies that are triggered
based on the source image version and if there is an image version at
all, encapsulating the request in a single struct helps splitting the
different strategies in independent classes and pass the payload
generation configuration struct with the request to those classes.
BUG=chromium:430950
TEST=FEATURES=test emerge-link update_engine
Change-Id: Ie960c6510bf02d8e10f5e1adaa847a2048c59598
Reviewed-on: https://chromium-review.googlesource.com/258520
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|
| cd514b531a38da1497630fa68b6e3ef45871893d |
|
19-Feb-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Refactor inplace payload generator algorithm code.
Create a class, InplaceGenerator, that contains all functionality
related to the inplace delta generation method (minor version 1).
BUG=chromium:459701
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: Ib742f70030d6c2fcb1cc3138e0f4aef54eca6975
Reviewed-on: https://chromium-review.googlesource.com/251621
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/inplace_generator.h
|