History log of /system/update_engine/payload_generator/delta_diff_generator.cc
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
20bdc70e540e021c4a26553dfdbfd2fabe8ea432 08-Dec-2016 Alex Deymo <deymo@google.com> Parse Android .map files for unknown filesystem.

When generating a filesystem during the Android build, we also generate
a text file with .map extension with the list of files and the blocks
in the filesystem they are located.

For filesystems unsupported in delta_generator (like squashfs) we use
this text file to produce efficient delta payloads.

Bug: 28150981
Test: Added unittest for parsing. Generated a delta payload of a squashfs image.

Change-Id: I154e72ac785c6f508290daa901fa7958b446c010
/system/update_engine/payload_generator/delta_diff_generator.cc
05871fab8b32938ad66ce230250ff6db5563bd7a 01-Jun-2016 Alex Deymo <deymo@google.com> Allow to shrink ext2 filesystems.

There was a function a long time ago that didn't handle shrinking an
ext2 filesystem when computing the diff of the blocks that were not
part of a file nor metadata (mostly empty space blocks). This is now
handled in a different way, which is not related to how the filesystems
are parsed in the ABGenerator.

This patch removes the check that prevents the delta generator from
generating those payloads and fixes the inplace generator for partitions
that shrink in size.

Bug: 28846535
TEST=Generated a payload to a smaller ext2 system image, both for
minor version 1 and 2+.

Change-Id: Ide408c48b0032dbe478c8e348c40e1a0b5665ea8
/system/update_engine/payload_generator/delta_diff_generator.cc
a4073ef63482fd08c3678982f7d153360b088094 23-Mar-2016 Alex Deymo <deymo@google.com> Introduce PayloadVersion struct for version information.

This new little struct encapsulates the version information (major and
minor version numbers) and the zlib fingerprint information. Now,
instead of querying throughout if the version number is certain value,
we ask the PayloadVersion struct whether certain operation is allowed in
that version or not. This moves the logic of what's supported and
what's not to a single place and eliminates the need to pass several
booleans to the helper functions.

Bug: None
TEST=Unittest still pass.

Change-Id: Iaa6e7bc032db6479fdeab72255d7062fca1c07e5
/system/update_engine/payload_generator/delta_diff_generator.cc
dcbc0ae6a5c48b40a1bc7d3c2ed62ec2a9fe7748 18-Mar-2016 Sen Jiang <senj@google.com> Remove GetFilesystemSize.

We don't need to get filesystem size now, because the files passed to
delta_generator is already resized properly.

Also moved Is*Filesystem() to delta_diff_utils.

Test: ./update_engine_unittests
Bug: None

Change-Id: I442d69d1820d08c0f3bd63e7f1986fc4b0d6655e
/system/update_engine/payload_generator/delta_diff_generator.cc
55c4f9ba7f5c59e3345f2c1869464433ffa8dc2b 10-Feb-2016 Sen Jiang <senj@google.com> Include IMGDIFF operation in minor version 4 or up.

Try imgdiff if both source and target contains gzip.

Test: unit test & generated a payload with boot.img
Bug: 26675118
Change-Id: I2861d9b953ffbdec44fdfb42cff5687698ea205e
/system/update_engine/payload_generator/delta_diff_generator.cc
39910dcd1d68987ccee7c3031dc269233a8490bb 10-Nov-2015 Alex Deymo <deymo@google.com> Split payload application code into a subdirectory.

This patch splits from the main libupdate_engine code the part that
is strictly used to download and apply a payload into a new static
library, moving the code to subdirectories. The new library is divided
in two subdirectories: common/ and payload_consumer/, and should not
depend on other update_engine files outside those two subdirectories.
The main difference between those two is that the common/ tools are more
generic and not tied to the payload consumer process, but otherwise they
are both compiled together.

There are still dependencies from the new libpayload_consumer library
into the main directory files and DBus generated files. Those will be
addressed in follow up CLs.

Bug: 25197634
Test: FEATURES=test emerge-link update_engine; `mm` on Brillo.

Change-Id: Id8d0204ea573627e6e26ca9ea17b9592ca95bc23
/system/update_engine/payload_generator/delta_diff_generator.cc
82352f97b4621dbf6af8308ff0b0b17b0968b53a 10-Nov-2015 Sen Jiang <senj@google.com> delta_generator: Include per-operation source hash.

This adds a src_sha256_hash field in the operation if minor version is 3.

Bug: 23182225
TEST=cros_workon_make update_engine --test
TEST=Generated a minor version 3 payload, the manifest is about 50% larger.

Change-Id: I0e363fca4de24bae64ac83930afdd42a6ddc21c9
/system/update_engine/payload_generator/delta_diff_generator.cc
981eb1151f42480ea10cb5283ab2108454a16c92 26-Aug-2015 Sen Jiang <senj@google.com> Support multiple partitions in ImageConfig

ImageConfig now have a vector of partitions instead of hardcoded rootfs and
kernel.

Bug: 23420126
TEST=cros_workon_make update_engine --test

Change-Id: Id226cc04628b44f1fcbd58f03038809867bf9e40
/system/update_engine/payload_generator/delta_diff_generator.cc
b9ef491e87c17d809368ede0196569d8a106b963 22-Sep-2015 Sen Jiang <senj@google.com> Move partition info to PartitionUpdate in payload verison 2.

old_kernel_info, old_rootfs_info is now old_partition_info.
new_kernel_info, new_rootfs_info is now new_partition_info.

AddPartitionOperations is renamed to AddPartition because it now adds
partition info too.

Bug: 23694580
TEST=cros_workon_make update_engine --test
Generated a version 2 payload with delta_generator.

Change-Id: I07c7fc21198307e935679e62861cdac4b406c84f
/system/update_engine/payload_generator/delta_diff_generator.cc
70a6ab091beb261a501012bc09955de452dd4736 28-Aug-2015 Sen Jiang <senj@google.com> Move install operations to partitions field in major version 2.

install_operations and kernel_install_operations will be empty in major
version 2, they are now present in partitions field.

partition_info are still in old location, more refactoring are needed before
moving them.

This patch also moves the kLegancyPartitionName constants from install_plan.h
to payload_constants.h.

TEST=cros_generate_update_payload generate same payload version 1 as before.
Call delta_generator manually with --major_version=2, then cros payload show.
cros_workon_make update_engine --test

Bug: 23694580
Change-Id: If9e9531656480b1174e13b39af7ecb6a81060aac
/system/update_engine/payload_generator/delta_diff_generator.cc
ebdf17d4202c67933764135bfc1cece629829201 19-Aug-2015 Sen Jiang <senj@google.com> Generate update operations per partition

In order to support multiple partitions, OperationsGenerator should take one
partition at a time, and be called multiple times. This also reduce code
repetition.

TEST=FEATURES=test emerge-peppy update_engine; cros_generate_update_payload
generate same payload as before.

BUG: 23037923
Change-Id: Iefbe87401217873e6dbdc616791667cfd1ec76a3
/system/update_engine/payload_generator/delta_diff_generator.cc
aea4c1cea20dda7ae7e85fc8924a2d784f70d806 20-Aug-2015 Alex Deymo <deymo@google.com> Re-license update_engine to Apache2

This patch automatically replaced the license on all text files from
Chromium OS (BSD style) to AOSP (Apache2), keeping the original year as
a reference.

The license header was added to .gyp and .gypi files, the NOTICE was
replaced with a copy of the Apache2 license and MODULE_LICENSE_* file
was updated.

BUG=b/23084294
TEST=grep 'Chromium OS Authors' doesn't find anything.

Change-Id: Ie5083750755f5180a8a785b24fe67dbf9195cd10
/system/update_engine/payload_generator/delta_diff_generator.cc
8cc502dacbccdab96824d42287f230ce04004784 10-Aug-2015 Sen Jiang <senj@chromium.org> update_engine: Change OperationsGenerator to use BlobFileWriter

BUG=chromium:517280
TEST=Unit test for BlobFileWriter

Change-Id: Ib49925676331acee97ff6b4cec38a81ca8b157a1
Reviewed-on: https://chromium-review.googlesource.com/291441
Tested-by: Sen Jiang <senj@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Sen Jiang <senj@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
b42b98db059a12c44110588fc0b3d5f82d32a2f8 07-Jul-2015 Alex Deymo <deymo@chromium.org> update_engine: Load minor_version from the FilesystemInterface.

In the normal use case, cros_generate_update_payload doesn't pass the
requested minor_version to delta_generator which means we need to detect
it from the old image. To detect it, we where looking at the
update_engine.conf file in the mounted source directory. This patch uses
the recently added FilesystemInterface::LoadSettings() method to read
the update_engine.conf settings from the source rootfs partition.

In order to make this information available when autodetecting the
minor_version and to help future FilesystemInterface mocking for
testing, we move the FilesystemInterface instance from inside the
OperationsGenerator class to the PartitionConfig, so the filesystems are
opened before calling GenerateUpdatePayloadFile.

This patch then deprecates the --old_dir flag removing the requirement
to run delta_generator and cros_generate_update_payload as root when
generating deltas.

BUG=chromium:305832
TEST=Ran cros_generate_update_payload to generate deltas from old (link
FSI) and new (ToT link) images.

Change-Id: I915679d62aae2d1a2acee68cc8c1ec691cc363ad
Reviewed-on: https://chromium-review.googlesource.com/283643
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
14158570d3995008dc93a628004118b87a6bca01 13-Jun-2015 Alex Deymo <deymo@chromium.org> update_engine: Split delta_diff_generator file.

The DeltaDiffGenerator class includes both an OperationsGenerator using the
A-to-B operations and a set of common methods used also by the inplace generator.
The delta_diff_generator.{h,cc} files also include a single function to generate
the payload (GenerateUpdatePayloadFile) that centralizes the logic of generating
the operations and writing the payload.

This patch splits these three parts in different files. The common delta diff
function are moved to the delta_diff_utils.{h,cc} files. The operations generator
class that uses A-to-B operations is now in a new ab_generator.{h,cc} pair of files
that implement the ABGenerator() class. Finally, the payload file writing methods
are now in a single PayloadFile class.

This allow us to create payload files without the need to generate images and
their deltas. This will be used in a follow up CL to remove the image generation
logic from the unittests.

BUG=chromium:351589
TEST=Ran unittests. Regenerate a payload with and without this patch; got the same results.

Change-Id: I6816d2c805ba8c0c5c9423c720131a100a15ebaa
Reviewed-on: https://chromium-review.googlesource.com/280838
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
249fbf5be69e8de37fbac33c1570a58f902b1f6f 16-Jun-2015 Alex Deymo <deymo@chromium.org> update_engine: Avoid emitting a MOVE to 0.

Workaround the bug where we can't emit a MOVE to the block 0 by excluding
it from the inplace generator graph and re-adding the operation later.

BUG=chromium:500423
TEST=delta_generator from lulu-R43-6946.31.0 to R45-7174.0.0, test image.

Change-Id: I3f16c2ae58b0f31869a39af6a45f7798960c722b
Reviewed-on: https://chromium-review.googlesource.com/278020
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Trybot-Ready: Filipe Brandenburger <filbranden@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
1beda780333ce51d7872603b70712772eb2383fb 07-Jun-2015 Alex Deymo <deymo@chromium.org> update_engine: Remove unused IsIdempotentOperation().

All the minor-version=2 are idempotent, so since we switched to minor
version 2, this function becomes trivial now. This patch removes this
function from the code and moves the ExtentRanges class to the
payload_generator/ directory since now it is only used there.

BUG=None
TEST=Unittest still pass.

Change-Id: Ib9dbbdded0ca2ef2128bb6c470de7a00720c4038
Reviewed-on: https://chromium-review.googlesource.com/275806
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
35589c2b9a0e20b42661b132890128d8025c1954 07-Jun-2015 Alex Deymo <deymo@chromium.org> update_engine: Remove rootfs_mountpt from ImageConfig.

The mounted rootfs is not used anymore during opeartions generation so
this patch removes the setting from the ImageConfig. The --new_dir flag
is now deprecated because of this.

The --old_dir is still being used when no --minor_version is passed, to
autodetect the minor version of the old image.

With this patch, the rootfs and kernel partitions have a more generic
description in the config.

BUG=None
TEST=Unittest still pass.

Change-Id: I0398493a44f8c09159a93a716f2168f45e8f8df9
Reviewed-on: https://chromium-review.googlesource.com/275805
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
6b9e38ef1180efe55e4a82bb18536d1b53fe493d 05-Jun-2015 Alex Deymo <deymo@chromium.org> update_engine: Use the FilesystemInterface to generate deltas.

This patch removes the FilesystemIterator used to iterate the file data
blocks, the metadata.{cc,h} files used to iterate the metadata blocks and
the ReadUnwrittenBlocks() method used to process the rest of the blocks.
Instead, these three cases are handled by the same DeltaReadFilesystem()
method using the FilesystemInterface abstraction for ext2.

The sub-file block handling was changed from (filename, offset, size) to
just the list of blocks (or Extents) and the parsing of the filesystem
was removed from the methods that produce operations. The filename is
kept just as a label for logging purposes.

This patch implies that the filesystem doesn't need to be mounted in
order to generate the payload, as all the data is accessed as blocks
in a partition. The mountpoint was removed from all the operations
generators, but it is still required to mount the old filesystem to
detect the minor version supported by the old update_engine. Also,
since the list of blocks is never accessed using the FIGETBSZ ioctl,
no payload_generator/ test require root privileges, nor the
delta_generator binary.

BUG=chromium:305832,chromium:331965
TEST=Updated unittest to use extents.

Change-Id: Ia2ea9433190258f70e3bce09f896b18326a7abf9
Reviewed-on: https://chromium-review.googlesource.com/275804
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
52490e776a9d34a94fb18ab1fe7d416425e94dda 04-Jun-2015 Alex Deymo <deymo@chromium.org> update_engine: Further move Extent manipulation to extent_utils.

This patch moves more Extent manipulation functions to extent_utils.
It moves NormalizeExtents() and creates a new ExtentsSublist() function
that will be used by a follow up CL.

BUG=None
TEST=Added unittests.

Change-Id: Icf0ef0af208aa45c9f44e1a73662b3efd8bbbb45
Reviewed-on: https://chromium-review.googlesource.com/275801
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
5c6c65570013bbdbd67f9bf6391dd295ef5b5ee6 03-Jun-2015 Alex Deymo <deymo@chromium.org> update_engine: Split Extent utils from graph_utils.

"Graph" related utils should only concern parts of the code using the
inplace generator, since other generators don't use a dependency graph.

This patch splits the Extent related utils from the graph related ones
creating a new extent_utils.h file.

BUG=None
TEST=unittest still pass.

Change-Id: I0941698b0a47a6cc222e8dc062fc54eb3cdf4de2
Reviewed-on: https://chromium-review.googlesource.com/274899
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
46bd4e20c20d6cb1a2f18d39237b9345d576669e 14-May-2015 Gilad Arnold <garnold@chromium.org> update_engine: Fix splitting and merging of operation.

As previously implemented, these features retained the original
operation type, thus potentially violating an invariant of the delta
format by which an operation must have the minimum possible size. This
could take form in either direction; in one case, we've seen this fail
payload verification when a large REPLACE_BZ was split into small
operations and the compressed blob of one was larger than the size of
the written blocks (i.e. compression was disadvantageous).

Instead, when splitting and merging operations we now choose an optimal
operation type among REPLACE and REPLACE_BZ. This means that every
split/merged data blob is compressed and its size compared to its
uncompressed form, storing the smaller of the two and setting the
operation type accordingly.

This also refactors unit tests and adds more test cases to ensure that
operation types are chosen optimally and that the conversion is correct
in both directions.

Finally, this re-enables the splitting of REPLACE_BZ operations, which
was disabled (CL:270581) to prevent payload breakages.

BUG=chromium:487595
TEST=Unit tests

Change-Id: Ib348b76a96eba6dd612cafa412e1231c43389f96
Reviewed-on: https://chromium-review.googlesource.com/271277
Trybot-Ready: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: David Zeuthen <zeuthen@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
14c0f1b1e2ada315b3ee63084435b899e51c5bc5 13-May-2015 Gilad Arnold <garnold@chromium.org> update_engine: Disable splitting of fragmented REPLACE_BZ operations.

The said feature was introduced in CL:269130 and violated an invariant
of the delta payload format. This leaves most of the logic in place but
disables invoking it during delta generation. It will be re-enabled once
the logic is fixed.

BUG=chromium:487436
TEST=Unit tests

Change-Id: I59bf79749f0548d2935aecf489849ac2f9cb1f1e
Reviewed-on: https://chromium-review.googlesource.com/270581
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
41e3474eebe3748a45aba4b297a7a313a1006c6c 11-May-2015 Gilad Arnold <garnold@chromium.org> update_engine: Merge contiguous operations (reprise).

This is reapplying CL:269604 with a small fix that prevents it from
adding spurious src_length fields when merging payload operations. This
also revises relevant unit tests to catch the said error. The original
change is described below.

Merges operations of type SOURCE_COPY, REPLACE, or REPLACE_BZ if they
have the same type, are contiguous, have only one destination extent,
and their combined block count does not exceed the chunk size.

BUG=chromium:461651
BUG=chromium:486497
TEST=Unit test (fails before fix, passes after)
TEST=Trybot

Change-Id: I9f3d7f653454e27177428f2d7c0e6485184a7f8b
Reviewed-on: https://chromium-review.googlesource.com/270268
Trybot-Ready: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
542032d92433045a66e888588d133a47c7fcf956 11-May-2015 Gilad Arnold <garnold@chromium.org> Revert "update_engine: Merge contiguous operations."

This reverts commit 6a3ea94766bb4972abbd629779914d80dada1245.

The CL is suspected to close the tree due to paygen failures.
See comments inline (delta_diff_generator.cc) for the culprit.

BUG=chromium:486497
TEST=trybot

Change-Id: Iaed694a79371e7de208b0de90f8e9cefd61dd7e6
Reviewed-on: https://chromium-review.googlesource.com/270183
Trybot-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Cheng-Yi Chiang <cychiang@chromium.org>
Commit-Queue: Cheng-Yi Chiang <cychiang@chromium.org>
Tested-by: Cheng-Yi Chiang <cychiang@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
58455ae7fe87a312ae648d871a92e1485d0e9989 04-May-2015 Allie Wood <alliewood@chromium.org> update_engine: Merge contiguous operations.

Merges operations of type SOURCE_COPY, REPLACE, or REPLACE_BZ if they
have the same type, are contiguous, have only one destination extent,
and their combined block count does not exceed the chunk size.

BUG=chromium:461651
TEST=`cros flash --src-image-to-delta`, checking generated payloads
using `cros payload show --list_ops`, unit tests.
CQ-DEPEND=CL:269134

Change-Id: I5f71e6828561f2e0a6a8ab3e3aae4c362ea2ddde
Reviewed-on: https://chromium-review.googlesource.com/269604
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
c4797c4c84bd7b25c83c841d65d2599a587c8c10 08-May-2015 Gilad Arnold <garnold@chromium.org> update_engine: Delta generator should not read past old image size.

This fixes a problem that was introduced by CL:246670 and caused the
delta generator to fail when generating deltas from a source image
that's smaller than the target image.

The bug is fixed by ensuring that the reading and comparing of unused
blocks is not attempted for block indexes that exceed the size of the
old image. Instead, these blocks are assumed to be new and their content
packaged into the payload.

BUG=chromium:477876
TEST=Unit tests
TEST=delta_generator successfully applied to FSI version 2913.331.0

Change-Id: Icc3163cca25ebeaefe05c2ba852664b6f0504053
Reviewed-on: https://chromium-review.googlesource.com/270075
Trybot-Ready: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
db1d5a6204ead78024bcfc854cbf084cc53f1734 04-May-2015 Allie Wood <alliewood@chromium.org> update_engine: Sort operations by output block.

After fragmenting SOURCE_COPY, REPLACE, and REPLACE_BZ operations,
payload generator will sort the operations by the start block of their
first destination extents. This reduces seeking when writing during
payload application.

BUG=chromium:461649
TEST=`cros flash --src-image-to-delta`, checking generated payloads
using `cros payload show --list_ops`.
CQ-DEPEND=CL:269130

Change-Id: I1e2d8e940cd0ce66a1ec62711653ba8b34b0f0a5
Reviewed-on: https://chromium-review.googlesource.com/269134
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
0d73e03afb30dd402255fd030293b145de492af6 02-May-2015 Allie Wood <alliewood@chromium.org> update_engine: Split fragmented REPLACE_BZ ops.

Splits REPLACE_BZ operations with multiple destination extents into
REPLACE_BZ operations with only one destination extent each.

BUG=chromium:461643
TEST=`cros flash --src-image-to-delta` and unit tests.
CQ-DEPEND=CL:268520

Change-Id: Ia5c055047c4bb863871507f32d028e4e28cb6af6
Reviewed-on: https://chromium-review.googlesource.com/269130
Reviewed-by: Allie Wood <alliewood@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
ba4c8b7d7b331be33604bb9e417f1108ba17a868 28-Apr-2015 Allie Wood <alliewood@chromium.org> update_engine: Split fragmented REPLACE operations.

Splits REPLACE operations with multiple destination extents into REPLACE
operations with only one destination extent each.

BUG=chromium:461643
TEST=`cros flash --src-image-to-delta` and unit tests.
CQ-DEPEND=CL:267697

Change-Id: Id14ec9f23a462f94821454289857e9f25e756d24
Reviewed-on: https://chromium-review.googlesource.com/268520
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
b6656a757e8379b63597c1417a8062c5cf416651 14-Apr-2015 Allie Wood <alliewood@chromium.org> update_engine: Split fragmented SOURCE_COPY ops.

Splits SOURCE_COPY operations with multiple destination extents into
individual SOURCE_COPY operations with only one destination extent each.

Also normalizes the extent lists before splitting the operation.

BUG=chromium:461641
TEST=`cros flash --src-image-to-delta` and unit tests.
CQ-DEPEND=CL:267360

Change-Id: I4d4fbe36165d98df7f36e4a1ddacc9b62ee97002
Reviewed-on: https://chromium-review.googlesource.com/267697
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
9f6f0a53509476be5f0ba152ec182d46a8d3c976 30-Mar-2015 Allie Wood <alliewood@chromium.org> update_engine: Implement SOURCE_COPY/SOURCE_BSDIFF operations.

Implement SOURCE_COPY and SOURCE_BSDIFF in DeltaPerformer. These new
operations are the same as MOVE and BSDIFF, respectively, except
that they read from the source partition rather than the destination
partition.

Also fills in the source paths in omaha_response_handler_action, which
were previously not being set.

BUG=chromium:461632
TEST=`FEATURES=test emerge-link update_engine` and manual testing.

Change-Id: I81eba5780c73ed920a5db72702d9d7c67dbaa673
Reviewed-on: https://chromium-review.googlesource.com/263747
Reviewed-by: Allie Wood <alliewood@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
568734533c25a5783ea004aeb0da38244dcd3e5b 28-Mar-2015 Allie Wood <alliewood@chromium.org> update_engine: Remove sparse hole extents.

Whenever there is an extent beginning with a sparse hole, that extent
will be filtered out. Extents are now read in from the partition rather
than the file itself in ReadFileToDiff. src/dst_length are set to the
number of blocks in src/dst_extents * block_size.

This was tested manually with _GenerateSinglePayload to generate a delta
payload between test images.

BUG=chromium:469792,chromium:474497
TEST=`FEATURES=test emerge-link update_engine` and manual testing.

Change-Id: I384ec3f16f7fd9087159817358308f22d29e9edf
Reviewed-on: https://chromium-review.googlesource.com/264442
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
703022b71fc6a89796f2f97448b1a419007a52ca 07-Apr-2015 Shawn N <shawnn@chromium.org> Revert "update_engine: Remove sparse hole extents."

Speculatively reverting commit 96b659d794be39762e8e7fd9f72fe6f723e21ec8 due to crbug.com/474497.

Change-Id: I17fd91c8568b30eafea9e70c9f2255ac2dc459a6
Reviewed-on: https://chromium-review.googlesource.com/264306
Reviewed-by: Shawn N <shawnn@chromium.org>
Commit-Queue: Shawn N <shawnn@chromium.org>
Tested-by: Shawn N <shawnn@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
8e2c4ef89b1c2c8779d003b40c70c75db1839bfb 28-Mar-2015 Allie Wood <alliewood@chromium.org> update_engine: Remove sparse hole extents.

Whenever there is an extent beginning with a sparse hole, that extent
will be filtered out. Extents are now read in from the partition rather
than the file itself in ReadFileToDiff.

BUG=chromium:469792
TEST=`FEATURES=test emerge-link update_engine`

Change-Id: I615df120f6e46f66d500128561e0ddfcd407c13a
Reviewed-on: https://chromium-review.googlesource.com/262965
Reviewed-by: Allie Wood <alliewood@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
477aec2166a571cbe28081d806c3226e8b31b6e9 25-Mar-2015 Alex Deymo <deymo@chromium.org> update_engine: Refactor OperationsGenerator into a base class.

This refactor cleans up the interface of the algorithms that generate
the list of rootfs and kernel operations removing the mention of a
Graph from it. The Graph class is only used by the in-place generator
because it requires to keep track of dependencies between operations
reading or writting the same block. The full update generator, using
only REPLACE or REPLACE_BZ doesn't need to use a graph to do that, but
in order to reuse some code, the interface was hacked that way.

This patch now uses two vectors of "AnnotatedOperations", which are
a mere InstallOperation as defined by the .proto file plus a name
used for logging purposes only. Both rootfs and kernel operations
have now the same type on the interface, allowing to share common
functions handling those.

BUG=chromium:331965
TEST=FEATURES=test emerge-link update_engine

Change-Id: I78566bbecb948634b7ecc8d086766ce67a79b43e
Reviewed-on: https://chromium-review.googlesource.com/262281
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
fdf005161523d9873a281f2a96d51dd4a3261f4f 02-Mar-2015 Allie Wood <alliewood@chromium.org> update_engine: Add fds for the source partitions.

Add new fds for the source partition, one for the rootfs and another for
the kernel. These are opened if we have a delta update with minor
version 2.

This change also adds support for changing the minor versions in tests.
There is a new private member, supported_minor_version_, which defaults
to kSupportedMinorPayloadVersion. It is set in the unit tests with calls
to SetSupportedVersion.

BUG=chromium:463573
TEST=`FEATURES=test emerge-link update_engine`

Change-Id: Ib988c91eb450b2499c615ae65b271691dfd9c651
Reviewed-on: https://chromium-review.googlesource.com/260950
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
de942f30cf986cf6bfc55fb5f9af6d7fea4ae51b 13-Mar-2015 Alex Deymo <deymo@chromium.org> update_engine: Generate valid delta files for squashfs.

This patch generates a valid (but inefficient) payload when a delta
payload is requested for a squashfs filesystem.

Since we are reusing the full payload generator for this purpose, this
patch relaxes the requirement of chunk_size to be positive in the
full_update_generator.cc code, replacing it by the default value of
1 MiB when nothing is passed.

BUG=chromium:430950
TEST=FEATURES=test emerge-link update_engine
TEST=cros_generate_update_payload for a delta payload works with
squashfs
TEST=`cros flash` a squashfs device with a delta payload works.

Change-Id: I88c7a571874c4c4697f528d44c52091aa1aed0c5
Reviewed-on: https://chromium-review.googlesource.com/260444
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
9b244df41f1bdaddd87b7dbd8e1559556059ed1b 12-Mar-2015 Alex Deymo <deymo@chromium.org> update_engine: Refactor operation generation methods.

We have so far three methods to generate the list of operations to
update the kernel and the rootfs partitions, which are different
strategies: full (using only REPLACE and REPLACE_BZ), delta "in-place"
(using also MOVE and BSDIFF) and "source" delta, or delta minor
version 2 (using SOURCE_COPY and SOURCE_BSDIFF instead).

As we consider ways to support updates for squashfs, the two delta
methods don't support filesystems other than ext2, so we need to use
yet another different strategy for those.

This patch splits the code that generates the list of operations for
the kernel and the rootfs in three methods with the same interface.

No functional changes were made. Some local functions were exposed
as public static methods. The new graph_types.cc is added to avoid
a link-time error when the kInvalidIndex member is not inlined.

BUG=chromium:430950
TEST=FEATURES=test emerge-link update_engine

Change-Id: Ib25ff7a6f17d0990637596dcd6b59568b9e15d26
Reviewed-on: https://chromium-review.googlesource.com/259462
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
f1cbe1783dd025bd7243a8df12f20548196cc023 06-Mar-2015 Alex Deymo <deymo@chromium.org> update_engine: Refactor payload generation request args in a class.

This patch introduces a new struct PayloadGenerationConfig encapsulating
the payload generation request performed by the user in the command
line.

Since we have several payload generation strategies that are triggered
based on the source image version and if there is an image version at
all, encapsulating the request in a single struct helps splitting the
different strategies in independent classes and pass the payload
generation configuration struct with the request to those classes.

BUG=chromium:430950
TEST=FEATURES=test emerge-link update_engine

Change-Id: Ie960c6510bf02d8e10f5e1adaa847a2048c59598
Reviewed-on: https://chromium-review.googlesource.com/258520
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
fffd23e1ed1903beeb893640c611001175380186 26-Feb-2015 Allie Wood <alliewood@chromium.org> update_engine: Payload generator emits SOURCE_COPY and SOURCE_BSDIFF ops.

When generating delta updates, payload generator will now emit
SOURCE_COPY and SOURCE_BSDIFF operations instead of MOVE and BSDIFF when
the minor version is 2.

BUG=chromium:461167
TEST=`FEATURES=test emerge-link update_engine`

Change-Id: I1e7553312db7362d44d320d457e4a59756bfe5b9
Reviewed-on: https://chromium-review.googlesource.com/254780
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
cd514b531a38da1497630fa68b6e3ef45871893d 19-Feb-2015 Allie Wood <alliewood@chromium.org> update_engine: Refactor inplace payload generator algorithm code.

Create a class, InplaceGenerator, that contains all functionality
related to the inplace delta generation method (minor version 1).

BUG=chromium:459701
TEST=`FEATURES=test emerge-link update_engine`

Change-Id: Ib742f70030d6c2fcb1cc3138e0f4aef54eca6975
Reviewed-on: https://chromium-review.googlesource.com/251621
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
3283a55293ec9e645224c48d0c3cf6f8867395e7 19-Feb-2015 Allie Wood <alliewood@chromium.org> update_engine: Add new operation codes for a to b operations

Add SOURCE_COPY and SOURCE_BSDIFF operations to protobuf. These
operations are for delta minor version 2 and their input is the source
partition instead of the target partition.

BUG=chromium:459363
TEST=`FEATURES=test emerge-link update_engine`

Change-Id: I2c5c83ab5063ff6ce7fc2ec8acfe809d9526b6da
Reviewed-on: https://chromium-review.googlesource.com/250951
Reviewed-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
78750a450ce84b2d335134402d041aa8773ab8ef 12-Feb-2015 Allie Wood <alliewood@chromium.org> update_engine: Detect supported minor version from conf file.

If the delta minor version is not given as a command line argument, the
payload generator will look for the minor version in
etc/update_engine.conf in the old image.

BUG=chromium:455493
TEST=`FEATURES=test emerge-link update_engine`
CQ-DEPEND=CL:249956

Change-Id: I0f8e3f3c05432169770f2e8e283e8f1fb363def3
Reviewed-on: https://chromium-review.googlesource.com/249955
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
f68bbbc952aa9a71898e4939b5f36187fa564a50 09-Feb-2015 Alex Vakulenko <avakulenko@chromium.org> update_engine: replace std::vector<char> with chromeos::Blob

To make update engine consistent with the rest of platform2 code
replaced std::vector<char> as the container of binary data with
chromeos::Blob.

BUG=None
TEST=`FEATURES=test emerge-link update_engine`

Change-Id: I6385fd2257d15aa24bfa74ac35512c2a06c33012
Reviewed-on: https://chromium-review.googlesource.com/247793
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
b8ccad0a8ac2632da2788b47d85e62e334bbe652 09-Feb-2015 Allie Wood <alliewood@chromium.org> update_engine: Add command line option for payload minor version.

Adds minor_version option to delta_generator for delta updates. This
defaults to kSupportedMinorPayloadVersion when the value is not given.
Stores the value in the DeltaArchiveManifest protobuf. Checks
that the specified version is supported, and fails if it is not.

BUG=chromium:455489
TEST=delta_diff_generator unit tests and manual testing.

Change-Id: I95fb5576d8f33fe8be738488c5ea85d8a46e917b
Reviewed-on: https://chromium-review.googlesource.com/247802
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
063181c4fb0ab86c6aaba307c0cc00b901c4522b 05-Feb-2015 Gilad Arnold <garnold@chromium.org> update_engine: Fix delta generator mishandling of unchanged blocks.

In a previous CL:181515 I made the delta generator filter out blocks
that were being MOVEed onto themselves, subsequently discarding empty
MOVE operations entirely. This resulted in unchanged blocks being
considered "unwritten". However, I did not realize that the delta
generator was scanning and packing all these blocks into one large
REPLACE_BZ at the end, without even checking whether any of them has
changed in the new image relative to the old image. Recently, we
realized that this causes deltas between largely similar (or worse,
identical) images to bloat.

It should be noted that this inefficiency existed before the
aforementioned feature was introduced, although it only applied to truly
unused filesystem blocks. However, such blocks being mostly zero, they
compressed well and likely did not affect the size of the delta much.

This CL fixes the problem by addressing the general problem: we do not
overwrite the content of previously unwritten blocks if they are
identical to the ones in the old image. Furthermore, if none of the
unwritten blocks has changed, we omit the said REPLACE_BZ operation
entirely. In the case of a delta between identical image, the result is
a payload with no operations at all (a good thing). In the particular
case of the failing panther_moblab build (see bug) this successfully
reduces the delta size from 625 MB to 331 bytes (!).

CQ-DEPEND=CL:246673
BUG=chromium:453659
TEST=Unit tests
TEST=Generated delta payload for previously failing panther_moblab build
TEST=Generated and tested two deltas for link (trivial N-to-N, real N-1)

Change-Id: I1c4e33d7cca5d59ba6725322970a329c1a3f7688
Reviewed-on: https://chromium-review.googlesource.com/246670
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
a461fc37778f6c1c5aad7901417a01933ae2697a 19-Nov-2014 Alex Deymo <deymo@chromium.org> update_engine: Used brace initialization instead of ad-hoc function.

SetWithValue<T> returns a set<T> with the passed value. This can be
achieved using the brace initialization introduced in C++11 with
just set<T>{value}.

BUG=None
TEST=Unittest still pass.

Change-Id: I24362098df3ea153600062fbb259833a37efa666
Reviewed-on: https://chromium-review.googlesource.com/230622
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
020600db42428e7e96bb0d2e05da225e68650955 06-Nov-2014 Alex Deymo <deymo@chromium.org> update_engine: Use C++11 range-based for loop when possible.

This cleanup just simplifies the notation of several for loops using
the new range-based syntax.

BUG=None
TEST=`FEATURES=test USE="lang clang" emerge-link update_engine`

Change-Id: I2120a1b1c3fd00e9a02a62abfa8427932080d0f9
Reviewed-on: https://chromium-review.googlesource.com/227897
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
f329b933db41d26644a97afef928eb1b319d6d99 30-Oct-2014 Alex Deymo <deymo@chromium.org> update_engine: Fix all the "using" declaration usage.

This patch removes unused "using" declarations, that is, declarations
included in a .cc file at a global scope such that "using foo::bar"
that later don't use the identifier "bar" at all.

This also unifies the usage of these identifiers in the .cc files
in favor of using the short name defined by the using declaration.
For example, in several cases the .h refer to a type like
"std::string" because using declarations are forbidden in header
files while the .cc includes "using std::string;" with the purpose
of just writting "string" in the .cc file. Very rarely, the full
identifier is used when a local name ocludes it, for example,
StringVectorToGStrv() and StringVectorToString() in utils.cc named
its argument just "vector" need to refer to std::vector with the
full name. This patch renames those arguments instead.

Finally, it also sorts a few lists of using declarations that weren't
in order.

BUG=None
TEST=FEATURES=test emerge-link update_engine

Change-Id: I30f6b9510ecb7e03640f1951c48d5bb106309840
Reviewed-on: https://chromium-review.googlesource.com/226423
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
02f7c1dee242f490143791dbb73fa23fa3007cfa 19-Oct-2014 Ben Chan <benchan@chromium.org> update_engine: Replace scoped_ptr with std::unique_ptr.

BUG=None
TEST=`FEATURES=test emerge-$BOARD update_engine`
TEST=`USE='clang asan' FEATURES=test emerge-$BOARD update_engine`

Change-Id: I55a2f7f53675faaac20ba25f72ed52cf938d7744
Reviewed-on: https://chromium-review.googlesource.com/224189
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Ben Chan <benchan@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
4a444ae459660ad4478011368c02c5f8abcea178 20-Sep-2014 Alex Deymo <deymo@chromium.org> update_engine: Limit bsdiff to 200 MiB files.

bsdiff and bspatch require a lot of RAM and time to run over very big
files. In released ChromeOS images, the biggest file is chrome with
a size of about 130 MiB in the worst case (link). Nevertheless,
ChromiumOS builds for ASan have much bigger files due to the memory
instrumentation.

This patch limit the usage of bsdiff/bspatch to files under 200 MiB,
something that should not affect any released board.

BUG=chromium:329248
TEST=cbuildbot amd64-generic-asan enabling the full vm_test

Change-Id: I24b6c9e56525858dab3758af8eddfe3acdf12e75
Reviewed-on: https://chromium-review.googlesource.com/219201
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
a77939e368597241fe0153bedce196d7152a5de5 10-Sep-2014 Gabe Black <gabeblack@chromium.org> update_engine: Use utils::FileSize when finding the size of a file.

The utils::FileSize funciton can find the size of a block device correctly.
Use it instead of the adhoc methods used around the codebase.

BUG=chromium:415867
TEST=Ran a butterfly-paladin tryjob with --hwtest.

Change-Id: Id6fd37f04b136b4265bde9b1f56c379a5d9c30f9
Reviewed-on: https://chromium-review.googlesource.com/217418
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Gabe Black <gabeblack@chromium.org>
Tested-by: Gabe Black <gabeblack@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
f9cb98c0cacccca803db48502c7bd461d04cb6b0 22-Sep-2014 Ben Chan <benchan@chromium.org> update_engine: Clean up angle brackets in template types.

C++11 no longer has the angle bracket pitfall in template types.

BUG=None
TEST=`FEATURES=test emerge-$BOARD update_engine`

Change-Id: I0168b9f208ad8e62ae614b8a3b8bcf31c58fa9f2
Reviewed-on: https://chromium-review.googlesource.com/219203
Commit-Queue: Ben Chan <benchan@chromium.org>
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
06c76a49bfd29c8abdb8abd5b646a6583783191a 05-Sep-2014 Ben Chan <benchan@chromium.org> Update code to include base/files/file_util.h

file_util.h was moved from base to base/files
(https://codereview.chromium.org/468253002). This CL updates platform2
code to include base/files/file_util.h instead of base/file_util.h.

BUG=chromium:411001
TEST=Trybot run on paladin, release, and chromiumos-sdk builders.

Change-Id: I488925b54615e131e508a460dc1a27f88168f936
Reviewed-on: https://chromium-review.googlesource.com/216851
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Ben Chan <benchan@chromium.org>
Commit-Queue: Ben Chan <benchan@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
88b591f24cb3f94f982d7024c2e8ed25c2cc26a2 29-Aug-2014 Alex Vakulenko <avakulenko@chromium.org> update_engine: Replace NULL with nullptr

Replaced the usage of NULL with nullptr. This also makes it possible to
use standard gtest macros to compare pointers in Update Manager's unit tests.
So, there is no need in custom UMTEST_... macros which are replaced with the
gtest macros (see change in update_engine/update_manager/umtest_utils.h):

UMTEST_ASSERT_NULL(p) => ASSERT_EQ(nullptr, p)
UMTEST_ASSERT_NOT_NULL(p) => ASSERT_NE(nullptr, p)
UMTEST_EXPECT_NULL(p) => EXPECT_EQ(nullptr, p)
UMTEST_EXPECT_NOT_NULL(p) => EXPECT_NE(nullptr, p)

BUG=None
TEST=FEATURES=test emerge-link update_engine
USE="clang asan" FEATURES=test emerge-link update_engine

Change-Id: I77a42a1e9ce992bb2f9f263db5cf75fe6110a4ec
Reviewed-on: https://chromium-review.googlesource.com/215136
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
157fe307f162d13f53e7f98f6e2da7c60c0ff001 12-Aug-2014 Alex Vakulenko <avakulenko@chromium.org> platform2: sweep the lint errors identified by the updated linter

cpplint.py has been updated and identified new issues in existing
code. Stuff like overridden functions that specify 'override' should
not be marked as 'virtual', and constructors with no parameters
should not be marked as 'explicit'.

BUG=None
TEST=cpplint.py `find ./platform2 -name *.cc -or -name *.h`

Change-Id: Ibb9de43286d874d076ffd5ebb1b13c36ec794f01
Reviewed-on: https://chromium-review.googlesource.com/211950
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
072359ca138504065e1e0c1189eb38c09576d324 18-Jul-2014 Alex Vakulenko <avakulenko@chromium.org> update_engine: fixed remaining linter and some spelling errors

Fixed remaining errors from cpplint as well as some spelling errors
mostly in comments.

BUG=None
TEST=FEATURES=test emerge-link update_engine

Change-Id: I484988ab846ac5a3c68c016ddccfb247f225ec27
Reviewed-on: https://chromium-review.googlesource.com/208897
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
923d8faa6b341ce5e16a760749e98cc6703ca2fd 16-Jul-2014 Alex Deymo <deymo@chromium.org> update_engine: Split payload signing and verification.

Payloads are only signed on server-side code (delta_generator) and
verified on both sides and unittest. This removes the dependency of
payload_generator/ code from delta_performer.cc by spliting the
payload signing and verification in two files.

Currently, both files are still included on all the built files.

This patch also includes some minor linter fixes.

BUG=chromium:394184
TEST=FEATURES="test" emerge-link update_engine; sudo emerge update_engine

Change-Id: Ia4268257f4260902bc37612f429f44ba7e8f65fd
Reviewed-on: https://chromium-review.googlesource.com/208540
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
d2779df63aaad8b65fc5d4badee7dbc9bed7f2b6 16-Jun-2014 Alex Vakulenko <avakulenko@chromium.org> update_engine: fixed warnings from cpplint

Fixed all the cpplint warnings in update engine.

BUG=None
TEST=Unit tests still pass.

Change-Id: I285ae858eec8abe0b26ff203b99a42a200ceb71c
Reviewed-on: https://chromium-review.googlesource.com/204027
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc
161c4a132743f15fc4757112b673085c2a7a7f29 17-May-2014 Alex Deymo <deymo@chromium.org> Move payload generator files to payload_generator/ directory.

This creates a new subdirectory payload_generator/ with all the
payload generator specific files.

The SConstruct file is updated to continue building all the files
together, including those in the subdirectories, since some parts
of the update_engine are using parts of the payload generation code.

To reduce this code coupling, a new payload_constants.h file is
introduced, with few constants used on the payload definition by both
the payload generation and the payload performer.

Finally, includes are updated and in some cases removed when they
weren't used. Order of includes is also fixed to comply with the
style guide.

BUG=chromium:374377
TEST=Build and unittests still pass. delta_generator still present on base directory.

Change-Id: I454bbc7a66c70ebb19fd596c352c7be40a081f3d
Reviewed-on: https://chromium-review.googlesource.com/200325
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/payload_generator/delta_diff_generator.cc