History log of /system/update_engine/payload_consumer/delta_performer.cc
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
5ed4f4c5b198066feefc4fd49107affa89dc559a 29-Jun-2017 Sen Jiang <senj@google.com> Add maximum timestamp to the payload.

Added a new field max_timestamp in the protobuf, from now on
update_engine will reject any payload without this field.
If the OS build timestamp is newer than the max_timestamp, the payload
will also be rejected to prevent downgrade.

Bug: 36232423
Test: update_engine_unittests

Merged-In: Ib20f5f35aaf41165013bada02bc8720917358237
Change-Id: Ib20f5f35aaf41165013bada02bc8720917358237
(cherry picked from commit 5011df680621eb477cad8b34f03fba5b542cc2f9)
(cherry picked from commit ccb01b2a3beb94094388de806bc15b210ebe8b11)
/system/update_engine/payload_consumer/delta_performer.cc
fc110efa5219b83802f532cb2caabcef2cb69a5e 23-Jun-2017 Amin Hassani <ahassani@google.com> Fix the long time ioctl problem.

On some boards ioctl() takes a long time (several seconds) to perform
discarding blocks for a few hundred MB (which might be expected). This
causes the UE (which is single threaded) to block for ioctl for several
seconds and prevents the message loop from responding to dbus requests.
'update_engine_client --status' relies on dbus signals to get the status
of the update engine. This can cause 'cros flash' or provisionning of
devices to fail due error returned from ssh invokation of
update_engine_client. This CL fixes the issue by doing
BLKDISCARD before BLKSECDISCARD.

Discarding the end of partition were originally added for b/28744609.

Also, BLKSECDISCARD never works on our android devices anyway; and it
causes a selinux denial to sys_rawio.

670 W update_engine: [0708/111132:WARNING:delta_performer.cc(154)] Error discarding
the last 152 KiB using ioctl(4733)

670 W update_engine: type=1400 audit(0.0:159): avc: denied { sys_rawio } for
capability=17 scontext=u:r:update_engine:s0 tcontext=u:r:update_engine:s0
tclass=capability permissive=0

BUG: 30035671
Test: unit tests pass
(cherry picked from https://chromium-review.googlesource.com/547015)
(cherry picked from commit 4155520522b40a5e5f9b098eb903213d65d1cee6)

Change-Id: I7044c5b4a3cba22c2db8bdbd56a29ca60d8a3824
/system/update_engine/payload_consumer/delta_performer.cc
cdd5206c6507fa5c70344c269053d30777f7d674 19-May-2017 Sen Jiang <senj@google.com> Support IsDeltaPayload per <package>.

Mixed full and delta payloads are now supported for multi-payload.

Bug: 36252799
Test: update_engine_unittests
Change-Id: Ic519f8399579c714a65ba9f9f948a389fcc249ed
(cherry picked from commit 4928112f8635f7a5b583171640aee2bda690ba34)
/system/update_engine/payload_consumer/delta_performer.cc
5ae865b431cc041feed82f1db28a38e99dbd631f 18-Apr-2017 Sen Jiang <senj@google.com> Resume multiple payloads.

DownloadAction will first go through all the already applied payload
and only downloading the manifest and filling in partitions info in
install plan without applying or downloading any operations.
And then resume the partially applied payload using states in prefs.

Moved constuction of MultiRangeHttpFetcher from UpdateAttempter to
DownloadAction, because we now need to setup the range for every
payload, also reduced code duplication.

Also fixed download progress for multi payload.

Bug: 36252799
Test: stop an update during second payload and resume the update
Change-Id: I9ee54a87d15d88c7a14a13575965b19c1773340b
(cherry picked from commit 7162b666318cdd6c75b73fa6a0b06b23559de3e5)
/system/update_engine/payload_consumer/delta_performer.cc
d2ff2a0ee883148c6c4c935791aaf150138a0366 06-Apr-2017 Sen Jiang <senj@google.com> Don't clear previous partitions in DeltaPerformer.

When we used to have a FilesystemVerifierAction before DownloadAction,
partitions in InstallPlan was filled in by FilesystemVerifierAction
with source hashes, but now that action is gone, DeltaPerformer is the
first action that touches partitions, so clearing partitions are not
needed. In multi payload case, partitions in previous payloads are
stored there so it should not be removed.

We could also move the partitions into Payload but that will make our
code more complex (keeping track of both payload index and partition
index in FilesystemVerifierAction and PostinstallRunnerAction) thus
more likely to have bugs.

This simple patch solves the same problem without polluting the code.

Bug: 36252799
Test: FilesystemVerifierAction verifies all partitions in all payloads.
Change-Id: I2aed02389bf047a6dedf59b306434ccea4eebca8
(cherry picked from commit cf6bd59a3cd149bd86d763e96ede81ce102d64d9)
/system/update_engine/payload_consumer/delta_performer.cc
0affc2c099fac47b999ba39f8f1bacb418e65a97 11-Feb-2017 Sen Jiang <senj@google.com> Parse multiple packages from Omaha response.

The multi-payload info are stored in OmahaResponse and InstallPlan, but
we still can only apply the first payload for now.

Bug: 36252799
Test: mma -j
Test: update_engine_unittests

Change-Id: I5ca63944ae9082670d0e67888409374f140d4245
(cherry picked from commit 2aba8a87d4fac245a2e2d238b3159f8eabce630f)
/system/update_engine/payload_consumer/delta_performer.cc
2703ef4466066d64d8021904e233b120f38c0272 16-Mar-2017 Sen Jiang <senj@google.com> Store raw payload hash blob in install plan.

We were using a custom sha256 pair in Omaha response, now that Omaha
has a standard hash_sha256 field in package, we should use that instead.

The difference is that hash_sha256 is encoded in hex instead of base64,
but the android payload property is still using base64, to be backward
compatible, we have to keep accepting base64 there, to avoid decoding
and then re-encoding to another encoding, we store the decoded raw hash.

Also removed the hash() related functions in HashCalculator, since it's
rarely used and the caller should encode it in whatever encoding they
want.
Also make use of RawHashOfBytes to simply code in a few places.

Bug: 36252799
Test: update_engine_unittests
Change-Id: Iaa02611b4c9cda3ead5de51e777e8caba6d99d93
(cherry picked from commit f14d51b6823522f6b2eb834f9e14d72c8363a3ad)
/system/update_engine/payload_consumer/delta_performer.cc
eecb0a5b696c830e3db5019654141dc58fbef825 20-May-2017 Alex Deymo <deymo@google.com> Remove IMGDIFF operation application support.

Remove support for applying IMGDIFF operations as this will be replaced
with PUFFIN.

Bug: None
Test: Unittests still build.
Change-Id: Idd8151a4fc772c70359afdd2da8d4a2f05bbd293
/system/update_engine/payload_consumer/delta_performer.cc
6765a68c5f3963ce986f0bb11951b3af40e3e098 19-May-2017 Alex Deymo <deymo@google.com> Move to use bsdiff/ prefix in includes.

The bspatch.h headers are now under a "bsdiff/" directory.

Change-Id: Ica2935e4163f617d60df1bc0846978b0e1a59f46
/system/update_engine/payload_consumer/delta_performer.cc
4b5da322bdb715c952f2fd5faf55c4bbca6bb057 26-Mar-2017 Tao Bao <tbao@google.com> Track the API change to ApplyImagePatch().

It now accepts a std::function as the sink callback.

Test: update_engine_unittests
Change-Id: I1d92197581069b36e14a5e25ee7c66079a987a6d
/system/update_engine/payload_consumer/delta_performer.cc
7a65909568c74307caa9c0eecc0721175509c4fc 11-May-2016 Sen Jiang <senj@google.com> Use bspatch as a library.

Test: mma
Bug: 26982501
Change-Id: Iafb6a2a4e6cb747004fbf1980e7f0b29108eb54f
/system/update_engine/payload_consumer/delta_performer.cc
5b91c6b141970c2b0095775a61e3f941417aa1ff 05-Aug-2016 Alex Deymo <deymo@google.com> New setting to mark postinstall as optional.

This setting allows to mark a postinstall script as optional. This
allows the postinstall program to fail when it is not strictly required
to run for the update to succeed.

Bug: 27178350
TEST=Added unittest. Sideloaded an update with an optional postinstall.

Change-Id: I41956d3308f3458b6bf94b835c9b5e470c84ca41
/system/update_engine/payload_consumer/delta_performer.cc
5c6bb1d8f89b87e782e26fe74789b6f510704394 27-Jul-2016 Chih-Hung Hsieh <chh@google.com> Fix clang-tidy performance warnings in update_engine.

* Use const reference type for parameters, local variables,
and for-loop index variables to avoid unnecessary copy.
* Convert some for-loops to for-rang loops.

Bug: 30407689
Bug: 30413223
Bug: 30413862
Change-Id: I78996b3f799639fc57ced45e110807625be7dcce
Test: build with WITH_TIDY=1
/system/update_engine/payload_consumer/delta_performer.cc
9d66d761f5fdbd7846810954c5245c5391b743c4 15-Jul-2016 Tamas Berghammer <tberghammer@google.com> Add type casts to silence warnings intorduced by protobuf v3.0.0

Bug: b/28974522
Change-Id: I1a5a26ea503747b786ed38ae5d6a586340a5fe2e
/system/update_engine/payload_consumer/delta_performer.cc
be2c47bad96b46985bdcedceabbe6baa1f8a23f5 15-Jun-2016 Sen Jiang <senj@google.com> Return correct error code when source operation hash mismatch.

Now it returns kDownloadStateInitializationError.

Test: delta update on a modified system.
Bug: 28769126

Change-Id: Iedfe938d6a0db8ae5ac137d073e61f550e3d71a9
/system/update_engine/payload_consumer/delta_performer.cc
72a68d829f6b0bc97368c64d35a860465716018c 16-Jun-2016 Alex Deymo <deymo@google.com> Show the name of the BLK* ioctl failure instead of the number.

The ioctl number is normally tied to the platform and doesn't allow a
quick debug of what was supported. This patch prints the name instead
of the error number.

Bug: 28744609
TEST=Deployed an update on Android and saw the ioctl names instead of numbers.

Change-Id: Ie3a69841f3bf9cd719ead0cdc80e82d2c4725c68
/system/update_engine/payload_consumer/delta_performer.cc
6714084915ea7b66accf933344b0d4cd042dff3d 15-Jun-2016 Alex Deymo <deymo@google.com> Improve logs when the update fails due to source hash mistmatch.

The source hash mismatch at the operation level is the most likelly
error when applying an incremental update on top of a modified image.
This error can also be caught when re-reading the whole target partition
at the end of the update, but if a discrepancy on any operation is
found when applying the operations, it will fail earlier.

This patch improves the log message when this situation occurs. This is
not a common error case so a more verbose error message is acceptable
here. Previously, the error message was not giving any clue about why
was it failing.

Bug: 29163155
TEST=Modified a partition and applied a payload. Error message is nicer.

Change-Id: Ia031ed2f42f9b2ecd18bed75c949c0beed6c229e
/system/update_engine/payload_consumer/delta_performer.cc
b86787cdb2ca619c8e2ef0a2e38af8353f4cf4ac 13-May-2016 Alex Deymo <deymo@google.com> Discard the tail of the target partitions when updating.

The partition is normally way bigger than the data it holds and the
remainder of the partition is often ignored by the system. This patch
discards the tail of the partition, past the end of the updated data.

This would ensure that the unused blocks in these partitions can be
reallocated by the SSD controller to other blocks, and that data from
previous updates does not interfere with the current version.

Bug: 28744609
TEST=strace -e trace=file,ioctl shows the device discarding the end of the updated partitions.

Change-Id: Ie6135ee1aef2fa594c40d84af4d1d46a8f53cc3a
/system/update_engine/payload_consumer/delta_performer.cc
3d0090674394076d6f998fc27f60b041e5718904 14-May-2016 Alex Deymo <deymo@google.com> Improve logging when source hash mismatches.

When applying an operation, the source hash may mismatch the one the
payload expects. The current log only shows the operation number across
all operations in the payload. This patch also includes the operation
number in the current partition and the partition name.

Bug: 28626303
TEST=deployed the wrong delta payload and saw the partition name and number.

Change-Id: I8d6d1639deffbc360fa7a02448d1ad28fa80aafe
/system/update_engine/payload_consumer/delta_performer.cc
5fb356cac8a87c62fa83005d99370aa2f8416064 26-Mar-2016 Alex Deymo <deymo@google.com> Mark block device as read-only before mounting.

Mounting a block device as read-only still allows the filesystem to
write to the block device, while keeping the user-facing filesystem
interface "read-only". This behavior will make dm-verity to fail after
reboot if any block is modified during postinstall.

This patch marks the block device as read-only before mounting it, and
marks them read-only or read-write before using them during the update.

Bug: 27859604
TEST=Added logging and deployed an update to brillo, devices are marked RO/RW as needed during a delta update.

Change-Id: I781293cc0b3447dac708470ba6efad4103bf2a58
/system/update_engine/payload_consumer/delta_performer.cc
fef85fd9ece49941db274559a938fe8b2c5157bf 25-Mar-2016 Sen Jiang <senj@google.com> Remove ComputeSourceHash mode in FileSystemVerification action.

This mode was used to calculate the source partition hash before download
the payload, and we will verify it against the hash in the payload.
Now that we are using per-operation source hash, this mode is no longer
needed.

Test: ./update_engine_unittests
Test: cros_workon_make update_engine --test
Bug: 26972259

Change-Id: Ie30a38cfd9f94e4efe02dfc8664e6785018261f6
/system/update_engine/payload_consumer/delta_performer.cc
0497d05764dcae38e7a7419a2f25eff043172850 23-Mar-2016 Alex Deymo <deymo@google.com> Fix ZERO and DISCARD operations.

These operations were broken (and not used in the generator) because
the code expects them to have a blob offset, but they don't. This bug
prevents them from being used in full payloads, but can still be used
in delta payloads.

Bug: 27858697
TEST=Deployed a full payload with ZERO operations to the new update_engine.

Change-Id: Ifc67bd7c3b2a81593d0347a345723cc34eaaf6b5
/system/update_engine/payload_consumer/delta_performer.cc
bc3e6b00d2ed50d5fd77f2a74348608e52935e6d 19-Jan-2016 Sen Jiang <senj@google.com> Implement IMGDIFF operation in the client.

This operation is used to apply update on gzipped data with much smaller
diff data than bsdiff.

update_engine only calls ApplyImagePatch() with all the data, the actuall
implementation of the imgdiff format is in bootable/recovery/applypatch.

Test: mma & added unittest
Bug: 26628339

Change-Id: I2b8a097aa68727b06be101cc06a3a896b835a815
/system/update_engine/payload_consumer/delta_performer.cc
390efedcb7e17587da765b6d682077cb7fa46ee1 18-Feb-2016 Alex Deymo <deymo@google.com> Parse postinstall parameters from the payload metadata.

Payload v2 includes a description of the post-install command it should
run, while in payload v1 we use the default values. This patch mounts
the partition on the new top-level directory called /postinstall that
should already be created.

Bug: 27177071
TEST=FEATURES=test emerge-link update_engine

Change-Id: Iaedf3b01e5e1ad57c68bd316b4b6e79cbab35bb6
/system/update_engine/payload_consumer/delta_performer.cc
f25eb491ff60f21659a7e2b230ee1c83957034c7 26-Feb-2016 Alex Deymo <deymo@google.com> Fix resuming canceled updates.

Resuming an interrupted update was broken in several ways. First,
DeltaPerformer::CanResumeUpdate was checking for the number of
resume-failures to be more than the limit, not less. Then, the
resume logic didn't work for payload v2 when there was a
metadata signature included in the payload. Finally, resuming an
update after reading the payload signature (in payload v2) was not
updating the checkpoint, but storing the signature causing it to
attempt to parse it again and fail.

Bug: 27047026
Bug: chromium:590410
TEST=Manual procedure:
1. Start an update: update_engine_client --update ...
2. Cancel the update: update_engine_client --cancel
3. Re-start the same update: update_engine_client --update ...
-> The update should resume from the previous point.

Change-Id: I60134de155aa073a7ba91174cceea7297e5f8d17
/system/update_engine/payload_consumer/delta_performer.cc
64d9878470aa7b388e971862181daf6260851602 06-Feb-2016 Alex Deymo <deymo@google.com> Replace is_full_update boolean with a payload_state enum.

The "is_full_update" flag in the InstallPlan is required to decide
whether we should run a FilesystemVerification step before start
downloading the payload (for delta payloads) or not (for full payloads).
This step is done before start downloading the payload and not after
downloading the metadata to avoid long delays in the connection which
would then drop and require a retry.

Since the not so recent inclusion of the source_data_hash field in the
delta operations, the source data is verified on each operation, so the
install plan field and the pre-download FilesystemVerification is not
needed anymore.

To help deprecate this process, which is not included in the non-Brillo
version, this patch changes the is_full_update field to a payload_state
enum with a third "unknown" state that will be changed to delta or full
once the payload metadata is parsed.

Bug: 25631949
TEST=unittests updated.
TEST=Pushed a delta update to edison-eng and a non-Brillo target.

Change-Id: I17d8bf58990d8465bb8487adc66601f1c1dfca6d
/system/update_engine/payload_consumer/delta_performer.cc
0103c36caa2e38e034e0d22185736b9ccfb35c58 20-Jan-2016 Alex Vakulenko <avakulenko@google.com> update_engine: Update libchrome APIs to r369476

The new libchrome has been ported from Chromium and some APIs have
changed. Make necessary changes at call sites.

Change-Id: I42e65bda7f1dbdf6f6e0ebf356d2cfea6b729193
/system/update_engine/payload_consumer/delta_performer.cc
05e0e38c30824b727f3582c90258d9bb19cc3f47 08-Dec-2015 Alex Deymo <deymo@google.com> Compile delta_generator for the host.

Bug: 24619596
TEST=mma on linux.

Change-Id: Icccae2b938b4fdd698b177beee96ffb62e9f63ae
/system/update_engine/payload_consumer/delta_performer.cc
889c65d1914ed8a151f451a2933fa94d6e577aa6 18-Nov-2015 Sen Jiang <senj@google.com> Switch supported minor version to 3.

Bug: 23182225
TEST=cros_workon_make update_engine --test

Change-Id: Ide798f89bd0e3f662d2deac96035780892bbfa08
/system/update_engine/payload_consumer/delta_performer.cc
542c19bf271011913a9f352f58e140224c936736 03-Dec-2015 Alex Deymo <deymo@google.com> Remove SystemState references from DeltaPerformer.

DeltaPerformer is used as part of the DownloadAction and had
references to the global SystemState. The common references to
BootControlInterface (to query the partitions based on the names
from the payload) and the HardwareInterface (for dev-mode bits) are
now referenced directly from the DeltaPerformer. The calls to
UpdateAttempter and PayloadState were moved to the
DownloadActionDelegate since these calls are received by classes
outside the payload_consumer.

Bug: 25773375
TEST=FEATURES=test emerge-link update_engine

Change-Id: Id2e81d5ccf835cad22f03b069c681dcce104c456
/system/update_engine/payload_consumer/delta_performer.cc
706a5abae125b8f5ba58a9b7e1e51570bbaa2089 23-Nov-2015 Alex Deymo <deymo@google.com> Remove SystemState dependency from HttpFetcher and InstallPlan.

The SystemState class is an aggregation of all the update_engine
singletons, making it easy to handle cross-dependencies between these
singletons. Nevertheless, since we split the code into a smaller
libpayload_consumer library we need to remove the global dependencies
on the SystemState class from this library and specialize those
dependencies to the actual required class.

Bug: 25773375
TEST=FEATURES=test emerge-link update_engine; mma

Change-Id: I8800157c969db6a8d168f33ac2c6aad4f34fa236
/system/update_engine/payload_consumer/delta_performer.cc
2ec4aab50fa8ed4afd9fa86a0125ed68674ebd01 14-Nov-2015 Sen Jiang <senj@google.com> Verify operation source hash if present.

Verify source for every operation that needs to read from source partition
so that we can skip verifying the whole source partition before applying
any operation.

Bug: 23182225
TEST=cros_workon_make update_engine --test

Change-Id: I13e0b450574cee5ef892839ee703d93680531f72
/system/update_engine/payload_consumer/delta_performer.cc
39910dcd1d68987ccee7c3031dc269233a8490bb 10-Nov-2015 Alex Deymo <deymo@google.com> Split payload application code into a subdirectory.

This patch splits from the main libupdate_engine code the part that
is strictly used to download and apply a payload into a new static
library, moving the code to subdirectories. The new library is divided
in two subdirectories: common/ and payload_consumer/, and should not
depend on other update_engine files outside those two subdirectories.
The main difference between those two is that the common/ tools are more
generic and not tied to the payload consumer process, but otherwise they
are both compiled together.

There are still dependencies from the new libpayload_consumer library
into the main directory files and DBus generated files. Those will be
addressed in follow up CLs.

Bug: 25197634
Test: FEATURES=test emerge-link update_engine; `mm` on Brillo.

Change-Id: Id8d0204ea573627e6e26ca9ea17b9592ca95bc23
/system/update_engine/payload_consumer/delta_performer.cc