| 90c57d7d59e350065da2dc973d359c28f613e95c |
08-Feb-2018 |
Amin Hassani <ahassani@google.com> |
update_engine: Disable puffin in minor version 4
This patch disables puffin for minor version 4, but keeps BROTLI_BSDIFF in the
same minor version. Also adds puffin for minor version 5, but it does not
enable it.
Bug: 30989362
BUG=b:72815313
BUG=chromium:810184
TEST=update_engine unittests pass
TEST=update_payload unittests pass
TEST=test_paycheck.sh on two images with minor version 4 pass
TEST=brill_update_payload generate/verify pass
Change-Id: I814fc126d67207501d419cff9c575cd13d877b48
Reviewed-on: https://chromium-review.googlesource.com/907223
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Amin Hassani <ahassani@chromium.org>
(cherry picked from commit 77d7cbcef1ca1afd36720ec7ec74039b5445101e)
ayload_generation_config.cc
|
| bd29001fa4d0a34c7239dffb490a648d65637ec2 |
03-Mar-2018 |
Tianjie Xu <xunchang@google.com> |
Enable brotli_bsdiff in update package generation am: de1685fd76 am: 8679c142bb
am: 1c14a6cf2d
Change-Id: I3cf6b81d24f146b3b73562e8f31cfbd83966e4e7
|
| 1c14a6cf2d4ec5eabdce2a674eb346cf679371ae |
03-Mar-2018 |
Tianjie Xu <xunchang@google.com> |
Enable brotli_bsdiff in update package generation am: de1685fd76
am: 8679c142bb
Change-Id: I41c20b1dbcd6691baee6d4577b6e1ea3b977ec57
|
| de1685fd76c0742df848985bf448388583e1299e |
21-Feb-2018 |
Tianjie Xu <xunchang@google.com> |
Enable brotli_bsdiff in update package generation
Brotli_bsdiff uses brotli to compress the bsdiff patches; and almost
always yields a smaller result than the old bsdiff with bz2. So there
is little point to try both methods in update_engine.
Example: walleye 4504078 -> 4585723 with bsdiff cache enabled.
time size
bz2 only: ~8min 494M
brotli only: ~12min 455M
bz2 & bro: ~16min 454M
In this example, trying bz2 merely saves ~1M at the cost of 4 minutes
generation time. So in this CL, we only use brotli_bsdiff if the operation
is allowed; and later we can optionally implement a new mode in bsdiff
to pick the compressor who generates the smallest patches.
Bug: 34220646
Test: Generate and verify a walleye package
Change-Id: Ideae1acea95ff204c2ba5b7637945307743f0a51
elta_diff_utils.cc
|
| 9c89e8499abeac3894718120d12b41301ffa3fc1 |
02-Feb-2018 |
Sen Jiang <senj@google.com> |
Split payload metadata logic from DeltaPerformer into its own class.
DeltaPerformer have code for both parsing payload and performing
operations. This change moves parsing payload header and validating
metadata signature to a new class PayloadMetadata so that
DeltaPerformer can focus on performing.
We will also have new code in another class that will use the new
PayloadMetadata class to parse payload metadata.
Bug: 65283633
Test: update_engine_unittests
Change-Id: Ie20b84713a0c66867a1de9d3d0cc29d0189b3c97
ayload_signer.cc
|
| dbc44fc4df6564152e131cc32922e2c211d7534a |
06-Feb-2018 |
Tianjie Xu <xunchang@google.com> |
Temporarily disable puffin to unblock ota generation am: 7d2800522c
am: 9026b98f5d
Change-Id: I8118428fde8f74e8cf96d69f30d1eae1654972ed
|
| eb36c8b31bd8ec12424118d13d2b3a80020d6a1c |
06-Feb-2018 |
Tianjie Xu <xunchang@google.com> |
Only find deflates for zip files if puffdiff is enabled am: 395906773e
am: 2d0cda7150
Change-Id: I5d39cec784fb7d012bdb744b3884bfa341dfbd92
|
| 7d2800522c9cbc134c13dc920119d5eb1aa2acb9 |
06-Feb-2018 |
Tianjie Xu <xunchang@google.com> |
Temporarily disable puffin to unblock ota generation
This unblocks the incremental OTA generation for now. And we will revert
this ad hoc fix when the bug is fixed in puffin.
Bug: 72815313
Test: generate an incremental OTA for walleye 4504078-4585723
Change-Id: Idab8a0496a59358275a4e6c333cd8eb8d4d6219f
ayload_generation_config.cc
|
| 395906773e5150569891840c42e4ce11546afa03 |
06-Feb-2018 |
Tianjie Xu <xunchang@google.com> |
Only find deflates for zip files if puffdiff is enabled
Because the deflates are useless if we don't do puffdiff.
Bug: 72815313
Test: generate an incremental OTA for walleye 4504078-4585723
Change-Id: I3fdb4cee93f0b046b4dace4e25693d1eda70310d
eflate_utils.cc
|
| 00cdc736380021c340b0ba55f4b2dd0304b61a34 |
30-Jan-2018 |
Tianjie Xu <xunchang@google.com> |
Enable puffin in update_engine am: 1a7bb2c4a2
am: 3237b0cbab
Change-Id: Id68ce0fd98d9924bc3c47da70c712e271332e4e7
|
| 1a7bb2c4a2b21700893b4382d9e8ea017e61c75b |
23-Jan-2018 |
Tianjie Xu <xunchang@google.com> |
Enable puffin in update_engine
Call the approximate functions to set up the deflates of a zip archive
file. This effectively enables puffdiff when generating an a/b
update package.
Test: Unittests pass. Generate and verify a package on sailfish
Change-Id: Id025c42529b8b9547d4c7d7ac57898450bc6a1c1
eflate_utils.cc
|
| 682cad5a12a41b9cf4e4c5af964403c3e0cb284a |
12-Jan-2018 |
Sen Jiang <senj@google.com> |
resolve merge conflicts of 0b77cb9fa6941163c4a4099471bef6ef564c0ffe to oc-mr1-dev-plus-aosp
Test: I solemnly swear I tested this conflict resolution.
Change-Id: If51b3d48a3553b81ad7aa79764e649debc844cac
|
| 1841408827c93f8ba7bb44ba95a8fb3438762b9d |
11-Jan-2018 |
Sen Jiang <senj@google.com> |
Merge remote-tracking branch 'goog/upstream-master'.
The following commits were reverted:
840703a Fix update over cellular network on guest account
eaad5d0 Do not merge to AOSP: Fixes the link to brillo-clang-format in CrOS
740efad Reboot even if a system update is not available.
Fixed a few sign compare warnings.
Had to ifdef out 2 SquashfsFilesystemTest because it depends on unsquashfs -m.
Test: update_engine_unittests
Change-Id: I6f4ca5003e78c76064ec60d0797505d8c18d00bf
Merged-In: I6f4ca5003e78c76064ec60d0797505d8c18d00bf
|
| c77e3a2d157aba4a016da5879809312c74c9f6bf |
15-Dec-2017 |
Amin Hassani <ahassani@chromium.org> |
Revert "update_engine: Add XZ compression for CrOS"
This reverts commit 717c324c334cc3bac954101f7a127fe8ef56e4e6.
Reason for revert: <canaries failed>
Original change's description:
> update_engine: Add XZ compression for CrOS
>
> This patch implements XZ compression for Chrome OS. It is only activated
> for only delta payload at this point. The XZ compression is typically
> slower than Bzip2 to compress, but faster to decompress. However, it
> uses slightly higher memory in decompression. It compresses about 5%-10%
> better than the current Bzip2.
>
> BUG=chromium:758792
> TEST=unittests pass; brillo_update_paylod {generate|verify} passes;
>
> Change-Id: Icdaf1d37d344c30a796845eb64daa00120f3598a
> Reviewed-on: https://chromium-review.googlesource.com/811895
> Commit-Ready: Amin Hassani <ahassani@chromium.org>
> Tested-by: Amin Hassani <ahassani@chromium.org>
> Reviewed-by: Alex Deymo <deymo@chromium.org>
> Reviewed-by: Ben Chan <benchan@chromium.org>
TBR=benchan@chromium.org,deymo@chromium.org,deymo@google.com,senj@chromium.org,ahassani@chromium.org
Change-Id: I43e8c5ebec84d395b87aa782fb4d478a5fe99279
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:795313
Reviewed-on: https://chromium-review.googlesource.com/830053
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
z_chromeos.cc
ip_unittest.cc
|
| 717c324c334cc3bac954101f7a127fe8ef56e4e6 |
12-Oct-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Add XZ compression for CrOS
This patch implements XZ compression for Chrome OS. It is only activated
for only delta payload at this point. The XZ compression is typically
slower than Bzip2 to compress, but faster to decompress. However, it
uses slightly higher memory in decompression. It compresses about 5%-10%
better than the current Bzip2.
BUG=chromium:758792
TEST=unittests pass; brillo_update_paylod {generate|verify} passes;
Change-Id: Icdaf1d37d344c30a796845eb64daa00120f3598a
Reviewed-on: https://chromium-review.googlesource.com/811895
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
z_chromeos.cc
ip_unittest.cc
|
| d8b67f49d7f8ae291cd4fa901b2d3767137704e7 |
06-Dec-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Remove the duplicate BlocksInExtents
This patch removes the duplicate BlocksInExtents from extent_utils.h and fixes
the remainder of the code to reflect this change.
BUG=none
TEST=unittests pass;
Change-Id: I76f5106f75072b20cd8f41f081b2f2b07aeac9a8
Reviewed-on: https://chromium-review.googlesource.com/812009
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Alex Deymo <deymo@google.com>
Reviewed-by: Sen Jiang <senj@chromium.org>
b_generator.cc
eflate_utils.cc
elta_diff_utils.cc
elta_diff_utils_unittest.cc
xt2_filesystem.cc
xt2_filesystem_unittest.cc
xtent_ranges.cc
xtent_utils.cc
xtent_utils.h
xtent_utils_unittest.cc
ull_update_generator_unittest.cc
raph_utils.cc
nplace_generator.cc
|
| 06c6d088b89656a120d40c4b578200c87af5c4c8 |
27-Nov-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Add an explanatory comment
BUG=none
TEST=none
Change-Id: Ie13a696506dbfa28ed9b278ee6c2d99cb1156db0
Reviewed-on: https://chromium-review.googlesource.com/790942
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Amin Hassani <ahassani@chromium.org>
elta_diff_utils.cc
|
| efa62d959cf8f225477ab85d8ff42d5150007cf8 |
09-Nov-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Adds BROTLI_BSDIFF operation
Brotli compression creates on average 10%-20% smaller output than bzip2
in addition to having faster decompressor. With recent changes in bsdiff
to compress the its patch with brotli, we can use it in the
update_engine as a new operation BROTLI_BSDIFF. This operation will be
turned on in minor version 4. However, this CL only adds support for it
in the client. It will not generate BROTLI_BSDIFF operations yet.
BUG=chromium:783437
TEST=unittests pass for both update_engine and update_payload;
'brillo_update_payload {generate|verify}' passes;
'scripts/paycheck.py payload.delta' passes;
Change-Id: Ie791ba5431561c95de6fbc031a8196dbfd912288
Reviewed-on: https://chromium-review.googlesource.com/764791
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
b_generator.h
elta_diff_utils.cc
ayload_generation_config.cc
|
| 3cd4df127c29eb90c0e203373edfbc9534c79169 |
25-Aug-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Generate PUFFDIFF operation
This patch adds functionality for generating PUFFDIFF operation, but
since the minor verion has not been increase, it actually does not
generate the operation itself. PUFFDIFF is used for patching deflate
based files.
BUG=chromium:767120
TEST=unittest passes; brillo_update_payload {generate|verify} passes;
CQ-DEPEND=CL:726945
Change-Id: I4ffdac8dce3740ef9fc2156cb0dad83a8364c8b5
Reviewed-on: https://chromium-review.googlesource.com/658298
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
eflate_utils.cc
eflate_utils.h
eflate_utils_unittest.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
ilesystem_interface.h
nplace_generator.cc
quashfs_filesystem.cc
quashfs_filesystem.h
quashfs_filesystem_unittest.cc
|
| f5a06d8e9f709824d768d7a15f1384668e7a4858 |
20-Oct-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Stop adding unneeded parameters for some operations.
Currently we are adding src_extents, src_length and dst_length for ZERO,
DISCARD and but they don't need it.
Also we are secretly adding dst_length for all REPLACE operations
which they don't need it either.
BUG=chromium:776576
TEST=unittests passed; brillo_update_payload {generate|verify} passed;
Change-Id: I9bf04071a40ef4b9a9d437830f9bdcfb507f18a2
Reviewed-on: https://chromium-review.googlesource.com/729061
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
b_generator.cc
b_generator_unittest.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
|
| 8e768e9359d01008b286db1f6c54a6c3972a56b9 |
29-Jun-2017 |
Sen Jiang <senj@google.com> |
Add maximum timestamp to the payload.
Added a new field max_timestamp in the protobuf, from now on
update_engine will reject any payload without this field.
If the OS build timestamp is newer than the max_timestamp, the payload
will also be rejected to prevent downgrade.
Bug: 36232423
Test: update_engine_unittests
Merged-In: Ib20f5f35aaf41165013bada02bc8720917358237
Change-Id: Ib20f5f35aaf41165013bada02bc8720917358237
(cherry picked from commit 5011df680621eb477cad8b34f03fba5b542cc2f9)
enerate_delta_main.cc
ayload_file.cc
ayload_generation_config.h
|
| e9f9661e99917c9b33e10fb6c4f10979b13f572b |
16-Jun-2017 |
Tianjie Xu <xunchang@google.com> |
Add multi thread support for A/B generator
Use threadpool to calculate file deltas in parallel. This decreases the
time consumption from 40 mins to 13 mins for the incremental payload generation
of two taimen system images.
BUG=b:62470452
TEST=Unit tests pass && generate an identical payload for incremental
taimen as before
Change-Id: Ie295504b758a1baa6b661c76bcbb6aa7881ded5d
(cherry picked from commit c4ad1ebc33abc088aca2909ba5cbaf7ae5e5659f)
Reviewed-on: https://chromium-review.googlesource.com/731724
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
elta_diff_utils.cc
elta_diff_utils.h
ull_update_generator.cc
|
| 924183b4ec3a66a36e681b2737472cf7bc6b6d68 |
27-Sep-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Add Squashfs (phase 2)
This patch replaces the Android container related Squashfs images with
their files in the update engine. The location of each new file is
defined as:
'Squashfs image file path' + / + 'the file path inside the Squashfs'
This works even if the files inside the Squashfs are not 4k aligned. But
we need 4k alignment before we add Puffin support. In some experiments
this could even reduce the delta payload size by like 1MB.
BUG=chromium:767120
TEST=cros_workon_make --board=amd64-generic --test update_engine; brillo_update_payload generate --payload=payload.delta --source_image=veyron_minnie-R61-9765.13.0.bin --target_image=veyron-minnie-R62-9804.0.0.bin
Change-Id: I3fbcc5cd0d724435955ed1bd1204dee0e41944be
Reviewed-on: https://chromium-review.googlesource.com/636106
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
eflate_utils.cc
eflate_utils.h
eflate_utils_unittest.cc
elta_diff_utils.cc
|
| ab5a0af8e6ebac0211da5b08010e4855ce463108 |
12-Oct-2017 |
Ben Chan <benchan@chromium.org> |
update_engine: migrate from base::MakeUnique to std::make_unique
base::MakeUnique is being deprecated as we can now use std::make_unique
when compiling code in the C++14 mode.
BUG=chromium:769107
CQ-DEPEND=CL:669672
TEST=Run unit tests.
Change-Id: I82f76647239b1eb3b98b19f6479788ffd86ce756
Reviewed-on: https://chromium-review.googlesource.com/716826
Commit-Ready: Ben Chan <benchan@chromium.org>
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
ip_unittest.cc
|
| d7da8f4118c10d00d02b6274edd2f88971a6cb22 |
23-Aug-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Add Squashfs (phase 1)
Currently the Android container in CrOS is a Squashfs image (around
500MB) and the update engine treats it as one single file. This patch
uses -map flag in 'unsquashfs' to get the location of files inside the
Squashfs image. and creates a list of files that can be used in payload
generator to create a delta. This patch adds support for both Squashfs
file system and its appropriate unittests. However, we do not activate
it in this patch, further patches will activate and use it in the
payload generator.
BUG=chromium:767120
TEST=cros_workon_make --board=amd64-generic --test update_engine;
CQ-DEPEND=CL:650518,CL:686034
Change-Id: I02179727cd736661704d0ca2d9f881754b332b36
Reviewed-on: https://chromium-review.googlesource.com/636105
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
quashfs_filesystem.cc
quashfs_filesystem.h
quashfs_filesystem_unittest.cc
|
| cd7edbe3eeff0b4379649eea8308c325882407ff |
19-Sep-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Replace vector<Extent> with RepeatedPtrField<Extent>
This patch removes references to vector<Extent> and replaces them with
RepeatedPtrField in payload_consumer. Extent itself is a protobuf item
and it makes sense to use google::protobuf::RepeatedPtrField instead of
vector because then we won't have any extra copy to vector. We can
directly use the list of extents given in the payload protobuf.
Also removed references to vector in files which did not use vector.
BUG=chromium:766397
TEST=FEATURES="test" emerge-amd64-generic update_engine
Change-Id: I1f12332ff4d6303c1e4b7470bb87bf934acdf81a
Reviewed-on: https://chromium-review.googlesource.com/672006
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
ip_unittest.cc
|
| caa9d86fb16b4a682f888cc1c3f5bdc108c1d7cc |
24-Aug-2017 |
Sen Jiang <senj@google.com> |
Don't fill in partitions in InstallPlan in ApplyPayload.
Partitions are filled in by DeltaPerformer based on information
in the payload and BootControl, existing partitions in InstallPlan
are ignored.
BUG=none
TEST=brillo_update_payload verify
Change-Id: Ia4326ba4671b5e74c2f898d3b29701acbf491a7b
(cherry picked from commit 4136062ed805d8d078998e3a3cff56754c474396)
Reviewed-on: https://chromium-review.googlesource.com/638555
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
enerate_delta_main.cc
|
| ddbf8f0521904514e76eecaf13c1ebfec84c64dc |
24-Aug-2017 |
Sen Jiang <senj@google.com> |
Fix applying REPLACE_XZ operation when verifying payload.
xz-embedded needs to initialize crc32 table.
Also changed the exit code to 1 if ApplyPayload fail.
BUG=none
TEST=brillo_update_payload verify
Change-Id: I1717b7463f7118bb94da2b780e0106995451b68f
(cherry picked from commit 8adc1ad36359b2e5b0a0d599409589d625064574)
Reviewed-on: https://chromium-review.googlesource.com/638554
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
enerate_delta_main.cc
|
| 4136062ed805d8d078998e3a3cff56754c474396 |
24-Aug-2017 |
Sen Jiang <senj@google.com> |
Don't fill in partitions in InstallPlan in ApplyPayload.
Partitions are filled in by DeltaPerformer based on information
in the payload and BootControl, existing partitions in InstallPlan
are ignored.
Test: brillo_update_payload verify
Change-Id: Ia4326ba4671b5e74c2f898d3b29701acbf491a7b
enerate_delta_main.cc
|
| 8adc1ad36359b2e5b0a0d599409589d625064574 |
24-Aug-2017 |
Sen Jiang <senj@google.com> |
Fix applying REPLACE_XZ operation when verifying payload.
xz-embedded needs to initialize crc32 table.
Also changed the exit code to 1 if ApplyPayload fail.
Test: brillo_update_payload verify
Change-Id: I1717b7463f7118bb94da2b780e0106995451b68f
enerate_delta_main.cc
|
| 546ded562a3e9dad2a3704977396091fa7befaa2 |
23-Aug-2017 |
Alex Deymo <deymo@google.com> |
delta_generator: Convert ABORT into an exit(1).
When verifying a signature in delta_generator, a workflow rarely used,
the program will abort if the signature is not valid. The unittests
exercise this flow by executing "delta_generator" with invalid keys
which causes it to abort during normal test runs. This creates lots of
unnecessary crash reports.
This patch replaces the CHECK() call with an exit(1) (returing 1 from
main() actually) to prevent sending SIGABRT and collecting crash
reports during the test run. Other minor linter fixes included.
Bug: None
Test: Manually ran the unittests.
Change-Id: I451a3150d72018652b703db7d3b82f4e787c8f43
enerate_delta_main.cc
|
| 71be9371f285df3b1aa64dfbae7e0f8853024565 |
04-Aug-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Use the bsdiff library instead of its executable
Currently we use bsdiff binary to generate BSDIFF and SOURCE_BSDIFF
operations. This involves writing both the source and destination
extents into disk and letting bsdiff read them again into memory. Now
that we actually can use the bsdiff library itself, why not use the
provided bsdiff library functions that can directly read source and
destination data from the memory that we already have. This CL will
reduce the patch generating time and memory footprint.
BUG=none
TEST=cros_workon_make --board=amd64-generic --test update_engine; \
brillo_update_payload generate --payload=payload.delta \
--target_image=veyron-minnie-R62-9804.0.0.bin --source_image=veyron_minnie-R61-9765.13.0.bin
Change-Id: Ib115eb5186c133d56dab91f73a2bde3176f5732d
Reviewed-on: https://chromium-review.googlesource.com/602887
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
elta_diff_utils.cc
elta_diff_utils.h
|
| 49fdb09d1ff2e078fa44c26540a5b82900df0f9a |
04-Aug-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Change imgdiff/imgpatch to a dummy puffdiff/puffpatch
Currently we never create imgdiff operation as it was never
activated. In near future we will be adding puffdiff and puffpatch
operations which replaces the imgdiff. This CL does not actually adds
puffdiff, but adds a placeholder for it.
BUG=none
TEST=cros_workon_make --board=amd64-generic --test update_engine
Change-Id: I6453048acb65c052354a1658f0c6fd41ad33e242
Reviewed-on: https://chromium-review.googlesource.com/602733
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| 1352093b7e5f1134ebfa99ad1991753ff4f3f33a |
26-Jul-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Add "verify" support to brillo_update_payload
This change adds a new command "verify" to brillo_update_payload to verify the
process of update by applying a delta or full payload to temporary target
partitions and comparing them with the original target partitions. This is
specially usefull when manually debuggin/testing delta performer operations.
BUG=none
TEST=brillo_update_payload verify --payload=payload.delta --source_image=link_8872.49.bin --target_image=link_9000.82.bin
Change-Id: I4b30bc8a1088f4f72b681c6095cca6863a715078
Reviewed-on: https://chromium-review.googlesource.com/585565
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
enerate_delta_main.cc
|
| a7ee9d0e6297a16d0f10cc7825c3b68867f19b24 |
03-Jun-2017 |
Andrey Ulanov <andreyu@google.com> |
update_engine: Remove zlib fingerprint checking
This is not needed as imgdiff is not being used and will be replaced by
puffin.
Presence of this code makes updating to newer version of zlib
impossible. (e.g. 1.2.8->1.2.11 update introduces some changes in
compression algorithm which make compressed output different).
BUG=None
TEST="ebuild zlib.ebuild test"
Change-Id: I50793f552a44faaba5771271fdcf2b4f4aeba6e4
Reviewed-on: https://chromium-review.googlesource.com/522949
Commit-Ready: Andrey Ulanov <andreyu@google.com>
Tested-by: Andrey Ulanov <andreyu@google.com>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
enerate_delta_main.cc
|
| 20053fc3d5a5628f593f71ee2c6dbede6c1186ea |
04-Aug-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Use the bsdiff library instead of its executable
Currently we use bsdiff binary to generate BSDIFF and SOURCE_BSDIFF
operations. This involves writing both the source and destination
extents into disk and letting bsdiff read them again into memory. Now
that we actually can use the bsdiff library itself, why not use the
provided bsdiff library functions that can directly read source and
destination data from the memory that we already have. This CL will
reduce the patch generating time and memory footprint.
BUG=none
TEST=cros_workon_make --board=amd64-generic --test update_engine; \
brillo_update_payload generate --payload=payload.delta \
--target_image=veyron-minnie-R62-9804.0.0.bin --source_image=veyron_minnie-R61-9765.13.0.bin
Change-Id: Ib115eb5186c133d56dab91f73a2bde3176f5732d
Reviewed-on: https://chromium-review.googlesource.com/602887
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
elta_diff_utils.cc
elta_diff_utils.h
|
| 5ef5d4561dbc40c128d1e9bbd8e2da525adcf581 |
04-Aug-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Change imgdiff/imgpatch to a dummy puffdiff/puffpatch
Currently we never create imgdiff operation as it was never
activated. In near future we will be adding puffdiff and puffpatch
operations which replaces the imgdiff. This CL does not actually adds
puffdiff, but adds a placeholder for it.
BUG=none
TEST=cros_workon_make --board=amd64-generic --test update_engine
Change-Id: I6453048acb65c052354a1658f0c6fd41ad33e242
Reviewed-on: https://chromium-review.googlesource.com/602733
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| 4b9775a26cd5ff26db61b7762667d607458082f0 |
26-Jul-2017 |
Amin Hassani <ahassani@google.com> |
update_engine: Add "verify" support to brillo_update_payload
This change adds a new command "verify" to brillo_update_payload to verify the
process of update by applying a delta or full payload to temporary target
partitions and comparing them with the original target partitions. This is
specially usefull when manually debuggin/testing delta performer operations.
BUG=none
TEST=brillo_update_payload verify --payload=payload.delta --source_image=link_8872.49.bin --target_image=link_9000.82.bin
Change-Id: I4b30bc8a1088f4f72b681c6095cca6863a715078
Reviewed-on: https://chromium-review.googlesource.com/585565
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
enerate_delta_main.cc
|
| 73b18b83ebae881cefa6fb2b0563a0b94c17bcde |
22-Jun-2017 |
Sen Jiang <senj@google.com> |
update_engine: Only split operation if it has multiple dst extents.
FragmentOperations() will split operations so that every operation will
have only one dst extent, but if the operation already meet this
requirement, it should be a no-op and we shouldn't waste time recompressing
it. This can save a lot of time if there's huge replace operation that
only has one dst extent.
As a side effect of not splitting those operations, if they are SOURCE_COPY
operations, they will not have src_length and dst_length set anymore,
because those were set in SplitSourceCopy() before this change, but these
lengths are never actually used in delta performer since the very first
implementation of PerformSourceCopyOperation():
https://chromium-review.googlesource.com/c/263747/
So it's safe to remove it from the manifest. We might as well just remove
src_length and dst_length in all SOURCE_COPY operations (even for those
are splitted) to reduce payload size a little bit.
BUG=b:62470452
TEST=generated a payload, all operations are still the same except
SOURCE_COPY don't have src_length and dst_length any more
Change-Id: I019ca9056479165e76cc25d7a4ba453088398928
(cherry picked from commit 929461ab11769339108658f2022d068446ca90f7)
Reviewed-on: https://chromium-review.googlesource.com/594909
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
b_generator.cc
b_generator_unittest.cc
|
| 3d32f85211f914faca4cb3a368b49c05c7e7ff98 |
29-Jun-2017 |
Simon Glass <sjg@chromium.org> |
update_engine: Reduce verbosity of delta_generator
This outputs ~4000 lines of the form:
0.00% 48 REPLACE_BZ <boot-operation-2>
which is not very useful in the build. Update it to show only unique lines
with a count of how many times that line was repeated.
BUG=chromium:737687
TEST=FEATURES=test emerge-reef update_engine
See the output. I have not added a specific test for this since it is
really just debug output IMO.
Change-Id: I15149733216ba0516c62f99e5968768ec4fa3d20
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/553579
Reviewed-by: Ben Chan <benchan@chromium.org>
ayload_file.cc
|
| 7ecda265a87236e83cf820364947a1618872b6be |
12-Jul-2017 |
Amin Hassani <ahassani@google.com> |
Open partitions with O_DSYNC flag only if the update is periodic.
Currently when updating we always open the target partition with flag O_DSYNC
(CL:562552), but this makes all infrastructure operations like 'cros flash',
provisioning, force update, paygen, etc much slower. This changes the update
engine to only add O_DSYNC flag if an update is triggered by periodic checks
(not interactively forced). This means if the user clicks on 'check for update'
it will be an interactive update and O_DSYNC will not be used. This change keeps
the AOSP partitions open without O_DSYNC flag. This CL uses non-interactive mode
for all unit tests but currently there are no integration test like provisioning
for triggering periodic updates.
Currently 'parrot' board canaries (only board with rotating HDD) is failing due
to timeouts related to slow updates. This CL potentially will clear that problem.
TEST=cros_workon_make --test, installed an image with/out the O_DSYCN flag and
measured the 'cros flash' time.
BUG=chromium:738027
Change-Id: If45fcf5e798b9c9353e09021ad812c859d983a65
Reviewed-on: https://chromium-review.googlesource.com/567360
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
enerate_delta_main.cc
|
| 0affc2c099fac47b999ba39f8f1bacb418e65a97 |
11-Feb-2017 |
Sen Jiang <senj@google.com> |
Parse multiple packages from Omaha response.
The multi-payload info are stored in OmahaResponse and InstallPlan, but
we still can only apply the first payload for now.
Bug: 36252799
Test: mma -j
Test: update_engine_unittests
Change-Id: I5ca63944ae9082670d0e67888409374f140d4245
(cherry picked from commit 2aba8a87d4fac245a2e2d238b3159f8eabce630f)
enerate_delta_main.cc
|
| 2703ef4466066d64d8021904e233b120f38c0272 |
16-Mar-2017 |
Sen Jiang <senj@google.com> |
Store raw payload hash blob in install plan.
We were using a custom sha256 pair in Omaha response, now that Omaha
has a standard hash_sha256 field in package, we should use that instead.
The difference is that hash_sha256 is encoded in hex instead of base64,
but the android payload property is still using base64, to be backward
compatible, we have to keep accepting base64 there, to avoid decoding
and then re-encoding to another encoding, we store the decoded raw hash.
Also removed the hash() related functions in HashCalculator, since it's
rarely used and the caller should encode it in whatever encoding they
want.
Also make use of RawHashOfBytes to simply code in a few places.
Bug: 36252799
Test: update_engine_unittests
Change-Id: Iaa02611b4c9cda3ead5de51e777e8caba6d99d93
(cherry picked from commit f14d51b6823522f6b2eb834f9e14d72c8363a3ad)
elta_diff_utils.cc
ayload_file.cc
|
| 5c02c1335b98b3a1c8759a664bf7b6a48773f913 |
27-Jun-2017 |
Ben Chan <benchan@chromium.org> |
Replace brillo::make_unique_ptr with base::MakeUnique/base::WrapUnique.
base::MakeUnique mimics std::make_unique and is preferred over
brillo::make_unique_ptr (which is identical to base::WrapUnique).
brillo::make_unique_ptr will eventually be deprecated and removed as its
functionality is well covered by base::MakeUnique and base::WrapUnique.
BUG=chromium:704644
TEST=Run unit tests.
Change-Id: I771b312d04643a1c74dd843947ed30bda510eed2
Reviewed-on: https://chromium-review.googlesource.com/458562
Commit-Ready: Ben Chan <benchan@chromium.org>
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Alex Deymo <deymo@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
apfile_filesystem.cc
ip_unittest.cc
|
| fca4775f0b1ad4e7c013bf626b42d06f6b74738e |
16-Jun-2017 |
Amin Hassani <ahassani@google.com> |
Contained clang specific pragmas inside #if defined(__clang__) #endif
BUG=none
TEST=cbuildbot --remote lakitu-release builds fine.
Change-Id: Ia1b7df02c5726a44cc9261db492a805f7de7a5a6
Reviewed-on: https://chromium-review.googlesource.com/538921
Commit-Ready: Amin Hassani <ahassani@chromium.org>
Tested-by: Amin Hassani <ahassani@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
elta_diff_utils.cc
xt2_filesystem.cc
xt2_filesystem.h
|
| 929461ab11769339108658f2022d068446ca90f7 |
22-Jun-2017 |
Sen Jiang <senj@google.com> |
Only split operation if it has multiple dst extents.
FragmentOperations() will split operations so that every operation will
have only one dst extent, but if the operation already meet this
requirement, it should be a no-op and we shouldn't waste time recompressing
it. This can save a lot of time if there's huge replace operation that
only has one dst extent.
As a side effect of not splitting those operations, if they are SOURCE_COPY
operations, they will not have src_length and dst_length set anymore,
because those were set in SplitSourceCopy() before this change, but these
lengths are never actually used in delta performer since the very first
implementation of PerformSourceCopyOperation():
https://chromium-review.googlesource.com/c/263747/
So it's safe to remove it from the manifest. We might as well just remove
src_length and dst_length in all SOURCE_COPY operations (even for those
are splitted) to reduce payload size a little bit.
Bug: 62470452
Test: generated a payload, all operations are still the same except
SOURCE_COPY don't have src_length and dst_length any more
Change-Id: I019ca9056479165e76cc25d7a4ba453088398928
b_generator.cc
b_generator_unittest.cc
|
| c4ad1ebc33abc088aca2909ba5cbaf7ae5e5659f |
16-Jun-2017 |
Tianjie Xu <xunchang@google.com> |
Add multi thread support for A/B generator
Use threadpool to calculate file deltas in parallel. This decreases the
time consumption from 40 mins to 13 mins for the incremental payload generation
of two taimen system images.
Bug: 62470452
Test: Unit tests pass && generate an identical payload for incremental
taimen as before
Change-Id: Ie295504b758a1baa6b661c76bcbb6aa7881ded5d
elta_diff_utils.cc
elta_diff_utils.h
ull_update_generator.cc
|
| 013ebf9cc81b1be2eaef6a0e1d19ff806a8fcdb8 |
08-Jun-2017 |
Amin Hassani <ahassani@google.com> |
Merge branch 'cros/upstream' into 'cros/master'
Fixed:
connection_manager_interface.h: remove line 53-56
fix the assert in common/http_fetcher_unittest.cc:637
BUG=none
TEST=sent squashed merge to buildbots and HWtest was success.
Change-Id: I15037921592a2b7a4dd76dff71417d03caf97cf6
|
| 948ea5885a76f2d79e39aa178b83c4a955207458 |
03-Jun-2017 |
Andrey Ulanov <andreyu@google.com> |
update_engine: Remove zlib fingerprint checking
This is not needed as imgdiff is not being used and will be replaced by
puffin.
Presence of this code makes updating to newer version of zlib
impossible. (e.g. 1.2.8->1.2.11 update introduces some changes in
compression algorithm which make compressed output different).
BUG=None
TEST="ebuild zlib.ebuild test"
Change-Id: I50793f552a44faaba5771271fdcf2b4f4aeba6e4
Reviewed-on: https://chromium-review.googlesource.com/522949
Commit-Ready: Andrey Ulanov <andreyu@google.com>
Tested-by: Andrey Ulanov <andreyu@google.com>
Reviewed-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Sen Jiang <senj@chromium.org>
enerate_delta_main.cc
|
| dcb18612576aeb59fbd8ba066b1b7a3a7bca8490 |
24-Feb-2017 |
Christopher Ferris <cferris@google.com> |
Disable macro redefined warnings on some includes.
The new kernel headers do:
#define __bitwise __bitwise__
However, the code redefines __bitwise without undef'ing it first.
The files affected are the include files from external/e2fsprogs,
so the warnings will only be disabled around those includes.
This is a temporary fix, b/35721782 filed to fix this.
Test: Builds without warningers/errors.
Change-Id: I2eae12558c23be9bb3bf3931467b32eee5d57720
elta_diff_utils.cc
xt2_filesystem.cc
xt2_filesystem.h
|
| 20bdc70e540e021c4a26553dfdbfd2fabe8ea432 |
08-Dec-2016 |
Alex Deymo <deymo@google.com> |
Parse Android .map files for unknown filesystem.
When generating a filesystem during the Android build, we also generate
a text file with .map extension with the list of files and the blocks
in the filesystem they are located.
For filesystems unsupported in delta_generator (like squashfs) we use
this text file to produce efficient delta payloads.
Bug: 28150981
Test: Added unittest for parsing. Generated a delta payload of a squashfs image.
Change-Id: I154e72ac785c6f508290daa901fa7958b446c010
elta_diff_generator.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
enerate_delta_main.cc
apfile_filesystem.cc
apfile_filesystem.h
apfile_filesystem_unittest.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| 95699a944e03564282f64f561bc7f4bd6278400f |
09-Dec-2016 |
Alex Deymo <deymo@google.com> |
Make FileWriter errors verbose.
Print the errno message whenever there's a failure in the underlying
write function.
Bug: 32943635
Test: delta_generator --payload /path/to/nonexistent/place
Change-Id: Iba2961b7b7cd523ed5e651f6025f1d8e5f5e00a2
ayload_file.cc
|
| efbeecf05165d1d4547a58bb2e76ffe5799ff6ea |
16-Nov-2016 |
Rahul Chaudhry <rahulchaudhry@google.com> |
system/update_engine/payload_generator: cleanup compiler warnings (unused value).
system/update_engine/payload_generator/delta_diff_utils.cc:124:7: warning: Value stored to 'new_src' is never read
system/update_engine/payload_generator/delta_diff_utils.cc:128:7: warning: Value stored to 'new_dst' is never read
system/update_engine/payload_generator/extent_ranges.cc:248:5: warning: Value stored to 'out_blocks' is never read
Bug: 26936282
Test: WITH_TIDY=1 WITH_STATIC_ANALYZER=1 mm
Change-Id: I55c0bbafc134d01482e533e69954d7d14ec241b7
elta_diff_utils.cc
xtent_ranges.cc
|
| 5b91c6b141970c2b0095775a61e3f941417aa1ff |
05-Aug-2016 |
Alex Deymo <deymo@google.com> |
New setting to mark postinstall as optional.
This setting allows to mark a postinstall script as optional. This
allows the postinstall program to fail when it is not strictly required
to run for the update to succeed.
Bug: 27178350
TEST=Added unittest. Sideloaded an update with an optional postinstall.
Change-Id: I41956d3308f3458b6bf94b835c9b5e470c84ca41
ayload_file.cc
ayload_generation_config.cc
ayload_generation_config.h
ayload_generation_config_unittest.cc
|
| 5c6bb1d8f89b87e782e26fe74789b6f510704394 |
27-Jul-2016 |
Chih-Hung Hsieh <chh@google.com> |
Fix clang-tidy performance warnings in update_engine.
* Use const reference type for parameters, local variables,
and for-loop index variables to avoid unnecessary copy.
* Convert some for-loops to for-rang loops.
Bug: 30407689
Bug: 30413223
Bug: 30413862
Change-Id: I78996b3f799639fc57ced45e110807625be7dcce
Test: build with WITH_TIDY=1
b_generator.cc
b_generator_unittest.cc
elta_diff_utils.cc
ake_filesystem.cc
ake_filesystem.h
nplace_generator.cc
nplace_generator.h
ayload_signer_unittest.cc
|
| 9d66d761f5fdbd7846810954c5245c5391b743c4 |
15-Jul-2016 |
Tamas Berghammer <tberghammer@google.com> |
Add type casts to silence warnings intorduced by protobuf v3.0.0
Bug: b/28974522
Change-Id: I1a5a26ea503747b786ed38ae5d6a586340a5fe2e
elta_diff_utils.cc
elta_diff_utils_unittest.cc
xtent_utils.cc
|
| 05871fab8b32938ad66ce230250ff6db5563bd7a |
01-Jun-2016 |
Alex Deymo <deymo@google.com> |
Allow to shrink ext2 filesystems.
There was a function a long time ago that didn't handle shrinking an
ext2 filesystem when computing the diff of the blocks that were not
part of a file nor metadata (mostly empty space blocks). This is now
handled in a different way, which is not related to how the filesystems
are parsed in the ABGenerator.
This patch removes the check that prevents the delta generator from
generating those payloads and fixes the inplace generator for partitions
that shrink in size.
Bug: 28846535
TEST=Generated a payload to a smaller ext2 system image, both for
minor version 1 and 2+.
Change-Id: Ide408c48b0032dbe478c8e348c40e1a0b5665ea8
elta_diff_generator.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
|
| 26dd80fb16617f0d78be938a9e2592f2c0e50365 |
11-Jun-2016 |
Alex Deymo <deymo@google.com> |
Generate SOURCE_COPY and ZERO operations beyond the end of the filesystem.
This patch analyzes the whole partition size, regardless of what the
size of the filesystem in the partition is. This makes the blocks in
the ext2 partitions past the end of the filesystem (normally verity
hashes) be compared against the old blocks and produce SOURCE_COPY
operations for those as well if possible.
Bug: 29272878
TEST=Generated a payload to itself of a build with verity hashes, only
SOURCE_COPY (and REPLACE_BZ of zeroed blocks) are produced.
Change-Id: If7f2733da41c333d4a8247b1473ad06fb63466d6
elta_diff_utils.cc
|
| a57ce77c1e7c0e7c1caa825e3f8ca204c3e5900a |
14-May-2016 |
Alex Deymo <deymo@google.com> |
payload_generator: Mark all zero blocks in the old partition as visited.
When processing duplicated and zeroed blocks, discard all the zeroed
blocks from the source partition since it doesn't make sense to waste
I/O reading them, and all the new zeroed blocks are taken care of in a
different way.
This also solves the problem that those zeroed blocks might actually not
be written in the source slot when flashing an image with fastboot.
Bug: 28626303
TEST=deployed an update on a device after flashing it with fastboot.
Change-Id: I42012a84634a5b35f181b6b22e8a738ba3b70957
elta_diff_utils.cc
|
| 4108c32f268c9d50b7aab7599733bd04526bacdf |
12-Apr-2016 |
Sen Jiang <senj@google.com> |
Don't set data_offset and data_length if data blob is empty.
ZERO operation was failing because we were setting these fields in
delta payload.
Bug: 27858697
Bug: 27156473
Test: apply a minor v4 payload.
Change-Id: I4b6abf3dde052b181a85d417f17123788855b69c
nnotated_operation.cc
elta_diff_utils.cc
ull_update_generator.cc
|
| 726aeca1c665910593761a1990ee50f22d98f53f |
06-Apr-2016 |
Alex Deymo <deymo@google.com> |
delta_generator: Use REPLACE_XZ in deltas when allowed.
This patch moves the generation of the best allowed full operation to a
new function which now includes ZERO, REPLACE, REPLACE_BZ and REPLACE_XZ
when allowed. This function is used to produce a full operation
whenever it is needed in the "full" payload generator and the AB "delta"
payload generator.
Bug: 24578399
TEST=brillo_update_payload generated REPLACE_XZ operations for a delta payload that was using REPLACE_BZ operations and merged them.
TEST=brillo_update_payload generated a full payload with REPLACE, REPLACE_BZ and REPLACE_XZ operations.
TEST=Updated unittests.
Change-Id: I29618094a84ef8050f8dd7ede0fd70056dc9d650
b_generator.cc
b_generator.h
b_generator_unittest.cc
nnotated_operation.cc
nnotated_operation.h
elta_diff_utils.cc
elta_diff_utils.h
ull_update_generator.cc
|
| a78ee102c57832b9d5a274c4d25ea23c7171729c |
01-Apr-2016 |
Sen Jiang <senj@google.com> |
Make update_engine_unittests depend on bsdiff.
Added build artifacts path to PATH for unittests.
Test: mma; ./update_engine_unittests
Bug: 26955860
Change-Id: I09401cb883be5210887d7c789fdaec4145626fa0
elta_diff_utils.cc
|
| d3251ef5abddc1c42fa0910191831454264e3cda |
30-Mar-2016 |
Alex Deymo <deymo@google.com> |
Fix memory leak in unittests with malformed data.
The MalformedZipTest was not calling End() on the extent writer,
leaking memory in the unittest.
Bug: chromium:598967
TEST=USE="clang asan" FEATURES=test emerge-link update_engine
Change-Id: I2d1fefee051d57ac17906e8ea69ba71f43386f0b
ip_unittest.cc
|
| 0497d05764dcae38e7a7419a2f25eff043172850 |
23-Mar-2016 |
Alex Deymo <deymo@google.com> |
Fix ZERO and DISCARD operations.
These operations were broken (and not used in the generator) because
the code expects them to have a blob offset, but they don't. This bug
prevents them from being used in full payloads, but can still be used
in delta payloads.
Bug: 27858697
TEST=Deployed a full payload with ZERO operations to the new update_engine.
Change-Id: Ifc67bd7c3b2a81593d0347a345723cc34eaaf6b5
ull_update_generator.cc
ayload_generation_config.cc
|
| d41606fb3c25f6cb8191e154c87391ab285a8a2f |
23-Mar-2016 |
Alex Deymo <deymo@google.com> |
Enable REPLACE_XZ operations in full payloads.
This patch enables REPLACE_XZ operations in full payloads when the
operation is allowed.
Bug: 24578399
TEST=Deployed a payload to an edison-eng using XZ operations.
Change-Id: I3ac92df8e90d825da67d495266a5bb4d940ad93e
ull_update_generator.cc
|
| a4073ef63482fd08c3678982f7d153360b088094 |
23-Mar-2016 |
Alex Deymo <deymo@google.com> |
Introduce PayloadVersion struct for version information.
This new little struct encapsulates the version information (major and
minor version numbers) and the zlib fingerprint information. Now,
instead of querying throughout if the version number is certain value,
we ask the PayloadVersion struct whether certain operation is allowed in
that version or not. This moves the logic of what's supported and
what's not to a single place and eliminates the need to pass several
booleans to the helper functions.
Bug: None
TEST=Unittest still pass.
Change-Id: Iaa6e7bc032db6479fdeab72255d7062fca1c07e5
b_generator.cc
elta_diff_generator.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
nplace_generator.cc
ayload_file.cc
ayload_generation_config.cc
ayload_generation_config.h
ayload_signer_unittest.cc
|
| 260f03bc4d3a3de436e056c686c814444358823a |
21-Mar-2016 |
Sen Jiang <senj@google.com> |
Fix unittest key path in Brillo.
The unittests weren't able to find the keys if it's not run from the
update_engine_unittests directory.
Test: /data/nativetest/update_engine_unittests/update_engine_unittests
Bug: 26955860
Change-Id: I4c189cca2714986d3d2b2669b54a5dff9a488964
elta_diff_utils_unittest.cc
xt2_filesystem_unittest.cc
ayload_signer_unittest.cc
|
| 246bf21fd0074a1897c28a7c4703c2223e6224b6 |
23-Mar-2016 |
Alex Deymo <deymo@google.com> |
Implement Xz compressor functions.
The new XzCompress() function is similar to BzipCompress() function but
uses the Xz compression algorithm.
This patch simplifies the unittests of the compressors and reuses the
client-side decompresor implementation instead of repeating the
implementation in the delta generator. This patch removes the unused
compression/decompression functions.
Bug: 24578399
TEST=Added unittests.
Change-Id: Id858112b50f4aa2597f184dc23a86af772f4f190
zip.cc
zip.h
enerate_delta_main.cc
z.h
z_android.cc
z_chromeos.cc
ip_unittest.cc
|
| dcbc0ae6a5c48b40a1bc7d3c2ed62ec2a9fe7748 |
18-Mar-2016 |
Sen Jiang <senj@google.com> |
Remove GetFilesystemSize.
We don't need to get filesystem size now, because the files passed to
delta_generator is already resized properly.
Also moved Is*Filesystem() to delta_diff_utils.
Test: ./update_engine_unittests
Bug: None
Change-Id: I442d69d1820d08c0f3bd63e7f1986fc4b0d6655e
elta_diff_generator.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
ayload_generation_config.cc
|
| 990c27b0df5bd77ca804479922a62ba0ee9266ba |
16-Mar-2016 |
Sen Jiang <senj@google.com> |
Use pre-generated ext image instead of create it on the fly.
mkfs.ext3 is not available in target device, so we don't use it.
Also removed all usage of dd.
This fixes a UtilsTest and the first step of integration test, the rest is
still failing because bsdiff is missing.
Test: ./update_engine_unittests
Test: cros_workon_make update_engine --test
Bug: 26955860
Change-Id: Ic600bd6fab1d6839f38d5ef767fbffc5a35571a4
xt2_filesystem_unittest.cc
|
| e0d04282910d423ddc950dd38224c001efc41d42 |
01-Mar-2016 |
Sen Jiang <senj@google.com> |
Check if zlib is compatible.
Add a --zlib_fingerprint flag in delta_generator to check if the zlib of
source image is compatible, and only enable IMGDIFF operation if it is.
Test: Added unittest to check /etc/zlib_fingerprint
Bug: 27156099
Change-Id: Ida346a99430c95cdc8a43e1055d6efa08f07ca25
enerate_delta_main.cc
|
| 923886a86013fbba05d00dd64d4ddc9b9a6dffe4 |
14-Mar-2016 |
Sen Jiang <senj@google.com> |
Fix PayloadSignerTest.* in Brillo.
Added modules to install missing private and public keys.
There's no openssl binary in target device, rewrote SignHash() using
openssl library functions.
Test: GTEST_FILTER="PayloadSignerTest.*" ./update_engine_unittests
Bug: 26955860
Change-Id: Ie7f229d5456f641394fd57f5a6e9ca3cc5c5f0cd
ayload_signer.cc
|
| d15c546ed794293d0a63770467a0f3c4c84c6214 |
10-Mar-2016 |
Alex Deymo <deymo@google.com> |
Implement susped, resume and cancel for the Postinstall action.
This patch sends SIGSTOP/SIGCONT to the running postinstall program to
suspend/resume the child process, and SIGKILL when cancelling it.
Bug: 27272144
TEST=Added unittest to check the signal being sent.
Change-Id: Iebe9bd34448ad1d0a5340c82e1fd839ff8c69dd2
xt2_filesystem_unittest.cc
|
| 55c4f9ba7f5c59e3345f2c1869464433ffa8dc2b |
10-Feb-2016 |
Sen Jiang <senj@google.com> |
Include IMGDIFF operation in minor version 4 or up.
Try imgdiff if both source and target contains gzip.
Test: unit test & generated a payload with boot.img
Bug: 26675118
Change-Id: I2861d9b953ffbdec44fdfb42cff5687698ea205e
b_generator.cc
elta_diff_generator.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
xtent_utils.cc
xtent_utils.h
enerate_delta_main.cc
nplace_generator.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| c90be63117cdad2622694b6f9ac2aec65fa80bf0 |
18-Feb-2016 |
Alex Deymo <deymo@google.com> |
Fix Ext2FilesystemTest unittests on Brillo.
The Ext2FilesystemTest tests use a set of pre-generated filesystems
stored in sample_images.tar.bz2 and attempt to parse those. This patch
installs the used filesystems in the same directory as the Chrome OS
test environment would generate those files, relative to the unittest
program executed: in a gen/ subdirectory.
Also, an empty filesystem that was previously created from the unittests
is now pre-generated.
The update_engine_unittests program now depends on those modules being
installed and those unittests pass.
Bug: 26955860
TEST=/data/nativetest/update_engine_unittests/update_engine_unittests --gtest_filter=Ext2FilesystemTest.*
Change-Id: I40082e328cec89a7ef4f06d3b8125a953b29efad
xt2_filesystem_unittest.cc
|
| 5fe0c4ede81b82ae3425ddbbb698eceef14cbc78 |
17-Feb-2016 |
Alex Deymo <deymo@google.com> |
Fix unittest build in x86_64.
Some size_t values (generally resulting from X.size() calls) were
compared to off_t values (such as utils::FileSize()) which resulted
in a compile error on x86_64 targets. This patch fixes those issues
with explicit casts in the unittests since the range of the expected
values is small enough.
Bug: 26955860
TEST=`mma` on edison-eng and brilloemulator_x86_64-eng.
Change-Id: I2d09d356e1b97ab6527b17596fe20d425da6d519
b_generator_unittest.cc
ull_update_generator_unittest.cc
|
| 80f70ff45f8ea9a679c0c3ed0dc143dd2fe2b63e |
11-Feb-2016 |
Alex Deymo <deymo@google.com> |
Build unittests in Brillo.
Many unittests do not pass for simple reasons that will be addressed
later. This CL includes all the changes to make the unittests build.
In particular, the generated DBus mocks, required to build several
unittests are now included here.
The dbus-constants.h files were moved to the system_api repo, so they
can be removed from here.
The unittest build is only enabled for Brillo targets, since non-Brillo
targets don't even build DBus.
Bug: 26955860
TEST=`mmma` on edison-eng (and aosp_arm-eng).
Change-Id: Ib38241f0a6eb99b1d60d72db6bcfd125d38e3fad
b_generator_unittest.cc
lock_mapping_unittest.cc
ycle_breaker_unittest.cc
elta_diff_utils_unittest.cc
xt2_filesystem_unittest.cc
xtent_ranges_unittest.cc
xtent_utils_unittest.cc
ake_filesystem.cc
ull_update_generator_unittest.cc
raph_utils_unittest.cc
nplace_generator_unittest.cc
ayload_file_unittest.cc
ayload_signer_unittest.cc
arjan_unittest.cc
ip_unittest.cc
|
| 98e691cc6debda3ef671acba001bebd45da36377 |
05-Feb-2016 |
Alex Deymo <deymo@google.com> |
New "properties" sub-command to export payload properties.
The new brillo_update_payload sub-command "properties" dumps a list of
properties for a given signed or unsigned payload. These properties are
normally included in the Omaha response, and extracted from python in
chromite.
This new sub-command helps to encapsulate the properties used by the
server side wehn serving a payload and to let the Android application
pass these required properties.
The properties include the payload and metadata hash and size.
Bug: 26991255
TEST=FEATURES=test emerge-link update_engine
TEST=mmma system/update_engine
TEST=`brillo_update_payload properties` for signed and unsigned payloads.
Change-Id: I4602ea4b8dc269e4cc66df4293ef9765d8dd031d
enerate_delta_main.cc
ayload_signer.cc
ayload_signer.h
ayload_signer_unittest.cc
|
| 0103c36caa2e38e034e0d22185736b9ccfb35c58 |
20-Jan-2016 |
Alex Vakulenko <avakulenko@google.com> |
update_engine: Update libchrome APIs to r369476
The new libchrome has been ported from Chromium and some APIs have
changed. Make necessary changes at call sites.
Change-Id: I42e65bda7f1dbdf6f6e0ebf356d2cfea6b729193
ycle_breaker.cc
enerate_delta_main.cc
raph_types.h
|
| 542c19bf271011913a9f352f58e140224c936736 |
03-Dec-2015 |
Alex Deymo <deymo@google.com> |
Remove SystemState references from DeltaPerformer.
DeltaPerformer is used as part of the DownloadAction and had
references to the global SystemState. The common references to
BootControlInterface (to query the partitions based on the names
from the payload) and the HardwareInterface (for dev-mode bits) are
now referenced directly from the DeltaPerformer. The calls to
UpdateAttempter and PayloadState were moved to the
DownloadActionDelegate since these calls are received by classes
outside the payload_consumer.
Bug: 25773375
TEST=FEATURES=test emerge-link update_engine
Change-Id: Id2e81d5ccf835cad22f03b069c681dcce104c456
enerate_delta_main.cc
|
| 05feee0fc8f4f10b425f04e1f5cdd4831bbfb969 |
12-Nov-2015 |
Sen Jiang <senj@google.com> |
delta_generator: Include postinstall calls in the payload major version 2.
Added a new flag --new_postinstall_config_file which takes a path to a key
value store config file and saves this information to the PartitionUpdate
field.
The config file looks like this:
RUN_POSTINSTALL_root=true
POSTINSTALL_PATH_root=postinstall
FILESYSTEM_TYPE_root=ext4
"root" can be changed to any partition name.
Bug: 24537566
TEST=Generated a payload v2 with postinstall.
TEST=Added unittest.
Change-Id: Ied3c7bc2cfb54f4bcc69207f1e5bd473f72024fe
enerate_delta_main.cc
ayload_file.cc
ayload_file.h
ayload_generation_config.cc
ayload_generation_config.h
ayload_generation_config_unittest.cc
|
| 39910dcd1d68987ccee7c3031dc269233a8490bb |
10-Nov-2015 |
Alex Deymo <deymo@google.com> |
Split payload application code into a subdirectory.
This patch splits from the main libupdate_engine code the part that
is strictly used to download and apply a payload into a new static
library, moving the code to subdirectories. The new library is divided
in two subdirectories: common/ and payload_consumer/, and should not
depend on other update_engine files outside those two subdirectories.
The main difference between those two is that the common/ tools are more
generic and not tied to the payload consumer process, but otherwise they
are both compiled together.
There are still dependencies from the new libpayload_consumer library
into the main directory files and DBus generated files. Those will be
addressed in follow up CLs.
Bug: 25197634
Test: FEATURES=test emerge-link update_engine; `mm` on Brillo.
Change-Id: Id8d0204ea573627e6e26ca9ea17b9592ca95bc23
b_generator.cc
b_generator.h
b_generator_unittest.cc
nnotated_operation.cc
lob_file_writer.cc
lob_file_writer_unittest.cc
lock_mapping.cc
lock_mapping_unittest.cc
zip.cc
ycle_breaker.cc
ycle_breaker_unittest.cc
elta_diff_generator.cc
elta_diff_utils.cc
elta_diff_utils_unittest.cc
xt2_filesystem.cc
xt2_filesystem_unittest.cc
xtent_ranges.cc
xtent_ranges_unittest.cc
xtent_utils.cc
xtent_utils.h
xtent_utils_unittest.cc
ull_update_generator.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
raph_utils.cc
raph_utils_unittest.cc
nplace_generator.cc
nplace_generator_unittest.cc
ayload_file.cc
ayload_file_unittest.cc
ayload_generation_config.cc
ayload_generation_config.h
ayload_signer.cc
ayload_signer_unittest.cc
aw_filesystem.cc
arjan.cc
arjan_unittest.cc
ip_unittest.cc
|
| 82352f97b4621dbf6af8308ff0b0b17b0968b53a |
10-Nov-2015 |
Sen Jiang <senj@google.com> |
delta_generator: Include per-operation source hash.
This adds a src_sha256_hash field in the operation if minor version is 3.
Bug: 23182225
TEST=cros_workon_make update_engine --test
TEST=Generated a minor version 3 payload, the manifest is about 50% larger.
Change-Id: I0e363fca4de24bae64ac83930afdd42a6ddc21c9
b_generator.cc
b_generator.h
b_generator_unittest.cc
elta_diff_generator.cc
ayload_generation_config.cc
|
| af1575fc52d529436856ecbce3c3cd0593016b25 |
09-Nov-2015 |
Sen Jiang <senj@google.com> |
Fix logging percentage in BlobFileWriter.
Should clear number of stored blobs when enable logging.
Bug: 25293923
TEST=Generated a full payload, the percentage is correct.
Change-Id: Ie779d699db1b12105bbf77fbae3a02c41d28401b
lob_file_writer.cc
lob_file_writer.h
ull_update_generator.cc
|
| 3e728feb92bfd5514bff55aa7dc455da80e18906 |
05-Nov-2015 |
Sen Jiang <senj@google.com> |
Do not include signature dummy operation in major version 2.
It only exists for compatibility reason, for major version 2, there's no
point to add this any more.
Bug: None
TEST=Applied the new payload to a device.
Change-Id: I5803ab755415a1ba3d7460d82956bfe6e9fd4547
ayload_file.cc
ayload_signer.cc
ayload_signer.h
|
| 644f618b8bbaa096c1fcdcf7d6ed06e2538ea83c |
07-Oct-2015 |
Sen Jiang <senj@google.com> |
delta_generator: Include metadata signature in major version 2.
--metadata_signature_file flag does what it should now.
Note that you should only pass this flag if the payload you are signing is
version 2.
Bug: 23981164
TEST=unit test added.
Change-Id: I613cd6a5fef188eca37c46c3f8a0a41d1c22f2fd
enerate_delta_main.cc
ayload_file.cc
ayload_signer.cc
ayload_signer.h
ayload_signer_unittest.cc
|
| 720df3eeaecfc9ad229440c59e08c8d52f13dc1e |
01-Oct-2015 |
Sen Jiang <senj@google.com> |
Skip metadata signature when hashing payload in major version 2.
We should skip metadata signature when hashing the payload, so that the
payload signature won't depend on metadata signature.
VerifySignedPayload will also verify metadata signature now if it exist.
Bug: 23981164
TEST=cros_workon_make update_engine --test
Change-Id: I3e52b7bf8ddf1539bbb6934e8a5ec1112b94ae62
enerate_delta_main.cc
ayload_file.cc
ayload_signer.cc
ayload_signer.h
ayload_signer_unittest.cc
|
| abdc0115548985d8ca3f3f2aeb1283741ce409a6 |
22-Oct-2015 |
Alex Deymo <deymo@google.com> |
Fix building delta_generator.
Previous CL had a few include renames missing.
Bug: None
Test: sudo emerge update_engine
Change-Id: Ifcdf5f11d52561cf3f976f18c0d66b95dd3c0d66
b_generator.cc
b_generator_unittest.cc
elta_diff_utils.cc
|
| 0bc2611b49aba30202fdb9c23871430eb67e4738 |
20-Oct-2015 |
Alex Deymo <deymo@google.com> |
Move bzip.h to payload_generator.
The in-memory bzip compression and decompression functions are only
used during payload generation and testing. This patch moves those
functions to the payload_generator/ directory and keeps the streamed
version (bzip_extent_writter.h) in the update_engine daemon.
Bug: None
Test: FEATURES=test emerge-link update_engine; `mm` on Brillo.
Change-Id: I78227fdac15df6461b23debb41c72207b22fb5db
zip.cc
zip.h
ull_update_generator.cc
ip_unittest.cc
|
| aef1c6f0b5a0059a103251a9a93bd0be43bd50a2 |
07-Oct-2015 |
Sen Jiang <senj@google.com> |
PayloadVerifier should not depend on DeltaPerformer to load payload.
The implementation in DeltaPerformer is designed for situation that payload
might only partially available, but in PayloadVerifier we already have the
whole payload. So I implemented LoadPayload in PayloadSigner which logic is
simpler and supports both version 1 and 2.
VerifySignedPayload is also moved to PayloadSigner since it's not used in
update engine daemon.
This patch also fixed wrong metadata size out in version 2 and misspelling
of metadata in WritePayload in PayloadFile.
Bug: 23981164
TEST=unit test added.
Change-Id: Id1917fc891dbf2075978a273d1a4ee3c4ecf0571
enerate_delta_main.cc
ayload_file.cc
ayload_file.h
ayload_signer.cc
ayload_signer.h
ayload_signer_unittest.cc
|
| 8241ffb63697b9ae7e91b52e71d36fe17470e43c |
15-Oct-2015 |
Alex Deymo <deymo@google.com> |
Print operation names using InstallOperationTypeName().
PayloadFile had a local array to convert from the operation type enum
to a human-readable string name. Since the enum is converted to an int
this method will not fail to compile whenever new operations are added.
This patch removes such array and uses the existing
InstallOperationTypeName() function to convert the enum to string. This
fixes a buffer overflow in test code that generates new operations.
Bug: chromium:543593
Test: USE="clang asan" FEATURES=test emerge-link update_engine
Change-Id: Ib8d80711d039edf18a323aefaad4ce96e5dfc2e3
ayload_file.cc
|
| 3f39d5cc753905874d8d93bef94f857b8808f19e |
13-Oct-2015 |
Alex Vakulenko <avakulenko@google.com> |
update_engine: Rename "chromeos" -> "brillo" in include paths and namespaces
libchromeos is transitioning to libbrillo and chromeos namespaces
and include directory is changing to brillo.
Bug: 24872993
Change-Id: I770659a95be380a50fe3b2ba9f91d65818f40945
b_generator.cc
b_generator.h
b_generator_unittest.cc
nnotated_operation.cc
nnotated_operation.h
lob_file_writer.cc
lob_file_writer.h
lob_file_writer_unittest.cc
lock_mapping.cc
lock_mapping.h
lock_mapping_unittest.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
xt2_filesystem.cc
xt2_filesystem.h
xt2_filesystem_unittest.cc
ake_filesystem.cc
ake_filesystem.h
ilesystem_interface.h
ull_update_generator.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
ayload_file.cc
ayload_file.h
ayload_signer.cc
ayload_signer.h
ayload_signer_unittest.cc
aw_filesystem.h
|
| 25fbdf83064c78dfb4d9143be25f080ee2c56730 |
13-Oct-2015 |
Sen Jiang <senj@google.com> |
Merge "delta_generator: Add --out_metadata_size_file flag."
|
| e5e5fe926e9ea45b1a381af1bee91a86643ffd72 |
05-Oct-2015 |
Alex Deymo <deymo@google.com> |
Move InstallPlan partitions to a list of partitions.
This patch changes the InstallPlan instance from having hard-coded
rootfs and kernel paritions to have a list of partitions with a
name, source and target information.
The FilesystemVerifierAction, DeltaPerformer and PostInstallAction were
adapter to use the list of partitions instead.
In delta payloads (only supported in the current major version 1) the
list of operations is still fixed: the rootfs first and the kernel.
This list is now populated by the FilesystemVerifierAction including
the size of these partitions, until the whole source partition hash
checking is deprecated (b/23182225).
The PostIntallAction now relies on the DeltaPerformer to populate the
post-install information from the payload. This means that in rollback
we won't run any device-specific post-install operation, and will
simply flip the slots in the bootloader.
Bug: 24667689
Test: Updated unittests. Tested on a dragonboard and a link.
Change-Id: I8277e3190ac74e57832a58dc0730e3713f48af8a
enerate_delta_main.cc
|
| d8ae872e3b5395580b3679cc92f1c5e52e409c16 |
09-Oct-2015 |
Sen Jiang <senj@google.com> |
delta_generator: Add --out_metadata_size_file flag.
delta_generator will write the metadata size in text to the given file.
This flag will only be accepted when generating or signing the payload.
Bug: 24751934
TEST=called delta_generator with this flag.
Change-Id: I824079dc4c18534a80ded77902f4ea8f6dbbd20f
enerate_delta_main.cc
|
| 7e13b005a84763110ce539ba8f64d08b47c016c9 |
09-Oct-2015 |
Sen Jiang <senj@google.com> |
Merge "delta_generator: Accept a list of partitions in the command line."
|
| b8060e435f0edb8efef4891a99fa18f642e01aa2 |
25-Sep-2015 |
Sen Jiang <senj@google.com> |
Parse metadata signature size in payload version 2.
This patch only parse the field and skip the signature without verifying it.
Bug: 23946683
TEST=unit test added.
Change-Id: I53e049c35f8c21d325aeb415ac9a2daf980fcda1
ayload_file.cc
|
| 5df68364fb035040059e86c13b9f49d32a51dd98 |
22-Sep-2015 |
Sen Jiang <senj@google.com> |
delta_generator: Accept a list of partitions in the command line.
New flags are old_partitions, new_partitions and partition_names.
They all accecpt a colon separated string.
Bug: 24387911
TEST=Called delta_generator with the new flags to generate a payload.
Change-Id: Iad0669e53a3326410700902e4e2d45ca7ca067c1
enerate_delta_main.cc
|
| 981eb1151f42480ea10cb5283ab2108454a16c92 |
26-Aug-2015 |
Sen Jiang <senj@google.com> |
Support multiple partitions in ImageConfig
ImageConfig now have a vector of partitions instead of hardcoded rootfs and
kernel.
Bug: 23420126
TEST=cros_workon_make update_engine --test
Change-Id: Id226cc04628b44f1fcbd58f03038809867bf9e40
elta_diff_generator.cc
enerate_delta_main.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| 0d04ce38f933c06b34a696c5f56f776baa0925b8 |
02-Oct-2015 |
Sen Jiang <senj@google.com> |
Merge "Use file size instead of filesystem size for all partitions."
|
| 2d621a3cf2da7ff3b2a178594cb97ada6c4350d3 |
01-Oct-2015 |
Alex Deymo <deymo@google.com> |
Implement REPLACE_XZ on the update_engine.
This patch introduces the REPLACE_XZ operation, very similar to
REPLACE_BZ but using the XzExtentWriter instead. Minor cleanup of the
growing list of operations included in this patch.
Bug: 23604708
Test: Added unittests.
Change-Id: Iefa9e2cda609fceab9311add16b7369cb88c98a2
nnotated_operation.cc
nnotated_operation.h
|
| f2af4c6a44bfc9fdb31b3cceea14a9fb2d63d61a |
01-Oct-2015 |
Sen Jiang <senj@google.com> |
Use file size instead of filesystem size for all partitions.
Bug: 23420126
TEST=cros_workon_make update_engine --test
Change-Id: I994f527403a31234d7a3fed82ba43220149a555f
ayload_generation_config.cc
|
| a59304af39a65c9e9105d5f23cd14e064f0ff9c4 |
30-Sep-2015 |
Sen Jiang <senj@google.com> |
Merge "Add metadata signature size field in payload major version 2."
|
| b552a68af39efa42d462a9272d3dc47eb39c6db8 |
30-Sep-2015 |
Alex Deymo <deymo@google.com> |
Check all signatures regardless of the version.
The update_engine daemon had a fixed version number for the public key
used to verify both the metadata and whole payload signatures. The
public key itself is installed by the signer, implying that the source
code and the signer need to be in sync if we ever need to roll the
payload key.
This situation becomes more of a problem if we don't control when the
version number included in the source code is updated in the built
image sent for payload generation and signing.
This patch makes update_engine ignore the version number associated
with a signature and instead tries to verify all the signatures
included in the payload against the public key found in the code. This
effectively deprecates the key version number. To be compatible with
old versions, the version number 1 is included in all signatures.
Bug: 23601118
Test: Added unittests.
Change-Id: I4f96cc207ad6b9c011def5ce586d0e0e85af28ab
enerate_delta_main.cc
ayload_signer.cc
ayload_signer_unittest.cc
|
| f4bb3e6fec85d1243d27c47af2a1778586c9919e |
29-Sep-2015 |
Sen Jiang <senj@google.com> |
Add metadata signature size field in payload major version 2.
This field is always 0 for now.
Bug: 23981164
TEST=Generated a v2 payload, it has this field now.
Change-Id: Ie6cff3c7ff478e83cf46bf6ac378f95ef197cad3
ayload_file.cc
|
| cbf09896be9a627cd04dd54ef6875bed88daa3d8 |
12-Sep-2015 |
Alex Deymo <deymo@google.com> |
Include the major version in update_engine.conf
This patch moves all the payload definition version numbers to
payload_constants.h and exposes the supported one in the .conf file.
Bug: 23946683
Test: Added unittest to match the .conf file with the code.
Change-Id: I7d84d2aa3c85d2b4d5da8bf102aa9bf99acc0136
ull_update_generator_unittest.cc
enerate_delta_main.cc
ayload_generation_config.cc
|
| b9ef491e87c17d809368ede0196569d8a106b963 |
22-Sep-2015 |
Sen Jiang <senj@google.com> |
Move partition info to PartitionUpdate in payload verison 2.
old_kernel_info, old_rootfs_info is now old_partition_info.
new_kernel_info, new_rootfs_info is now new_partition_info.
AddPartitionOperations is renamed to AddPartition because it now adds
partition info too.
Bug: 23694580
TEST=cros_workon_make update_engine --test
Generated a version 2 payload with delta_generator.
Change-Id: I07c7fc21198307e935679e62861cdac4b406c84f
elta_diff_generator.cc
ayload_file.cc
ayload_file.h
ayload_file_unittest.cc
|
| 625406cee9a90ac2ed895f480286b7f0e8497f38 |
17-Sep-2015 |
Sen Jiang <senj@google.com> |
payload_generator: Remove PartitionName enum.
payload_generator should be able to generate payload for any partition name
in major version 2, so we should use a string for partition name.
Bug: 23620828
TEST=cros_workon_make update_engine --test
Change-Id: Iee92af5d82e74b9be6ae793850a4ae6c31e008c5
b_generator.cc
elta_diff_utils_unittest.cc
ull_update_generator.cc
ull_update_generator_unittest.cc
nplace_generator.cc
nplace_generator_unittest.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| 70a6ab091beb261a501012bc09955de452dd4736 |
28-Aug-2015 |
Sen Jiang <senj@google.com> |
Move install operations to partitions field in major version 2.
install_operations and kernel_install_operations will be empty in major
version 2, they are now present in partitions field.
partition_info are still in old location, more refactoring are needed before
moving them.
This patch also moves the kLegancyPartitionName constants from install_plan.h
to payload_constants.h.
TEST=cros_generate_update_payload generate same payload version 1 as before.
Call delta_generator manually with --major_version=2, then cros payload show.
cros_workon_make update_engine --test
Bug: 23694580
Change-Id: If9e9531656480b1174e13b39af7ecb6a81060aac
elta_diff_generator.cc
ayload_file.cc
ayload_file.h
ayload_file_unittest.cc
|
| 7d9bd921a8f92338ca44a6ee368c58450da594df |
14-Sep-2015 |
Alex Deymo <deymo@google.com> |
Include missing REPLACE_XZ switch case.
The operation was included in the .proto file but not in this switch
case.
Bug: 23179128
Test: emerge-link update_engine
Change-Id: I99887a1dbcf6237d268bcd0c117d3766f113df64
nnotated_operation.cc
|
| ef366e126180b537e085e31f1e514741f021a054 |
11-Sep-2015 |
Alex Deymo <deymo@google.com> |
delta_generator: Accept --metadata_signature_file flag.
The flag currently doesn't actually include the signature in the
payload, but need the binary to accept the flag before we update
the caller script.
Bug: 23981164
Test: `delta_generator --metadata_signature_file=foo` doesn't complain
Change-Id: I81dd4b17be4f66482bc041a0310c8ef13e9bb566
enerate_delta_main.cc
|
| 262c7be2e5ca3ea6d0a814200392895fd3faa931 |
09-Sep-2015 |
Alex Deymo <deymo@google.com> |
Remove Chrome OS specific verity_utils module.
The size of the updated region in the partition should match the size
the verified boot chain requires, or at least be bigger. In Chrome OS,
the kernel partition includes a command line with the size of the
rootfs verity will verify, but the rootfs partition is significantly
bigger (about 800 MB more) so we only update what is required.
In the more general Brillo case, the size of the updated area depends
on how verity is used. This patch removes this logic from the
delta_generator and uses the size of the filesystem, which should
match the size of the verity region in Chrome OS devices using ext2.
Bug: 23084776
Test: sudo emerge_update; `mma` on AOSP
Change-Id: I37f341263d86f127d937b4afc24e9d085131cb08
enerate_delta_main.cc
ayload_generation_config.cc
ayload_generation_config.h
erity_utils.cc
erity_utils.h
erity_utils_unittest.cc
|
| 6f20dd4fc8861d93d188cd27323d2f9746464aaf |
19-Aug-2015 |
Alex Deymo <deymo@chromium.org> |
Fix includes and DBus headers to build on Android.
This patch include a set of small changes to includes and names in
order to be compatible with the Android build system.
First, the DBus .xml file needs to be renamed to .dbus-xml and the
output directory must match the soruce directory, since the headers are
generated with automatic Makefile rules.
Some headers required by bionic were missing, such as endian.h and
xattr.h. In the xattr.h case, we include the header from the glibc
(or bionic) instead of libattr, since libattr is not available in
AOSP in the first place, and we are not linking against it Chrome OS,
so we shold include the right headers.
Bug: 23084776
TEST=`FEATURES=test emerge-link update_engine` works; `mma` in AOSP doesn't have these errors anymore (but still doesn't build).
Change-Id: Ifeedfe9d894ca78baa03940ac55563cfcd464e76
ayload_file.cc
ayload_signer.cc
|
| 46e9b1735a16518d563b3c7e7227408f8fc3e891 |
31-Aug-2015 |
Sen Jiang <senj@google.com> |
Add major_version flag in delta_generator.
Only affects major version number in the payload for now, actual implementation
of major version 2 will follow.
TEST=Call delta_generator manually with --major_version=2 then cros payload
show.
Bug: 23694580
Change-Id: I5baca5a868ec00dc2f6ea63d89e295d43283fd32
enerate_delta_main.cc
ayload_file.cc
ayload_file.h
ayload_generation_config.h
|
| ebdf17d4202c67933764135bfc1cece629829201 |
19-Aug-2015 |
Sen Jiang <senj@google.com> |
Generate update operations per partition
In order to support multiple partitions, OperationsGenerator should take one
partition at a time, and be called multiple times. This also reduce code
repetition.
TEST=FEATURES=test emerge-peppy update_engine; cros_generate_update_payload
generate same payload as before.
BUG: 23037923
Change-Id: Iefbe87401217873e6dbdc616791667cfd1ec76a3
b_generator.cc
b_generator.h
elta_diff_generator.cc
ull_update_generator.cc
ull_update_generator.h
ull_update_generator_unittest.cc
nplace_generator.cc
nplace_generator.h
perations_generator.h
|
| 5456c19a2115ad1f44357eb82fe75cb36039f4c8 |
20-Aug-2015 |
Sen Jiang <senj@google.com> |
Call SetOperationBlob in ChunkProcessor
BUG=None
TEST=FEATURES=test emerge-peppy update_engine
Change-Id: I0b946ce2ad945088f9f57cff4a13a1913e022769
nnotated_operation.cc
ull_update_generator.cc
|
| aea4c1cea20dda7ae7e85fc8924a2d784f70d806 |
20-Aug-2015 |
Alex Deymo <deymo@google.com> |
Re-license update_engine to Apache2
This patch automatically replaced the license on all text files from
Chromium OS (BSD style) to AOSP (Apache2), keeping the original year as
a reference.
The license header was added to .gyp and .gypi files, the NOTICE was
replaced with a copy of the Apache2 license and MODULE_LICENSE_* file
was updated.
BUG=b/23084294
TEST=grep 'Chromium OS Authors' doesn't find anything.
Change-Id: Ie5083750755f5180a8a785b24fe67dbf9195cd10
b_generator.cc
b_generator.h
b_generator_unittest.cc
nnotated_operation.cc
nnotated_operation.h
lob_file_writer.cc
lob_file_writer.h
lob_file_writer_unittest.cc
lock_mapping.cc
lock_mapping.h
lock_mapping_unittest.cc
ycle_breaker.cc
ycle_breaker.h
ycle_breaker_unittest.cc
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
xt2_filesystem.cc
xt2_filesystem.h
xt2_filesystem_unittest.cc
xtent_ranges.cc
xtent_ranges.h
xtent_ranges_unittest.cc
xtent_utils.cc
xtent_utils.h
xtent_utils_unittest.cc
ake_filesystem.cc
ake_filesystem.h
ilesystem_interface.h
ull_update_generator.cc
ull_update_generator.h
ull_update_generator_unittest.cc
enerate_delta_main.cc
raph_types.cc
raph_types.h
raph_utils.cc
raph_utils.h
raph_utils_unittest.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
perations_generator.h
ayload_file.cc
ayload_file.h
ayload_file_unittest.cc
ayload_generation_config.cc
ayload_generation_config.h
ayload_signer.cc
ayload_signer.h
ayload_signer_unittest.cc
aw_filesystem.cc
aw_filesystem.h
arjan.cc
arjan.h
arjan_unittest.cc
opological_sort.cc
opological_sort.h
opological_sort_unittest.cc
erity_utils.cc
erity_utils.h
erity_utils_unittest.cc
|
| ac6246ae239518a27ab4e89ee01ba1b1d76d440f |
13-Aug-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Extend payload metadata with Brillo fields.
In order to update more than two partitions, we moved the partition
update description to a new message PartitionUpdate.
The InstallOperation message is extended with a per-operation source
data hash to validate the source data right before we apply the
operation instead of relying on the slow whole-partition hash.
This patch also includes two new operation types: ZERO and DISCARD to
be included starting with delta minor_version=3 and full payloads in
Brillo. It doesn't include the implementation for those yet.
BUG=b:23179128
TEST=Unittest still pass.
Change-Id: I5b5fc8e2af6684ce655bf1fa1b82d1d2285a4e3a
Reviewed-on: https://chromium-review.googlesource.com/293505
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
nnotated_operation.cc
|
| a12ee11c78ac6d7c2605921a4006b6a7416e0c35 |
13-Aug-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Move InstallOperation to the top level.
The InstallOperation message in the protobuf is a nested message
inside the DeltaArchiveManifest message, making all references to
operation types be very long names like
DeltaArchiveManifest_InstallOperation_Type_REPLACE_BZ while most other
messages are not nested in the DeltaArchiveManifest message.
To improve readability and to prepare for future update metadata
changes, this patch moves the InstallOperation message to the top level
and replaces all references to operation types with the new shorter
version like InstallOperation::REPLACE_BZ.
This change only impacts the scope of the generated classes and the
serialized format of the protobuf. This exact same question was
addressed by protobuf maintainers here:
https://groups.google.com/forum/#!topic/protobuf/azWAPa6hP4A
Finally coding style and indentation was automatically updated due to
the shorter names.
BUG=b:23179128
TEST=Unittest still pass.
Change-Id: I55add54265934cd1fd3e9cb786c5d3f784902d17
Reviewed-on: https://chromium-review.googlesource.com/293504
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
b_generator.cc
b_generator_unittest.cc
nnotated_operation.cc
nnotated_operation.h
ycle_breaker.cc
ycle_breaker_unittest.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
xtent_utils_unittest.cc
ull_update_generator.cc
ull_update_generator_unittest.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
ayload_file.cc
ayload_file.h
|
| 0b3db6b6040f53eb9859e614f7fe4c681213d33a |
11-Aug-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Remove all references to glib.
The message loop in production code switched from
chromeos::GlibMessageLoop to chromeos::BaseMessageLoop used in
chromeos::Dameon. This patch moves all the unittests also to use
chromeos::BaseMessageLoop or chromeos::FakeMessageLoop when I/O is not
required.
This patch removes all (now unused) references to glib.
BUG=chromium:499886
TEST=`grep glib[^c] . -R` return no results. Also, unittests work.
Change-Id: Iddfb17def501829d932ed1f70faad3e00268898e
Reviewed-on: https://chromium-review.googlesource.com/292157
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
enerate_delta_main.cc
|
| 8cc502dacbccdab96824d42287f230ce04004784 |
10-Aug-2015 |
Sen Jiang <senj@chromium.org> |
update_engine: Change OperationsGenerator to use BlobFileWriter
BUG=chromium:517280
TEST=Unit test for BlobFileWriter
Change-Id: Ib49925676331acee97ff6b4cec38a81ca8b157a1
Reviewed-on: https://chromium-review.googlesource.com/291441
Tested-by: Sen Jiang <senj@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Sen Jiang <senj@chromium.org>
b_generator.cc
b_generator.h
b_generator_unittest.cc
nnotated_operation.cc
nnotated_operation.h
lob_file_writer.cc
lob_file_writer.h
lob_file_writer_unittest.cc
elta_diff_generator.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
ull_update_generator.cc
ull_update_generator.h
ull_update_generator_unittest.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
perations_generator.h
|
| 461b259af8815d782200782c5ba3599d8de4a66c |
25-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Use libchromeos to launch subprocesses.
The Subprocess class handles the execution of suprocesses in the
update_engine such as the post-install script and bspatch operations.
This patch migrates this class from using glib functions to use
libchromeos classes with equivalent functionality.
Callsites and unittests were updated to match the new interface.
BUG=chromium:499886
TEST=Unittest still pass. Deployed on link and cros flash another image
using a delta payload.
Change-Id: Ia64d39734e220675113f393a6049e9a9b0fe8409
Reviewed-on: https://chromium-review.googlesource.com/288837
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
enerate_delta_main.cc
|
| 7fad7b7d3d7dcdaa7b17855fe333fcc4da000b46 |
22-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Replace gthread with libchrome equivalents.
This patch replaces the gthread usage in the multi-threaded full
payload generator with equivalent functions from libchrome. In the
new code, a thread-pool is used instead of creating one thread per
work and waiting for the threads to finish.
BUG=chromium:499886
TEST=unittests still pass.
Change-Id: I00fd56de2a789b5f007f1a4ab88680809bbeb5f0
Reviewed-on: https://chromium-review.googlesource.com/287635
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
ull_update_generator.cc
ull_update_generator.h
enerate_delta_main.cc
|
| cfe8c15afc6c725a9626577db6504675310498d8 |
18-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Set a sane chunk_size default.
When generating deltas of normal ChromeOS images, the expected maximum
size of a file is about 120 MiB, which is reasonble to handle in
memory during payload generation and application. Nevertheless, if we
find a very big file (or pseudo-file) we could easilly run out of
memory when trying to produce a BSDIFF for it. This patch limits
the chunk_size to 200 MiB, which should not impact any normal
ChromeOS payload.
BUG=chromium:485397
TEST=cros_generate_update_payload produces the same payload.
Change-Id: I1c43e619f424253cab20f0d384c7e90d10bd707d
Reviewed-on: https://chromium-review.googlesource.com/286535
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
enerate_delta_main.cc
|
| 2d3b2d635e50c6886e285afb86c3187d9e0bd360 |
18-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Implement soft/hard chunk size limits.
The current approach uses a single chunk_size value, that defaults to
-1 (or "whole files") for delta payloads. The "whole files" default is
the preferred value since BSDIFF and SOURCE_BSDIFF operations (the most
common operations in real delta payloads) will benefit from having
access to the whole file in the old and new partition, in particular in
cases like the chrome binary (~120 MiB in size).
On the other hand, very big chunks have a big memory footprint in most
cases. Current implementations of BSDIFF, REPLACE_BZ and REPLACE will
require as much private memory as the destination chunk_size or more.
Because of this and due to the lack of old data, a small chunk_size
(1 MiB) is used for full payloads.
To break this tension between having a big chunk_size for operations
that will benefit from it versus having a small chunk_size for cases
where it doesn't change anything, this patch introduces two chunk
size values: a hard and soft limit.
The hard_chunk_size has the same meaning as the old chunk_size: no
operation should have a destination bigger than the provided hard
limit. The soft_chunk_size is the preferred chunk size for an
operation when a bigger chunk will not benefit significantly the
final payload size. Having a small chunk size for operations like
REPLACE_BZ, REPLACE, SOURCE_COPY and MOVE is important to keep the
memory footprint low when it the extra memory is not required.
The new soft_chunk_size limit is used when merging operations
(previously hard-coded to 1 MiB) and when generating new
operations for zeroed and moved blocks.
BUG=chromium:485397
TEST=Run cros_generate_update_payload for full and delta payloads.
Inspected the chunk size in the resulting payloads.
Change-Id: I370048a81913ad23a151cfef6690627b7fff7277
Reviewed-on: https://chromium-review.googlesource.com/286568
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
b_generator.cc
b_generator.h
b_generator_unittest.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
ull_update_generator.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
nplace_generator.cc
nplace_generator.h
ayload_generation_config.cc
ayload_generation_config.h
|
| 56a088eded3be1fde86ae471c583483881c0a013 |
20-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Remove deprecated flags.
These flags are not passed from cros_generate_update_payload anymore.
BUG=chromium:305832
TEST=`cros_generate_update_payload --old_image $FOO --image $BAR` still
works.
Change-Id: I6fb009de677398b28ed09146d5f7c9877efd4210
Reviewed-on: https://chromium-review.googlesource.com/286890
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
enerate_delta_main.cc
|
| 896fdbaa1cc9a422ac36ef455d02c3cef1f27816 |
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Prevent the InplaceGenerator to add MOVEs to block 0.
While we normally don't generate MOVE operations to/from the block 0,
the cycle breaker logic can use the block 0 as scratch space, creating
MOVE operations to and from the block 0.
This patch prevents it from picking the block 0 as scratch space and
logs a message when that would happend.
BUG=chromium:480751,chromium:500423
TEST=Added unittest.
Change-Id: I91f1b3c426a9d06aae5b685e2e901c7e448d8677
Reviewed-on: https://chromium-review.googlesource.com/286623
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
b_generator.cc
elta_diff_utils.cc
elta_diff_utils.h
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| 3b2c7d0e6d859e6fc75c82b3417f87cf5968a49d |
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Sort full operations by destination.
The inplace generator moves all the full operations to the end of the
list. This patch sorts those operations by the destination.
This patch also cleans up the Vertex class by using the new
AnnotatedOperation class instead of the operation and file_name
separated.
BUG=None
TEST=Unittest still pass. minor_version=1 full operations appear in
order according to "cros payload".
Change-Id: Ia0c15939086cec52dc855cbc3afa913f8cbebf6b
Reviewed-on: https://chromium-review.googlesource.com/286213
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
b_generator.cc
ycle_breaker.cc
ycle_breaker_unittest.cc
elta_diff_utils.cc
elta_diff_utils.h
raph_types.h
raph_utils.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
|
| f6165357bfbb74b22a4da5780a474f439611e167 |
17-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Resolve read-after-write dependencies in the kernel.
BUG=chromium:510909
TEST=Ran delta_generator on rambi from 6415.1.0 to 7272.0.0 dev-channel.
Resulting kernel operations are in order.
Change-Id: I8843c1497b095af7520404d9845db9037bb5e262
Reviewed-on: https://chromium-review.googlesource.com/286212
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
nplace_generator.cc
nplace_generator.h
|
| f0061358b5f741baeeb9177b838b289d2ce318f3 |
01-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Handle moved and zero blocks in payload generation.
Delta generation uses only the file name to detect when files changed.
Therefore, renaming a file doesn't get detected and the new name is
assumed to be a new file. On the other hand, free-space blocks are
often just zeros so they can be easilly encoded independently from the
rest.
This patch detects blocks that moved and blocks with zeros and handles
those blocks beforehand regardless of where are they in the filesystem
(file data, metadata, free space, etc). For blocks that moved,
SOURCE_COPY or MOVE operations are created as needed. For blocks with
zeros we use REPLACE_BZ operations.
Blocks processed in this way will be excluded from other files,
metadata or free space processing done. This solves perfomance issues
when trying to process with bsdiff files with very long runs of zeros.
CQ-DEPEND=CL:283643
BUG=chromium:504445
TEST=Unittest added. Run delta_generator between cid canary 7195 and
7201 and it doesn't take forever to finish.
Change-Id: I892fe7456608e83a2946133da335eb4fbd19a645
Reviewed-on: https://chromium-review.googlesource.com/283172
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
lock_mapping.cc
lock_mapping.h
lock_mapping_unittest.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
xtent_ranges.cc
xtent_ranges.h
xtent_ranges_unittest.cc
|
| b42b98db059a12c44110588fc0b3d5f82d32a2f8 |
07-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Load minor_version from the FilesystemInterface.
In the normal use case, cros_generate_update_payload doesn't pass the
requested minor_version to delta_generator which means we need to detect
it from the old image. To detect it, we where looking at the
update_engine.conf file in the mounted source directory. This patch uses
the recently added FilesystemInterface::LoadSettings() method to read
the update_engine.conf settings from the source rootfs partition.
In order to make this information available when autodetecting the
minor_version and to help future FilesystemInterface mocking for
testing, we move the FilesystemInterface instance from inside the
OperationsGenerator class to the PartitionConfig, so the filesystems are
opened before calling GenerateUpdatePayloadFile.
This patch then deprecates the --old_dir flag removing the requirement
to run delta_generator and cros_generate_update_payload as root when
generating deltas.
BUG=chromium:305832
TEST=Ran cros_generate_update_payload to generate deltas from old (link
FSI) and new (ToT link) images.
Change-Id: I915679d62aae2d1a2acee68cc8c1ec691cc363ad
Reviewed-on: https://chromium-review.googlesource.com/283643
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
b_generator.cc
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_utils.cc
elta_diff_utils.h
ull_update_generator.cc
enerate_delta_main.cc
nplace_generator.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| 2e9533be9eb2e022d653400f47a354a7f06bf9db |
27-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Expose the update_engine.conf in the FilesystemInterface.
The old image contains a /etc/update_engine.conf file that states the
payload version supported by that image. Currently we read this file
directly from the mounted filesystem and parse its contents. To stop
mounting the filesystem we need to retrieve the information on this file
somehow.
This patch extends the FilesystemInterface with a method to load the
update_engine.conf settings from the filesystem and implemets it using
ext2fs on ext2 filesystems.
CQ-DEPED=CL:282380
BUG=chromium:305832
TEST=Added new unittests with and without this file.
Change-Id: I41b41e8aac58c645fb40aabfe340cde8821e405a
Reviewed-on: https://chromium-review.googlesource.com/282381
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
xt2_filesystem.cc
xt2_filesystem.h
xt2_filesystem_unittest.cc
ake_filesystem.cc
ake_filesystem.h
ilesystem_interface.h
aw_filesystem.h
|
| 14158570d3995008dc93a628004118b87a6bca01 |
13-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Split delta_diff_generator file.
The DeltaDiffGenerator class includes both an OperationsGenerator using the
A-to-B operations and a set of common methods used also by the inplace generator.
The delta_diff_generator.{h,cc} files also include a single function to generate
the payload (GenerateUpdatePayloadFile) that centralizes the logic of generating
the operations and writing the payload.
This patch splits these three parts in different files. The common delta diff
function are moved to the delta_diff_utils.{h,cc} files. The operations generator
class that uses A-to-B operations is now in a new ab_generator.{h,cc} pair of files
that implement the ABGenerator() class. Finally, the payload file writing methods
are now in a single PayloadFile class.
This allow us to create payload files without the need to generate images and
their deltas. This will be used in a follow up CL to remove the image generation
logic from the unittests.
BUG=chromium:351589
TEST=Ran unittests. Regenerate a payload with and without this patch; got the same results.
Change-Id: I6816d2c805ba8c0c5c9423c720131a100a15ebaa
Reviewed-on: https://chromium-review.googlesource.com/280838
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
b_generator.cc
b_generator.h
b_generator_unittest.cc
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
elta_diff_utils.cc
elta_diff_utils.h
elta_diff_utils_unittest.cc
xtent_utils.cc
xtent_utils.h
xtent_utils_unittest.cc
enerate_delta_main.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
ayload_file.cc
ayload_file.h
ayload_file_unittest.cc
ayload_generation_config.h
ayload_signer.cc
|
| 6c396a9b3020df9b0b58886fd6f36523bec29d3a |
01-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Fix full payload generation when using chunk_size.
We already check that the partition size and the chunk_size are multiples
of the block_size, but we assume that the partition size is a multiple of the
chunk_size and that was mostly always the case. This patch fixes that by
setting the right operation size for the last rootfs and kernel operation.
BUG=chromium:485397
TEST=Added unit test to check this case.
Change-Id: Ia025d299c07bbe0732cf8554b75f4d40b6e063f9
Reviewed-on: https://chromium-review.googlesource.com/282758
Reviewed-by: Gabe Black <gabeblack@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
ull_update_generator.cc
ull_update_generator_unittest.cc
|
| 8e66534e4c76de1b2882ef637cdb8f3c361cb48f |
01-Jul-2015 |
Alex Deymo <deymo@chromium.org> |
Revert "update_engine: Change the default chunk_size to 200MB"
The chunk_size handling in payload_generator has some problems. This commit break the full payload generation due to some bugs when the chunk_size is not a divisor of the partition size, breaking paygen in canaries.
This patch is not required for chromium:504447 since the bzip2 bug was already fixed.
This reverts commit def5fe30d2c5f38915f4e5986bdc54661f20ec3e.
BUG=chromium:504447
TEST=cros_generate_update_payload for a full payload doesn't generate a 200MiB kernel image.
Change-Id: I505afd84ca10c7256b66cd84f288c6db03a4d88e
Reviewed-on: https://chromium-review.googlesource.com/282767
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
enerate_delta_main.cc
|
| 2a9cb2af7c21b93f380d8ce28d932acdf6e25df7 |
27-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Remove unused ext2_utils.h
This file is not used anymore.
BUG=None
TEST=FEATURES=test emerge-link update_engine.
Change-Id: I9fce8b912c7126f137db01060351d9278a7b43aa
Reviewed-on: https://chromium-review.googlesource.com/282359
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
xt2_utils.h
|
| 2b6b1db46c3e8193c162d74f704cf789d0db6272 |
25-Jun-2015 |
Kevin Cernekee <cernekee@chromium.org> |
update_engine: Change the default chunk_size to 200MB
cros_generate_update_payload can fail generating deltas for images
containing very large (2GB) files. To work around this, we can set a
smaller default chunk size.
On standard Chrome OS builds, the largest single file is the Chrome
browser binary, weighing in at 130MB. So this change should not affect
those builds at all.
BUG=chromium:504447
TEST=trybot run
Change-Id: I5469d1cb70fd4f241a72e6a0c8507d289016c6fe
Signed-off-by: Kevin Cernekee <cernekee@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/281827
Reviewed-by: Alex Deymo <deymo@chromium.org>
enerate_delta_main.cc
|
| 249fbf5be69e8de37fbac33c1570a58f902b1f6f |
16-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Avoid emitting a MOVE to 0.
Workaround the bug where we can't emit a MOVE to the block 0 by excluding
it from the inplace generator graph and re-adding the operation later.
BUG=chromium:500423
TEST=delta_generator from lulu-R43-6946.31.0 to R45-7174.0.0, test image.
Change-Id: I3f16c2ae58b0f31869a39af6a45f7798960c722b
Reviewed-on: https://chromium-review.googlesource.com/278020
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Trybot-Ready: Filipe Brandenburger <filbranden@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
nplace_generator.cc
|
| a50870549b0fc16b659250449fffbd3eacc57d8b |
12-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Fix memory leak on new ext2_filesystem code.
ext2fs_get_pathname() allocates new memory to store the pathname and we
are supposed to free it. This patch adds the missing call to
ext2fs_free_mem().
BUG=None
TEST=`FEATURES=test USE="clang asan" VERBOSE=1 GTEST_FILTER="Ext2*" emerge-link update_engine` works now and didn't work before
Change-Id: I5244e7f9721875590c3b2455d469866d28c70c09
Reviewed-on: https://chromium-review.googlesource.com/277277
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Daisuke Nojiri <dnojiri@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
xt2_filesystem.cc
|
| bfeeb2e85b7881ec16badb0e104c600639102ce2 |
12-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Fix temp file leak in unittest.
This patch removes temp files when the test ends using the
ScopedPathUnlinker.
BUG=None
TEST=Ran unittest; ls /tmp doesn't show these files.
Change-Id: I41d0db07b5f1f08f816a116c5ece5accb9030291
Reviewed-on: https://chromium-review.googlesource.com/277088
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
elta_diff_generator_unittest.cc
|
| 1beda780333ce51d7872603b70712772eb2383fb |
07-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Remove unused IsIdempotentOperation().
All the minor-version=2 are idempotent, so since we switched to minor
version 2, this function becomes trivial now. This patch removes this
function from the code and moves the ExtentRanges class to the
payload_generator/ directory since now it is only used there.
BUG=None
TEST=Unittest still pass.
Change-Id: Ib9dbbdded0ca2ef2128bb6c470de7a00720c4038
Reviewed-on: https://chromium-review.googlesource.com/275806
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
elta_diff_generator_unittest.cc
xt2_filesystem.cc
xtent_ranges.cc
xtent_ranges.h
xtent_ranges_unittest.cc
xtent_utils.cc
xtent_utils_unittest.cc
raph_utils_unittest.cc
nplace_generator.cc
nplace_generator_unittest.cc
aw_filesystem.cc
|
| 35589c2b9a0e20b42661b132890128d8025c1954 |
07-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Remove rootfs_mountpt from ImageConfig.
The mounted rootfs is not used anymore during opeartions generation so
this patch removes the setting from the ImageConfig. The --new_dir flag
is now deprecated because of this.
The --old_dir is still being used when no --minor_version is passed, to
autodetect the minor version of the old image.
With this patch, the rootfs and kernel partitions have a more generic
description in the config.
BUG=None
TEST=Unittest still pass.
Change-Id: I0398493a44f8c09159a93a716f2168f45e8f8df9
Reviewed-on: https://chromium-review.googlesource.com/275805
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
ull_update_generator.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
nplace_generator.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| 6b9e38ef1180efe55e4a82bb18536d1b53fe493d |
05-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Use the FilesystemInterface to generate deltas.
This patch removes the FilesystemIterator used to iterate the file data
blocks, the metadata.{cc,h} files used to iterate the metadata blocks and
the ReadUnwrittenBlocks() method used to process the rest of the blocks.
Instead, these three cases are handled by the same DeltaReadFilesystem()
method using the FilesystemInterface abstraction for ext2.
The sub-file block handling was changed from (filename, offset, size) to
just the list of blocks (or Extents) and the parsing of the filesystem
was removed from the methods that produce operations. The filename is
kept just as a label for logging purposes.
This patch implies that the filesystem doesn't need to be mounted in
order to generate the payload, as all the data is accessed as blocks
in a partition. The mountpoint was removed from all the operations
generators, but it is still required to mount the old filesystem to
detect the minor version supported by the old update_engine. Also,
since the list of blocks is never accessed using the FIGETBSZ ioctl,
no payload_generator/ test require root privileges, nor the
delta_generator binary.
BUG=chromium:305832,chromium:331965
TEST=Updated unittest to use extents.
Change-Id: Ia2ea9433190258f70e3bce09f896b18326a7abf9
Reviewed-on: https://chromium-review.googlesource.com/275804
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
nnotated_operation.cc
nnotated_operation.h
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
xtent_mapper.cc
xtent_mapper.h
xtent_mapper_unittest.cc
ilesystem_iterator.cc
ilesystem_iterator.h
ilesystem_iterator_unittest.cc
raph_types.h
raph_utils.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
etadata.cc
etadata.h
etadata_unittest.cc
|
| 2b19cfbcdb1aa8c5d1f338d19312fe14b6734bd5 |
26-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Introduce FilesystemInterface abstraction.
The interaction with the filesystem in the payload generation process
is hard-coded in several places, making it hard to mock out or use a
different filesystem like squashfs for delta generation. For example,
the metadata, regular file data and non-file data are handled by three
different functions in a similar way, but with different code.
This patch introcudes a filesystem abstraction to map files or
pseudo-files (like the metadata, free-space, etc) into the same interface.
The interface includes three implementations: for parsing ext2 filesystems
using ext2fs (already used by the metadata parsing but not by the file
data processing), a raw one for monolitic partitions like the kernel
and a fake one used for testing without requiring to build/parse a real
ext2 filesystem.
BUG=chromium:331965
TEST=FEATURES=test emerge-link update_engine
Change-Id: I1e14cf8f3883c8e9a1d471c8193c8da60776aa7c
Reviewed-on: https://chromium-review.googlesource.com/275803
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
xt2_filesystem.cc
xt2_filesystem.h
xt2_filesystem_unittest.cc
ake_filesystem.cc
ake_filesystem.h
ilesystem_interface.h
aw_filesystem.cc
aw_filesystem.h
|
| 52490e776a9d34a94fb18ab1fe7d416425e94dda |
04-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Further move Extent manipulation to extent_utils.
This patch moves more Extent manipulation functions to extent_utils.
It moves NormalizeExtents() and creates a new ExtentsSublist() function
that will be used by a follow up CL.
BUG=None
TEST=Added unittests.
Change-Id: Icf0ef0af208aa45c9f44e1a73662b3efd8bbbb45
Reviewed-on: https://chromium-review.googlesource.com/275801
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
xtent_utils.cc
xtent_utils.h
xtent_utils_unittest.cc
|
| 2350445d4e713652ae65ee2783a191e3e68f6cba |
04-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Remove duplicated function.
BlocksInExtents() is now exposed in extent_utils.cc
BUG=None
TEST=Unittest still pass.
Change-Id: Ie4cab3adee1c5ec232b229e9c73ef4a311cf4fb6
Reviewed-on: https://chromium-review.googlesource.com/275800
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
elta_diff_generator_unittest.cc
|
| 5c6c65570013bbdbd67f9bf6391dd295ef5b5ee6 |
03-Jun-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Split Extent utils from graph_utils.
"Graph" related utils should only concern parts of the code using the
inplace generator, since other generators don't use a dependency graph.
This patch splits the Extent related utils from the graph related ones
creating a new extent_utils.h file.
BUG=None
TEST=unittest still pass.
Change-Id: I0941698b0a47a6cc222e8dc062fc54eb3cdf4de2
Reviewed-on: https://chromium-review.googlesource.com/274899
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
xtent_mapper.cc
xtent_utils.cc
xtent_utils.h
xtent_utils_unittest.cc
raph_utils.cc
raph_utils.h
raph_utils_unittest.cc
nplace_generator.cc
nplace_generator_unittest.cc
etadata.cc
|
| 46bd4e20c20d6cb1a2f18d39237b9345d576669e |
14-May-2015 |
Gilad Arnold <garnold@chromium.org> |
update_engine: Fix splitting and merging of operation.
As previously implemented, these features retained the original
operation type, thus potentially violating an invariant of the delta
format by which an operation must have the minimum possible size. This
could take form in either direction; in one case, we've seen this fail
payload verification when a large REPLACE_BZ was split into small
operations and the compressed blob of one was larger than the size of
the written blocks (i.e. compression was disadvantageous).
Instead, when splitting and merging operations we now choose an optimal
operation type among REPLACE and REPLACE_BZ. This means that every
split/merged data blob is compressed and its size compared to its
uncompressed form, storing the smaller of the two and setting the
operation type accordingly.
This also refactors unit tests and adds more test cases to ensure that
operation types are chosen optimally and that the conversion is correct
in both directions.
Finally, this re-enables the splitting of REPLACE_BZ operations, which
was disabled (CL:270581) to prevent payload breakages.
BUG=chromium:487595
TEST=Unit tests
Change-Id: Ib348b76a96eba6dd612cafa412e1231c43389f96
Reviewed-on: https://chromium-review.googlesource.com/271277
Trybot-Ready: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: David Zeuthen <zeuthen@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
|
| 14c0f1b1e2ada315b3ee63084435b899e51c5bc5 |
13-May-2015 |
Gilad Arnold <garnold@chromium.org> |
update_engine: Disable splitting of fragmented REPLACE_BZ operations.
The said feature was introduced in CL:269130 and violated an invariant
of the delta payload format. This leaves most of the logic in place but
disables invoking it during delta generation. It will be re-enabled once
the logic is fixed.
BUG=chromium:487436
TEST=Unit tests
Change-Id: I59bf79749f0548d2935aecf489849ac2f9cb1f1e
Reviewed-on: https://chromium-review.googlesource.com/270581
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
elta_diff_generator.cc
|
| 41e3474eebe3748a45aba4b297a7a313a1006c6c |
11-May-2015 |
Gilad Arnold <garnold@chromium.org> |
update_engine: Merge contiguous operations (reprise).
This is reapplying CL:269604 with a small fix that prevents it from
adding spurious src_length fields when merging payload operations. This
also revises relevant unit tests to catch the said error. The original
change is described below.
Merges operations of type SOURCE_COPY, REPLACE, or REPLACE_BZ if they
have the same type, are contiguous, have only one destination extent,
and their combined block count does not exceed the chunk size.
BUG=chromium:461651
BUG=chromium:486497
TEST=Unit test (fails before fix, passes after)
TEST=Trybot
Change-Id: I9f3d7f653454e27177428f2d7c0e6485184a7f8b
Reviewed-on: https://chromium-review.googlesource.com/270268
Trybot-Ready: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
nnotated_operation.cc
nnotated_operation.h
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
|
| 542032d92433045a66e888588d133a47c7fcf956 |
11-May-2015 |
Gilad Arnold <garnold@chromium.org> |
Revert "update_engine: Merge contiguous operations."
This reverts commit 6a3ea94766bb4972abbd629779914d80dada1245.
The CL is suspected to close the tree due to paygen failures.
See comments inline (delta_diff_generator.cc) for the culprit.
BUG=chromium:486497
TEST=trybot
Change-Id: Iaed694a79371e7de208b0de90f8e9cefd61dd7e6
Reviewed-on: https://chromium-review.googlesource.com/270183
Trybot-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Cheng-Yi Chiang <cychiang@chromium.org>
Commit-Queue: Cheng-Yi Chiang <cychiang@chromium.org>
Tested-by: Cheng-Yi Chiang <cychiang@chromium.org>
nnotated_operation.cc
nnotated_operation.h
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
|
| c31e4a742b56c122b50febc1f78fd1c3772ab7fd |
07-May-2015 |
Mike Frysinger <vapier@chromium.org> |
update_engine: drop SYSROOT checks
Since unittests are run inside of the sysroot now, the specific checks
for them in the tests are pointless. Delete all the related logic.
BUG=chromium:401872
TEST=precq passes (runs unittests)
Change-Id: I12cb5e6717718a35c0dda2510e63c7bbcfa1f470
Reviewed-on: https://chromium-review.googlesource.com/269643
Trybot-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
ayload_signer.cc
|
| 58455ae7fe87a312ae648d871a92e1485d0e9989 |
04-May-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Merge contiguous operations.
Merges operations of type SOURCE_COPY, REPLACE, or REPLACE_BZ if they
have the same type, are contiguous, have only one destination extent,
and their combined block count does not exceed the chunk size.
BUG=chromium:461651
TEST=`cros flash --src-image-to-delta`, checking generated payloads
using `cros payload show --list_ops`, unit tests.
CQ-DEPEND=CL:269134
Change-Id: I5f71e6828561f2e0a6a8ab3e3aae4c362ea2ddde
Reviewed-on: https://chromium-review.googlesource.com/269604
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
nnotated_operation.cc
nnotated_operation.h
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
|
| c4797c4c84bd7b25c83c841d65d2599a587c8c10 |
08-May-2015 |
Gilad Arnold <garnold@chromium.org> |
update_engine: Delta generator should not read past old image size.
This fixes a problem that was introduced by CL:246670 and caused the
delta generator to fail when generating deltas from a source image
that's smaller than the target image.
The bug is fixed by ensuring that the reading and comparing of unused
blocks is not attempted for block indexes that exceed the size of the
old image. Instead, these blocks are assumed to be new and their content
packaged into the payload.
BUG=chromium:477876
TEST=Unit tests
TEST=delta_generator successfully applied to FSI version 2913.331.0
Change-Id: Icc3163cca25ebeaefe05c2ba852664b6f0504053
Reviewed-on: https://chromium-review.googlesource.com/270075
Trybot-Ready: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
nplace_generator.cc
|
| db1d5a6204ead78024bcfc854cbf084cc53f1734 |
04-May-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Sort operations by output block.
After fragmenting SOURCE_COPY, REPLACE, and REPLACE_BZ operations,
payload generator will sort the operations by the start block of their
first destination extents. This reduces seeking when writing during
payload application.
BUG=chromium:461649
TEST=`cros flash --src-image-to-delta`, checking generated payloads
using `cros payload show --list_ops`.
CQ-DEPEND=CL:269130
Change-Id: I1e2d8e940cd0ce66a1ec62711653ba8b34b0f0a5
Reviewed-on: https://chromium-review.googlesource.com/269134
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
|
| 0d73e03afb30dd402255fd030293b145de492af6 |
02-May-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Split fragmented REPLACE_BZ ops.
Splits REPLACE_BZ operations with multiple destination extents into
REPLACE_BZ operations with only one destination extent each.
BUG=chromium:461643
TEST=`cros flash --src-image-to-delta` and unit tests.
CQ-DEPEND=CL:268520
Change-Id: Ia5c055047c4bb863871507f32d028e4e28cb6af6
Reviewed-on: https://chromium-review.googlesource.com/269130
Reviewed-by: Allie Wood <alliewood@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
|
| ba4c8b7d7b331be33604bb9e417f1108ba17a868 |
28-Apr-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Split fragmented REPLACE operations.
Splits REPLACE operations with multiple destination extents into REPLACE
operations with only one destination extent each.
BUG=chromium:461643
TEST=`cros flash --src-image-to-delta` and unit tests.
CQ-DEPEND=CL:267697
Change-Id: Id14ec9f23a462f94821454289857e9f25e756d24
Reviewed-on: https://chromium-review.googlesource.com/268520
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
|
| b6656a757e8379b63597c1417a8062c5cf416651 |
14-Apr-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Split fragmented SOURCE_COPY ops.
Splits SOURCE_COPY operations with multiple destination extents into
individual SOURCE_COPY operations with only one destination extent each.
Also normalizes the extent lists before splitting the operation.
BUG=chromium:461641
TEST=`cros flash --src-image-to-delta` and unit tests.
CQ-DEPEND=CL:267360
Change-Id: I4d4fbe36165d98df7f36e4a1ddacc9b62ee97002
Reviewed-on: https://chromium-review.googlesource.com/267697
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
|
| fddb6d42dc7cd42e12fb543c99f1c116343700f3 |
18-Apr-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Fix metadata handling when inode table is in a different block group.
The inode table is stored in some metadata blocks and is local to each
block group in an ext2 filesystem. These tables are normally stored near
the beginning of the block group, together with the inode and block
bitmaps.
Appartenly, mkfs.ext2 can place the inode table for a given block group
outside the range of blocks of that block group. We have seen smaller
rootfs (400 MiB) where all the inode tables are grouped together near
the beginning of the filesystem, in the first block group.
This patch avoids an integer overflow when this happens and lets the
ReadUnwrittenBlocks() handle those tables.
BUG=chromium:478261
TEST=Ran ./delta_generator with a cosmos test image creating the diff to
itself.
Change-Id: Id610b03bc22a9ecb1c66d6fdc0548e2657c29f9b
Reviewed-on: https://chromium-review.googlesource.com/266229
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
etadata.cc
|
| 9f6f0a53509476be5f0ba152ec182d46a8d3c976 |
30-Mar-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Implement SOURCE_COPY/SOURCE_BSDIFF operations.
Implement SOURCE_COPY and SOURCE_BSDIFF in DeltaPerformer. These new
operations are the same as MOVE and BSDIFF, respectively, except
that they read from the source partition rather than the destination
partition.
Also fills in the source paths in omaha_response_handler_action, which
were previously not being set.
BUG=chromium:461632
TEST=`FEATURES=test emerge-link update_engine` and manual testing.
Change-Id: I81eba5780c73ed920a5db72702d9d7c67dbaa673
Reviewed-on: https://chromium-review.googlesource.com/263747
Reviewed-by: Allie Wood <alliewood@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
nplace_generator.cc
|
| 568734533c25a5783ea004aeb0da38244dcd3e5b |
28-Mar-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Remove sparse hole extents.
Whenever there is an extent beginning with a sparse hole, that extent
will be filtered out. Extents are now read in from the partition rather
than the file itself in ReadFileToDiff. src/dst_length are set to the
number of blocks in src/dst_extents * block_size.
This was tested manually with _GenerateSinglePayload to generate a delta
payload between test images.
BUG=chromium:469792,chromium:474497
TEST=`FEATURES=test emerge-link update_engine` and manual testing.
Change-Id: I384ec3f16f7fd9087159817358308f22d29e9edf
Reviewed-on: https://chromium-review.googlesource.com/264442
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
|
| 703022b71fc6a89796f2f97448b1a419007a52ca |
07-Apr-2015 |
Shawn N <shawnn@chromium.org> |
Revert "update_engine: Remove sparse hole extents."
Speculatively reverting commit 96b659d794be39762e8e7fd9f72fe6f723e21ec8 due to crbug.com/474497.
Change-Id: I17fd91c8568b30eafea9e70c9f2255ac2dc459a6
Reviewed-on: https://chromium-review.googlesource.com/264306
Reviewed-by: Shawn N <shawnn@chromium.org>
Commit-Queue: Shawn N <shawnn@chromium.org>
Tested-by: Shawn N <shawnn@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
|
| 8e2c4ef89b1c2c8779d003b40c70c75db1839bfb |
28-Mar-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Remove sparse hole extents.
Whenever there is an extent beginning with a sparse hole, that extent
will be filtered out. Extents are now read in from the partition rather
than the file itself in ReadFileToDiff.
BUG=chromium:469792
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: I615df120f6e46f66d500128561e0ddfcd407c13a
Reviewed-on: https://chromium-review.googlesource.com/262965
Reviewed-by: Allie Wood <alliewood@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
|
| b96ecdaafd27009a18e198f166d2b2f64a55d251 |
02-Apr-2015 |
Alex Vakulenko <avakulenko@chromium.org> |
platform2: Explicitly use base:: namespace with CommandLine
The current revision of libchrome is removing CommandLine class
from global namespace and only retain the one in base::.
Sweep the code to use the correct namespace in anticpation of
libchrome revision roll.
BUG=None
TEST=./build_packages
Change-Id: Ib7ca448dac1204c1e36ab053e775a5a5214997ac
Reviewed-on: https://chromium-review.googlesource.com/263785
Trybot-Ready: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
enerate_delta_main.cc
|
| 477aec2166a571cbe28081d806c3226e8b31b6e9 |
25-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Refactor OperationsGenerator into a base class.
This refactor cleans up the interface of the algorithms that generate
the list of rootfs and kernel operations removing the mention of a
Graph from it. The Graph class is only used by the in-place generator
because it requires to keep track of dependencies between operations
reading or writting the same block. The full update generator, using
only REPLACE or REPLACE_BZ doesn't need to use a graph to do that, but
in order to reuse some code, the interface was hacked that way.
This patch now uses two vectors of "AnnotatedOperations", which are
a mere InstallOperation as defined by the .proto file plus a name
used for logging purposes only. Both rootfs and kernel operations
have now the same type on the interface, allowing to share common
functions handling those.
BUG=chromium:331965
TEST=FEATURES=test emerge-link update_engine
Change-Id: I78566bbecb948634b7ecc8d086766ce67a79b43e
Reviewed-on: https://chromium-review.googlesource.com/262281
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
nnotated_operation.cc
nnotated_operation.h
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
ull_update_generator.cc
ull_update_generator.h
ull_update_generator_unittest.cc
enerate_delta_main.cc
raph_utils.cc
nplace_generator.cc
nplace_generator.h
perations_generator.h
|
| 08d5a45ac109292ff80c877c4045335a2f0b917d |
25-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Fake out GatherExtents in all tests.
DeltaDiffGenerator tests don't mock out GatherExtents(), requiring them
to run as root for no good reason.
This patch cleans up the test initialization and tear down to fake out
the GatherExtents() function making it return a fixed list of extents.
The extents are now generated based on the size of the files used during
the test instead of the actual positions of the temporary files in the
host filesystem.
BUG=chromium:351589
TEST=FEATURES=test emerge-link update_engine
Change-Id: I43d184722aa0d6b847977c3902330f0963232cc3
Reviewed-on: https://chromium-review.googlesource.com/262306
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Allie Wood <alliewood@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator_unittest.cc
|
| fdf005161523d9873a281f2a96d51dd4a3261f4f |
02-Mar-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Add fds for the source partitions.
Add new fds for the source partition, one for the rootfs and another for
the kernel. These are opened if we have a delta update with minor
version 2.
This change also adds support for changing the minor versions in tests.
There is a new private member, supported_minor_version_, which defaults
to kSupportedMinorPayloadVersion. It is set in the unit tests with calls
to SetSupportedVersion.
BUG=chromium:463573
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: Ib988c91eb450b2499c615ae65b271691dfd9c651
Reviewed-on: https://chromium-review.googlesource.com/260950
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
|
| de942f30cf986cf6bfc55fb5f9af6d7fea4ae51b |
13-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Generate valid delta files for squashfs.
This patch generates a valid (but inefficient) payload when a delta
payload is requested for a squashfs filesystem.
Since we are reusing the full payload generator for this purpose, this
patch relaxes the requirement of chunk_size to be positive in the
full_update_generator.cc code, replacing it by the default value of
1 MiB when nothing is passed.
BUG=chromium:430950
TEST=FEATURES=test emerge-link update_engine
TEST=cros_generate_update_payload for a delta payload works with
squashfs
TEST=`cros flash` a squashfs device with a delta payload works.
Change-Id: I88c7a571874c4c4697f528d44c52091aa1aed0c5
Reviewed-on: https://chromium-review.googlesource.com/260444
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
ull_update_generator.cc
enerate_delta_main.cc
|
| a26432a8fe5981ca1bb41cb6aacddfc3765666ac |
13-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Use the rootfs size specified by verity.
When using rootfs verification and generating a payload, it is important
to generate a payload that writes all the blocks verity expects to hash
in the hash tree, even if those blocks are never used by the filesystem.
When using squashfs, the filesystem is padded with zeros up to the
fs_size provided in the disk_layout.json, which is used as the size
of the dm-verity device. Because of this, it is important to always
generate a payload that writes all that.
This patch parses the kernel using vboot_host tools and updates the
rootfs_size value with the one specified to verity in the kernel
command line. When no verity options are found or the kernel is not
provided (as in the case of a full kernel update in a delta payload)
only the filesystem part is considered for the rootfs_size. This
means that the extra zeros in the source rootfs won't be used for
the delta payload when generating a full kernel payload, and also
means that the zeros after the squashfs won't be written when
generating the new rootfs with rootfs verification disabled.
BUG=chromium:463783
TEST=FEATURES=test emerge-link update_engine
TEST=Ran delta_generator with invalind kernels (in full and delta mode).
TEST=Ran cros_generate_update_payload with real images with both
squashfs and ext2.
Change-Id: Id151063722a20d27c50724f6b27f774a3436e3ea
Reviewed-on: https://chromium-review.googlesource.com/259839
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
enerate_delta_main.cc
ayload_generation_config.cc
ayload_generation_config.h
erity_utils.cc
erity_utils.h
erity_utils_unittest.cc
|
| 9b244df41f1bdaddd87b7dbd8e1559556059ed1b |
12-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Refactor operation generation methods.
We have so far three methods to generate the list of operations to
update the kernel and the rootfs partitions, which are different
strategies: full (using only REPLACE and REPLACE_BZ), delta "in-place"
(using also MOVE and BSDIFF) and "source" delta, or delta minor
version 2 (using SOURCE_COPY and SOURCE_BSDIFF instead).
As we consider ways to support updates for squashfs, the two delta
methods don't support filesystems other than ext2, so we need to use
yet another different strategy for those.
This patch splits the code that generates the list of operations for
the kernel and the rootfs in three methods with the same interface.
No functional changes were made. Some local functions were exposed
as public static methods. The new graph_types.cc is added to avoid
a link-time error when the kInvalidIndex member is not inlined.
BUG=chromium:430950
TEST=FEATURES=test emerge-link update_engine
Change-Id: Ib25ff7a6f17d0990637596dcd6b59568b9e15d26
Reviewed-on: https://chromium-review.googlesource.com/259462
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
ull_update_generator.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
raph_types.cc
raph_types.h
nplace_generator.cc
nplace_generator.h
etadata.cc
ayload_generation_config.cc
ayload_generation_config.h
|
| f1cbe1783dd025bd7243a8df12f20548196cc023 |
06-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Refactor payload generation request args in a class.
This patch introduces a new struct PayloadGenerationConfig encapsulating
the payload generation request performed by the user in the command
line.
Since we have several payload generation strategies that are triggered
based on the source image version and if there is an image version at
all, encapsulating the request in a single struct helps splitting the
different strategies in independent classes and pass the payload
generation configuration struct with the request to those classes.
BUG=chromium:430950
TEST=FEATURES=test emerge-link update_engine
Change-Id: Ie960c6510bf02d8e10f5e1adaa847a2048c59598
Reviewed-on: https://chromium-review.googlesource.com/258520
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
ull_update_generator.cc
ull_update_generator.h
ull_update_generator_unittest.cc
enerate_delta_main.cc
nplace_generator.h
ayload_generation_config.cc
ayload_generation_config.h
|
| 1f93d031baa4c0c4f390fcf5784b5179d15f289d |
11-Mar-2015 |
Alex Deymo <deymo@chromium.org> |
update_engine: Make dd less verbose.
update_engine unit tests use the "dd" command for setting up some test
cases. By default, dd print the status of the operation on success.
This patch removes the status message reducing the unnecessary logs.
BUG=None
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: I6dbb7029a8dce9acf1309da1844af558a2bdab56
Reviewed-on: https://chromium-review.googlesource.com/259190
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
etadata_unittest.cc
|
| fffd23e1ed1903beeb893640c611001175380186 |
26-Feb-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Payload generator emits SOURCE_COPY and SOURCE_BSDIFF ops.
When generating delta updates, payload generator will now emit
SOURCE_COPY and SOURCE_BSDIFF operations instead of MOVE and BSDIFF when
the minor version is 2.
BUG=chromium:461167
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: I1e7553312db7362d44d320d457e4a59756bfe5b9
Reviewed-on: https://chromium-review.googlesource.com/254780
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
enerate_delta_main.cc
nplace_generator.cc
|
| cd514b531a38da1497630fa68b6e3ef45871893d |
19-Feb-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Refactor inplace payload generator algorithm code.
Create a class, InplaceGenerator, that contains all functionality
related to the inplace delta generation method (minor version 1).
BUG=chromium:459701
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: Ib742f70030d6c2fcb1cc3138e0f4aef54eca6975
Reviewed-on: https://chromium-review.googlesource.com/251621
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
nplace_generator.cc
nplace_generator.h
nplace_generator_unittest.cc
etadata.cc
|
| 3283a55293ec9e645224c48d0c3cf6f8867395e7 |
19-Feb-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Add new operation codes for a to b operations
Add SOURCE_COPY and SOURCE_BSDIFF operations to protobuf. These
operations are for delta minor version 2 and their input is the source
partition instead of the target partition.
BUG=chromium:459363
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: I2c5c83ab5063ff6ce7fc2ec8acfe809d9526b6da
Reviewed-on: https://chromium-review.googlesource.com/250951
Reviewed-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
raph_utils.cc
|
| 425aa9764d6eac48cf06f726a30422d4e184c3d1 |
17-Feb-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Changes to delta minor version detection
Delta minor version should default to 1, the in-place algorithm minor
version, when the minor version is not given on the command line or in
the update_engine.conf file.
BUG=chromium:455493
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: I510a19c4a3b88bae656c19ad1a6d24598586b6a9
Reviewed-on: https://chromium-review.googlesource.com/250560
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
enerate_delta_main.cc
|
| 78750a450ce84b2d335134402d041aa8773ab8ef |
12-Feb-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Detect supported minor version from conf file.
If the delta minor version is not given as a command line argument, the
payload generator will look for the minor version in
etc/update_engine.conf in the old image.
BUG=chromium:455493
TEST=`FEATURES=test emerge-link update_engine`
CQ-DEPEND=CL:249956
Change-Id: I0f8e3f3c05432169770f2e8e283e8f1fb363def3
Reviewed-on: https://chromium-review.googlesource.com/249955
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
enerate_delta_main.cc
|
| f68bbbc952aa9a71898e4939b5f36187fa564a50 |
09-Feb-2015 |
Alex Vakulenko <avakulenko@chromium.org> |
update_engine: replace std::vector<char> with chromeos::Blob
To make update engine consistent with the rest of platform2 code
replaced std::vector<char> as the container of binary data with
chromeos::Blob.
BUG=None
TEST=`FEATURES=test emerge-link update_engine`
Change-Id: I6385fd2257d15aa24bfa74ac35512c2a06c33012
Reviewed-on: https://chromium-review.googlesource.com/247793
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
ull_update_generator.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
etadata.cc
ayload_signer.cc
ayload_signer.h
ayload_signer_unittest.cc
|
| 981a9fb68ec5fe56f57b3ecb117a0dc681bf5e83 |
09-Feb-2015 |
Alex Vakulenko <avakulenko@chromium.org> |
platform2: Switch over to using base64 functions from libchromeos
Replaced existing implementations of Base64Encode/Base64Decode
with the functions from libchromeos, which were added as part
of an earlier change (see CL:247690).
BUG=None
TEST=`FEATURES=test emerge-link cryptohome debugd metrics privetd update_engine`
Change-Id: I8cec677ce2c2fd3b97ca2228d35c2cf5cd133f4c
Reviewed-on: https://chromium-review.googlesource.com/247792
Reviewed-by: Vitaly Buka <vitalybuka@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
ayload_signer.cc
|
| b8ccad0a8ac2632da2788b47d85e62e334bbe652 |
09-Feb-2015 |
Allie Wood <alliewood@chromium.org> |
update_engine: Add command line option for payload minor version.
Adds minor_version option to delta_generator for delta updates. This
defaults to kSupportedMinorPayloadVersion when the value is not given.
Stores the value in the DeltaArchiveManifest protobuf. Checks
that the specified version is supported, and fails if it is not.
BUG=chromium:455489
TEST=delta_diff_generator unit tests and manual testing.
Change-Id: I95fb5576d8f33fe8be738488c5ea85d8a46e917b
Reviewed-on: https://chromium-review.googlesource.com/247802
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Allie Wood <alliewood@chromium.org>
Trybot-Ready: Allie Wood <alliewood@chromium.org>
Tested-by: Allie Wood <alliewood@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
enerate_delta_main.cc
|
| 063181c4fb0ab86c6aaba307c0cc00b901c4522b |
05-Feb-2015 |
Gilad Arnold <garnold@chromium.org> |
update_engine: Fix delta generator mishandling of unchanged blocks.
In a previous CL:181515 I made the delta generator filter out blocks
that were being MOVEed onto themselves, subsequently discarding empty
MOVE operations entirely. This resulted in unchanged blocks being
considered "unwritten". However, I did not realize that the delta
generator was scanning and packing all these blocks into one large
REPLACE_BZ at the end, without even checking whether any of them has
changed in the new image relative to the old image. Recently, we
realized that this causes deltas between largely similar (or worse,
identical) images to bloat.
It should be noted that this inefficiency existed before the
aforementioned feature was introduced, although it only applied to truly
unused filesystem blocks. However, such blocks being mostly zero, they
compressed well and likely did not affect the size of the delta much.
This CL fixes the problem by addressing the general problem: we do not
overwrite the content of previously unwritten blocks if they are
identical to the ones in the old image. Furthermore, if none of the
unwritten blocks has changed, we omit the said REPLACE_BZ operation
entirely. In the case of a delta between identical image, the result is
a payload with no operations at all (a good thing). In the particular
case of the failing panther_moblab build (see bug) this successfully
reduces the delta size from 625 MB to 331 bytes (!).
CQ-DEPEND=CL:246673
BUG=chromium:453659
TEST=Unit tests
TEST=Generated delta payload for previously failing panther_moblab build
TEST=Generated and tested two deltas for link (trivial N-to-N, real N-1)
Change-Id: I1c4e33d7cca5d59ba6725322970a329c1a3f7688
Reviewed-on: https://chromium-review.googlesource.com/246670
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
elta_diff_generator.cc
|
| a461fc37778f6c1c5aad7901417a01933ae2697a |
19-Nov-2014 |
Alex Deymo <deymo@chromium.org> |
update_engine: Used brace initialization instead of ad-hoc function.
SetWithValue<T> returns a set<T> with the passed value. This can be
achieved using the brace initialization introduced in C++11 with
just set<T>{value}.
BUG=None
TEST=Unittest still pass.
Change-Id: I24362098df3ea153600062fbb259833a37efa666
Reviewed-on: https://chromium-review.googlesource.com/230622
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
|
| 610277efc6f7e5239158dfa4bb3b1021804326e0 |
12-Nov-2014 |
Alex Deymo <deymo@chromium.org> |
update_engine: Add override when possible.
Google Style Guide requires to include the "override" keyword
when overriding a method on a derived class, so the compiler will
catch errors if the method is not overriding a member of the base
class.
This patch introduces the "override" keyword when possible.
BUG=None
TEST=FEATURES=test emerge-link update_engine
Change-Id: Ie83d115c5730f3b35b3d95859a54bc1a48e0be7b
Reviewed-on: https://chromium-review.googlesource.com/228928
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator_unittest.cc
ilesystem_iterator_unittest.cc
|
| aab50e31f0b80ed53a9b8d5dbabcf943974bd32c |
11-Nov-2014 |
Alex Deymo <deymo@chromium.org> |
Include the implemented header first in all projects.
The Google C++ style guide dictates that foo.cc and foo_unittest.cc
should include foo.h in the first place, so missing headers in foo.h
are detected with a compile error of the module implementing them and
not when another module uses them.
This CL sweeps across all the .cc file in platform2 enforcing this.
BUG=None
TEST=cbuildbot amd64-generic
Change-Id: I41835835caba13f54c3c844ecf552eb0e47efa9d
Reviewed-on: https://chromium-review.googlesource.com/228894
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
ull_update_generator_unittest.cc
raph_utils_unittest.cc
etadata.cc
etadata_unittest.cc
opological_sort_unittest.cc
|
| 10875d90cf67f883ba7c9ed13bc8d706aa8c6fbc |
11-Nov-2014 |
Alex Deymo <deymo@chromium.org> |
update_engine: Move test-only utils to test_utils.{h,cc}.
utils.{h,cc} contains a collections of basic or small functions used
in different parts of the codebase. The test_utils.{h,cc} instead
contains functions only required during testing split out to a
separated file to be reused in different tests.
This CL moves without changes some functions defined in utils.h that
were only used during unittests. Two other basic functions were replaced
by the same function already present in base/ (StringHasSuffix and
StringHasPrefix). The functions in test_utils.h now have their own
namespace chromeos_update_engine::test_utils so is clear they come
from the test_utils file, in the same way the ones from utils are
in their own namespace.
Some othe minor linter fixes included here.
BUG=chromium:351429
TEST=Unittest still pass.
Change-Id: I73ab72a14158cb21c8e1f404cbc728423bc8f34f
Reviewed-on: https://chromium-review.googlesource.com/229021
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
elta_diff_generator_unittest.cc
ilesystem_iterator.cc
ilesystem_iterator_unittest.cc
ull_update_generator_unittest.cc
etadata_unittest.cc
|
| 020600db42428e7e96bb0d2e05da225e68650955 |
06-Nov-2014 |
Alex Deymo <deymo@chromium.org> |
update_engine: Use C++11 range-based for loop when possible.
This cleanup just simplifies the notation of several for loops using
the new range-based syntax.
BUG=None
TEST=`FEATURES=test USE="lang clang" emerge-link update_engine`
Change-Id: I2120a1b1c3fd00e9a02a62abfa8427932080d0f9
Reviewed-on: https://chromium-review.googlesource.com/227897
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
enerate_delta_main.cc
etadata.cc
ayload_signer.cc
|
| f329b933db41d26644a97afef928eb1b319d6d99 |
30-Oct-2014 |
Alex Deymo <deymo@chromium.org> |
update_engine: Fix all the "using" declaration usage.
This patch removes unused "using" declarations, that is, declarations
included in a .cc file at a global scope such that "using foo::bar"
that later don't use the identifier "bar" at all.
This also unifies the usage of these identifiers in the .cc files
in favor of using the short name defined by the using declaration.
For example, in several cases the .h refer to a type like
"std::string" because using declarations are forbidden in header
files while the .cc includes "using std::string;" with the purpose
of just writting "string" in the .cc file. Very rarely, the full
identifier is used when a local name ocludes it, for example,
StringVectorToGStrv() and StringVectorToString() in utils.cc named
its argument just "vector" need to refer to std::vector with the
full name. This patch renames those arguments instead.
Finally, it also sorts a few lists of using declarations that weren't
in order.
BUG=None
TEST=FEATURES=test emerge-link update_engine
Change-Id: I30f6b9510ecb7e03640f1951c48d5bb106309840
Reviewed-on: https://chromium-review.googlesource.com/226423
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
ycle_breaker.cc
elta_diff_generator.cc
elta_diff_generator_unittest.cc
xtent_mapper.cc
ilesystem_iterator.cc
ull_update_generator.cc
raph_utils.cc
arjan_unittest.cc
|
| 02f7c1dee242f490143791dbb73fa23fa3007cfa |
19-Oct-2014 |
Ben Chan <benchan@chromium.org> |
update_engine: Replace scoped_ptr with std::unique_ptr.
BUG=None
TEST=`FEATURES=test emerge-$BOARD update_engine`
TEST=`USE='clang asan' FEATURES=test emerge-$BOARD update_engine`
Change-Id: I55a2f7f53675faaac20ba25f72ed52cf938d7744
Reviewed-on: https://chromium-review.googlesource.com/224189
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Ben Chan <benchan@chromium.org>
elta_diff_generator.cc
|
| 97b6f5a3f847f6ad4495dcecc11346caad4ab611 |
07-Oct-2014 |
Steve Fung <stevefung@chromium.org> |
update_engine: remove gflags dependency
We are switching to using chromeos/flag_helper.h instead to standardize the
code everywhere.
BUG=chromium:402636
TEST=`FEATURES=test emerge-panther update_engine`
CQ-DEPEND=I03b10d9e6148f1704de7a60128e1550156519272
CQ-DEPEND=I79112aa60d1f27ff0e5b8ee96f1ae46ee9a89a59
CQ-DEPEND=I6d08c080b6dafd59b23bbf185298c587e581c2ed
CQ-DEPEND=I218f6eab28b0160e8379a2716f5c33edeaf9841d
Change-Id: I81d0159f677b02a25640093c039c11e230af9b1b
Reviewed-on: https://chromium-review.googlesource.com/223216
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Steve Fung <stevefung@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Steve Fung <stevefung@chromium.org>
enerate_delta_main.cc
|
| 4a444ae459660ad4478011368c02c5f8abcea178 |
20-Sep-2014 |
Alex Deymo <deymo@chromium.org> |
update_engine: Limit bsdiff to 200 MiB files.
bsdiff and bspatch require a lot of RAM and time to run over very big
files. In released ChromeOS images, the biggest file is chrome with
a size of about 130 MiB in the worst case (link). Nevertheless,
ChromiumOS builds for ASan have much bigger files due to the memory
instrumentation.
This patch limit the usage of bsdiff/bspatch to files under 200 MiB,
something that should not affect any released board.
BUG=chromium:329248
TEST=cbuildbot amd64-generic-asan enabling the full vm_test
Change-Id: I24b6c9e56525858dab3758af8eddfe3acdf12e75
Reviewed-on: https://chromium-review.googlesource.com/219201
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
|
| a77939e368597241fe0153bedce196d7152a5de5 |
10-Sep-2014 |
Gabe Black <gabeblack@chromium.org> |
update_engine: Use utils::FileSize when finding the size of a file.
The utils::FileSize funciton can find the size of a block device correctly.
Use it instead of the adhoc methods used around the codebase.
BUG=chromium:415867
TEST=Ran a butterfly-paladin tryjob with --hwtest.
Change-Id: Id6fd37f04b136b4265bde9b1f56c379a5d9c30f9
Reviewed-on: https://chromium-review.googlesource.com/217418
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Gabe Black <gabeblack@chromium.org>
Tested-by: Gabe Black <gabeblack@chromium.org>
elta_diff_generator.cc
xtent_mapper.cc
xtent_mapper_unittest.cc
ilesystem_iterator.cc
ilesystem_iterator.h
|
| f9cb98c0cacccca803db48502c7bd461d04cb6b0 |
22-Sep-2014 |
Ben Chan <benchan@chromium.org> |
update_engine: Clean up angle brackets in template types.
C++11 no longer has the angle bracket pitfall in template types.
BUG=None
TEST=`FEATURES=test emerge-$BOARD update_engine`
Change-Id: I0168b9f208ad8e62ae614b8a3b8bcf31c58fa9f2
Reviewed-on: https://chromium-review.googlesource.com/219203
Commit-Queue: Ben Chan <benchan@chromium.org>
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
elta_diff_generator.cc
ull_update_generator.cc
enerate_delta_main.cc
ayload_signer.cc
ayload_signer.h
arjan.h
|
| 06c76a49bfd29c8abdb8abd5b646a6583783191a |
05-Sep-2014 |
Ben Chan <benchan@chromium.org> |
Update code to include base/files/file_util.h
file_util.h was moved from base to base/files
(https://codereview.chromium.org/468253002). This CL updates platform2
code to include base/files/file_util.h instead of base/file_util.h.
BUG=chromium:411001
TEST=Trybot run on paladin, release, and chromiumos-sdk builders.
Change-Id: I488925b54615e131e508a460dc1a27f88168f936
Reviewed-on: https://chromium-review.googlesource.com/216851
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Ben Chan <benchan@chromium.org>
Commit-Queue: Ben Chan <benchan@chromium.org>
elta_diff_generator.cc
|
| 05735a1879a553153458aae0a25fa5d42e3e408f |
03-Sep-2014 |
Ben Chan <benchan@chromium.org> |
update_engine: Include base/macros.h instead of base/basictypes.h
update_engine no longer uses the integer types from base/basictypes.h.
It should simply include base/macros.h for the DISALLOW_COPY_AND_ASSIGN
macro instead.
BUG=None
TEST=`FEATURES=test emerge-$BOARD update_engine`
Change-Id: Idf18bc69339b7170122b289342c724c862454062
Reviewed-on: https://chromium-review.googlesource.com/216140
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Ben Chan <benchan@chromium.org>
elta_diff_generator.h
xtent_mapper.h
xtent_mapper_unittest.cc
raph_types.h
raph_utils.cc
raph_utils.h
ayload_signer.h
|
| 88b591f24cb3f94f982d7024c2e8ed25c2cc26a2 |
29-Aug-2014 |
Alex Vakulenko <avakulenko@chromium.org> |
update_engine: Replace NULL with nullptr
Replaced the usage of NULL with nullptr. This also makes it possible to
use standard gtest macros to compare pointers in Update Manager's unit tests.
So, there is no need in custom UMTEST_... macros which are replaced with the
gtest macros (see change in update_engine/update_manager/umtest_utils.h):
UMTEST_ASSERT_NULL(p) => ASSERT_EQ(nullptr, p)
UMTEST_ASSERT_NOT_NULL(p) => ASSERT_NE(nullptr, p)
UMTEST_EXPECT_NULL(p) => EXPECT_EQ(nullptr, p)
UMTEST_EXPECT_NOT_NULL(p) => EXPECT_NE(nullptr, p)
BUG=None
TEST=FEATURES=test emerge-link update_engine
USE="clang asan" FEATURES=test emerge-link update_engine
Change-Id: I77a42a1e9ce992bb2f9f263db5cf75fe6110a4ec
Reviewed-on: https://chromium-review.googlesource.com/215136
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
elta_diff_generator.cc
elta_diff_generator_unittest.cc
ilesystem_iterator_unittest.cc
ull_update_generator.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
etadata.cc
etadata_unittest.cc
ayload_signer.cc
ayload_signer_unittest.cc
|
| 157fe307f162d13f53e7f98f6e2da7c60c0ff001 |
12-Aug-2014 |
Alex Vakulenko <avakulenko@chromium.org> |
platform2: sweep the lint errors identified by the updated linter
cpplint.py has been updated and identified new issues in existing
code. Stuff like overridden functions that specify 'override' should
not be marked as 'virtual', and constructors with no parameters
should not be marked as 'explicit'.
BUG=None
TEST=cpplint.py `find ./platform2 -name *.cc -or -name *.h`
Change-Id: Ibb9de43286d874d076ffd5ebb1b13c36ec794f01
Reviewed-on: https://chromium-review.googlesource.com/211950
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
elta_diff_generator.cc
|
| 5d5278068a007fc090368fba664fe130645880f7 |
18-Jul-2014 |
Alex Deymo <deymo@chromium.org> |
update_engine: Move ext2 tools to its own file in payload_generator.
update_engine doesn't need to understand about the rootfs filesystem
format, just how to apply the defined operations. This patch moves
the few ext2fs specific code the payload_generator and removes the
dependency on ext2fs from update_engine.
BUG=chromium:394184
TEST=FEATURES="test" emerge-link update_engine; sudo emerge update_engine
Change-Id: If750e4834d42bbeaeddc6eff7c4690dd916c73bf
Reviewed-on: https://chromium-review.googlesource.com/208992
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
xt2_utils.h
etadata.cc
etadata_unittest.cc
|
| 072359ca138504065e1e0c1189eb38c09576d324 |
18-Jul-2014 |
Alex Vakulenko <avakulenko@chromium.org> |
update_engine: fixed remaining linter and some spelling errors
Fixed remaining errors from cpplint as well as some spelling errors
mostly in comments.
BUG=None
TEST=FEATURES=test emerge-link update_engine
Change-Id: I484988ab846ac5a3c68c016ddccfb247f225ec27
Reviewed-on: https://chromium-review.googlesource.com/208897
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
ycle_breaker.cc
ycle_breaker.h
elta_diff_generator.cc
xtent_mapper.cc
xtent_mapper.h
ilesystem_iterator.h
ilesystem_iterator_unittest.cc
ull_update_generator.cc
ull_update_generator.h
raph_types.h
raph_utils.cc
raph_utils.h
etadata.h
arjan.h
arjan_unittest.cc
opological_sort.h
|
| 923d8faa6b341ce5e16a760749e98cc6703ca2fd |
16-Jul-2014 |
Alex Deymo <deymo@chromium.org> |
update_engine: Split payload signing and verification.
Payloads are only signed on server-side code (delta_generator) and
verified on both sides and unittest. This removes the dependency of
payload_generator/ code from delta_performer.cc by spliting the
payload signing and verification in two files.
Currently, both files are still included on all the built files.
This patch also includes some minor linter fixes.
BUG=chromium:394184
TEST=FEATURES="test" emerge-link update_engine; sudo emerge update_engine
Change-Id: Ia4268257f4260902bc37612f429f44ba7e8f65fd
Reviewed-on: https://chromium-review.googlesource.com/208540
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
enerate_delta_main.cc
ayload_signer.cc
ayload_signer.h
ayload_signer_unittest.cc
|
| d2779df63aaad8b65fc5d4badee7dbc9bed7f2b6 |
16-Jun-2014 |
Alex Vakulenko <avakulenko@chromium.org> |
update_engine: fixed warnings from cpplint
Fixed all the cpplint warnings in update engine.
BUG=None
TEST=Unit tests still pass.
Change-Id: I285ae858eec8abe0b26ff203b99a42a200ceb71c
Reviewed-on: https://chromium-review.googlesource.com/204027
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
ycle_breaker_unittest.cc
elta_diff_generator.cc
elta_diff_generator_unittest.cc
ull_update_generator_unittest.cc
enerate_delta_main.cc
raph_utils.cc
etadata.cc
etadata_unittest.cc
opological_sort.cc
opological_sort_unittest.cc
|
| 736fcb57ca2dcf09710a797e4c23a64f6cd64e16 |
22-May-2014 |
Ben Chan <benchan@chromium.org> |
update_engine: Update to build against libchrome-271506.
BUG=chromium:375032
TEST=`FEATURES=test emerge-$BOARD update_engine`
Change-Id: Ia7be302c12b439ad8aece51c9cbe128086626620
Reviewed-on: https://chromium-review.googlesource.com/201045
Tested-by: Ben Chan <benchan@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Ben Chan <benchan@chromium.org>
enerate_delta_main.cc
|
| bc91a2743fd3ffabb3ead3181ad8da3060afe1f1 |
21-May-2014 |
Alex Deymo <deymo@chromium.org> |
Replace std::tr1::shared_ptr by std::shared_ptr.
This patch replaces C++ Technical Report 1 additions to C++03 with
the newer C++11 name.
Includes and using statements updated accordingly.
BUG=None
TEST=Build and unittests.
Change-Id: I9bdad6d39684545c786f5c76fb598a15b557d6eb
Reviewed-on: https://chromium-review.googlesource.com/200665
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: David Zeuthen <zeuthen@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
ull_update_generator.cc
|
| 7dc4c50c0b11a8865c1dc9542a3e8f5016bebb9b |
21-May-2014 |
Alex Deymo <deymo@chromium.org> |
Move IsDir() to utils.
This patch moves the generic IsDir() function to the utils.cc file
where other similar functions reside. It also adds unit tests for it.
BUG=None
TEST=Added unittests.
Change-Id: Iba05059f72a1156bb5f19ae0624ae7025af4c522
Reviewed-on: https://chromium-review.googlesource.com/200691
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
enerate_delta_main.cc
|
| 1f0237181ffb92c3ffab0bd5eaa5868af3c83541 |
21-May-2014 |
Alex Deymo <deymo@chromium.org> |
Add usage message to the delta_generator
This patch removes a warning from gflags about SetUsageMessage() not
being called, providing a description for the binary.
BUG=None
TEST=./delta_generator --help shows the message.
Change-Id: Ifec0f123871d442bb9cab4a8bb27d09bf43b36be
Reviewed-on: https://chromium-review.googlesource.com/200667
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
enerate_delta_main.cc
|
| 161c4a132743f15fc4757112b673085c2a7a7f29 |
17-May-2014 |
Alex Deymo <deymo@chromium.org> |
Move payload generator files to payload_generator/ directory.
This creates a new subdirectory payload_generator/ with all the
payload generator specific files.
The SConstruct file is updated to continue building all the files
together, including those in the subdirectories, since some parts
of the update_engine are using parts of the payload generation code.
To reduce this code coupling, a new payload_constants.h file is
introduced, with few constants used on the payload definition by both
the payload generation and the payload performer.
Finally, includes are updated and in some cases removed when they
weren't used. Order of includes is also fixed to comply with the
style guide.
BUG=chromium:374377
TEST=Build and unittests still pass. delta_generator still present on base directory.
Change-Id: I454bbc7a66c70ebb19fd596c352c7be40a081f3d
Reviewed-on: https://chromium-review.googlesource.com/200325
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
ycle_breaker.cc
ycle_breaker.h
ycle_breaker_unittest.cc
elta_diff_generator.cc
elta_diff_generator.h
elta_diff_generator_unittest.cc
xtent_mapper.cc
xtent_mapper.h
xtent_mapper_unittest.cc
ilesystem_iterator.cc
ilesystem_iterator.h
ilesystem_iterator_unittest.cc
ull_update_generator.cc
ull_update_generator.h
ull_update_generator_unittest.cc
enerate_delta_main.cc
raph_types.h
raph_utils.cc
raph_utils.h
raph_utils_unittest.cc
etadata.cc
etadata.h
etadata_unittest.cc
arjan.cc
arjan.h
arjan_unittest.cc
opological_sort.cc
opological_sort.h
opological_sort_unittest.cc
|