History log of /system/update_engine/libcurl_http_fetcher.cc
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
eb2565593f622bfaadb874c57011357e7ca5c62a 22-Mar-2017 Alex Deymo <deymo@google.com> Set the uid to AID_OTA_UPDATE when tagging sockets.

The AID_OTA_UPDATE uid is the one designated to tag OTA traffic.

Bug: 35423816
Test: Tested on marlin
Change-Id: I0e9f73a0ad978031a1cbbce3a94a7f28bc6781ce
/system/update_engine/libcurl_http_fetcher.cc
13e9518b53bbbc138bf20ac1e50be17b25fda6a7 17-Mar-2017 Alex Deymo <deymo@google.com> Untag and stop watch on network sockets before closing them.

This patch untags the network sockets tagged with qtaguid_tagSocket()
before closing them, releasing the socket from the qtaguid kernel
module. Without this patch, the socket won't be actually closed.

This patch also removes the socket watch right before closing it, which
forces the epoll to stop watching. Closing the file descriptor would
also stop the watch if no other copy of the socket is kept open.
Explicitly stoping the watch is safer.

Bug: 36264049
Test: Ran update_engine under strace; the /proc/net/xt_qtaguid/ctrl
is called and the socket object destroyed and removed from the epoll.

Change-Id: I75c8befe8929e8e60c9534e0e4072f63b2fab1f4
/system/update_engine/libcurl_http_fetcher.cc
63cfcf4976713be8b20348cc263109b0127a9ac8 24-Feb-2017 Alex Deymo <deymo@google.com> Tag network sockets.

This patch tags all update_engine network sockets with 0x55417243
("CrAU" in little-endian) so its network data usage can be tracked.

Bug: 35721166
Test: Ran uniitetst; /proc/net/xt_qtaguid/stats shows this tag.
Change-Id: I8a4076f2958d493a59a5e73af0b6b54f1733e7f2
/system/update_engine/libcurl_http_fetcher.cc
b20de692d8a258eeea7b88619e722b19edb5408f 05-Feb-2017 Alex Deymo <deymo@google.com> Cleanup the RetryTimeoutCallback().

When canceling a request or destroying the LibcurlHttpFetcher, a
RetryTimeoutCallback callback could be leaked if the fetcher was
waiting on a network retry.

This patch keeps track of the retry callback and cancels it on CleanUp,
making sure the callback is not leaked.

Bug: 34178297
Test: Added unittest to trigger this case.
Change-Id: I7016641a7f31429933779e55c77cbabb6289c3dd
/system/update_engine/libcurl_http_fetcher.cc
71f6762c7e04774439c8011a5032bea6f182276f 04-Feb-2017 Alex Deymo <deymo@google.com> Fix terminating a transfer while resolving proxies.

Calling TerminateTransfer() on an HttpFetcher should cancel the current
transfer regardless of where it is sitting. If TerminateTransfer() was
called right after BeginTransfer(), the fetcher would be waiting for
the proxy resolution callback which will kick the download. This patch
fixes this case by releasing the proxy callback when terminating a
transfer.

Bug: 34178297
Test: Added a unittest to trigger this case.
Change-Id: I282d04995bd0d03f9a469c80c1e263f9902e4be2
/system/update_engine/libcurl_http_fetcher.cc
14c0da88a93aa7b1aa71d5e7e923b537f0d419f3 21-Jul-2016 Alex Deymo <deymo@google.com> Remove libcurl support from update_engine_sideload.

While sideloading an update from recovery we don't need nor want to
download payload from HTTP(S) URLs, only local file:// are supported.
This patch moves libcurl_http_fetcher and certificate_checker files out
of libpayload_consumer dropping the libcurl and libssl dependencies from
it and the update_engine_sideload.

Bug: 27178350
TEST=build UE for Brillo and Android. Unittests still pass and
update_engine_sideload doesn't link to libcurl.

Change-Id: Iffefdb094654f7277dc825c041fe55aac9ee8756
/system/update_engine/libcurl_http_fetcher.cc
39910dcd1d68987ccee7c3031dc269233a8490bb 10-Nov-2015 Alex Deymo <deymo@google.com> Split payload application code into a subdirectory.

This patch splits from the main libupdate_engine code the part that
is strictly used to download and apply a payload into a new static
library, moving the code to subdirectories. The new library is divided
in two subdirectories: common/ and payload_consumer/, and should not
depend on other update_engine files outside those two subdirectories.
The main difference between those two is that the common/ tools are more
generic and not tied to the payload consumer process, but otherwise they
are both compiled together.

There are still dependencies from the new libpayload_consumer library
into the main directory files and DBus generated files. Those will be
addressed in follow up CLs.

Bug: 25197634
Test: FEATURES=test emerge-link update_engine; `mm` on Brillo.

Change-Id: Id8d0204ea573627e6e26ca9ea17b9592ca95bc23
/system/update_engine/libcurl_http_fetcher.cc
35b3584ab4041aa32e1d049bd6ddff5941b03df2 20-Oct-2015 Alex Deymo <deymo@google.com> Use CA certificates from standard location in Brillo.

Brillo images have all the CA certificates for the whole system in
/system/etc/security/cacerts. For update_engine, we would like to
accept only a sub-set of those, but we will start by accepting all
the installed certs.

Bug: 25089263
Test: `mm`, deployed on a -userdebug build and updated

Change-Id: Ib05e6ea1c5efb98a6320ce6288842e2277f67b1f
/system/update_engine/libcurl_http_fetcher.cc
3f39d5cc753905874d8d93bef94f857b8808f19e 13-Oct-2015 Alex Vakulenko <avakulenko@google.com> update_engine: Rename "chromeos" -> "brillo" in include paths and namespaces

libchromeos is transitioning to libbrillo and chromeos namespaces
and include directory is changing to brillo.

Bug: 24872993
Change-Id: I770659a95be380a50fe3b2ba9f91d65818f40945
/system/update_engine/libcurl_http_fetcher.cc
f123ae2065ef19c172ae67a4c11cf23f1b787204 24-Sep-2015 Alex Deymo <deymo@google.com> Remove leaked callback when CleanUp() from TimeoutCallback().

When CleanUp() is called indirectly from TimeoutCallback(), the
TimeoutCallback() itself is canceled before the new recurrent call is
scheduled. The CancelTask() call will trivially succeed because the
callback already triggered.

Before CL:281197, the g_source_destroy() call to remove the currently
running callback would prevent it from being re-scheduled even if it
returns TRUE from the callback.

This patch re-schedules the callback before calling CurlPerformOnce()
so CancelTask() would cancel the scheduled task.

Bug: chromium:535649
Test: Added unittest. Verified it fails without the change.

Change-Id: Ica742dab0eb8d9d5c5055c8afac9d775ad1e0012
/system/update_engine/libcurl_http_fetcher.cc
aea4c1cea20dda7ae7e85fc8924a2d784f70d806 20-Aug-2015 Alex Deymo <deymo@google.com> Re-license update_engine to Apache2

This patch automatically replaced the license on all text files from
Chromium OS (BSD style) to AOSP (Apache2), keeping the original year as
a reference.

The license header was added to .gyp and .gypi files, the NOTICE was
replaced with a copy of the Apache2 license and MODULE_LICENSE_* file
was updated.

BUG=b/23084294
TEST=grep 'Chromium OS Authors' doesn't find anything.

Change-Id: Ie5083750755f5180a8a785b24fe67dbf9195cd10
/system/update_engine/libcurl_http_fetcher.cc
c00c98a1dad941e5cc04ce0b0e766d40b3b384e1 18-Mar-2015 Alex Deymo <deymo@chromium.org> update_engine: Use portable string format functions.

Replace usage of %zu by PRIuS or calls to std::to_string when possible.
This patch includes other minor linter fixes.

BUG=None
TEST=emerge-link update_engine

Change-Id: I9ff2b3677ed4218a140f9e91a2389cc756941b03
Reviewed-on: https://chromium-review.googlesource.com/293629
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
29b815316519829f133b39dd1cae7ab6224c9da2 09-Jul-2015 Alex Deymo <deymo@chromium.org> update_engine: Watch file descriptors using chromeos::MessageLoop.

This patch removes all the calls to g_io_*() functions used to create
io_channels from a file descriptor and watch for them in the main loop.
Instead, we use the chromeos::MessageLoop backed with the glib
implementation.

This patch also removes the duplicated process handling work done in
P2PManager and uses the common Subprocess class instead.

BUG=chromium:499886
TEST=Added and updated unittests.

Change-Id: Ia093b060d2396325fce69b2bbdb62957ba7bfbc6
Reviewed-on: https://chromium-review.googlesource.com/284593
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
60ca1a7bca7cc804ec80b510483081ef894de4cd 19-Jun-2015 Alex Deymo <deymo@chromium.org> update_engine: Migrate time-based glib main loop calls to MessageLoop.

This patch replaces most calls to g_idle_add* and g_timeout_add* with
the equivalent MessageLoop::Post*Task(). To maintain compatibility with
unittests running the main loop and doing I/O we instantiate a
GlibMessageLoop for those tests.

BUG=chromium:499886
TEST=unittests still pass.

Change-Id: Ic87ba69bc47391ac3c36d1bfc3ca28d069666af1
Reviewed-on: https://chromium-review.googlesource.com/281197
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Trybot-Ready: Alex Deymo <deymo@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
6a9d3497bcf57b8b9f5765a2909a51c9f8119cd1 15-Jun-2015 Alex Vakulenko <avakulenko@chromium.org> platform2: Fix issues with new version of libchrome

libchrome r334380 has the following breaking changes that need to be fixed:
- base::JSONWriter::Write() and base::JSONWriter::WriteWithOptions() take
"const base::Value&" instead of "const base::Value*"
- base::JSONReader::Read() and base::JSONReader::ReadAndReturnError()
return a scoped_ptr<base::Value> instead of base::Value*
- base/safe_strerror_posix.h is moved to base/posix/safe_strerror.h
- safe_strerror() is now in "base" namespace
- StartsWithASCII(), EndsWith(), StringToUpperASCII(), LowerCaseEqualsASCII()
are now in "base" namespace
- ObserverList<T> is now in "base" namespace
- base::PrintTo(base::FilePath) used in gtest is now moved to libchrome-test
library and as such, unit test runners need to link to this library now.
- crypto::RSAPrivateKey::CreateSensitive() is now removed from //crypto, so
some of tests in chromeos-login that used that function had to be changed
to use crypto::GenerateRSAKeyPairNSS() directly.
- UnixDomanSocket class is now in "base" namespace
- Pickle class is now in "base" namespace

BUG=chromium:496469
TEST=`./build_packages`
CQ-DEPEND=CL:277662

Change-Id: I36e5fbf2e36a92068873ffbd44020c862a3ed9e3
Reviewed-on: https://chromium-review.googlesource.com/277671
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
Trybot-Ready: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
f68bbbc952aa9a71898e4939b5f36187fa564a50 09-Feb-2015 Alex Vakulenko <avakulenko@chromium.org> update_engine: replace std::vector<char> with chromeos::Blob

To make update engine consistent with the rest of platform2 code
replaced std::vector<char> as the container of binary data with
chromeos::Blob.

BUG=None
TEST=`FEATURES=test emerge-link update_engine`

Change-Id: I6385fd2257d15aa24bfa74ac35512c2a06c33012
Reviewed-on: https://chromium-review.googlesource.com/247793
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
10875d90cf67f883ba7c9ed13bc8d706aa8c6fbc 11-Nov-2014 Alex Deymo <deymo@chromium.org> update_engine: Move test-only utils to test_utils.{h,cc}.

utils.{h,cc} contains a collections of basic or small functions used
in different parts of the codebase. The test_utils.{h,cc} instead
contains functions only required during testing split out to a
separated file to be reused in different tests.

This CL moves without changes some functions defined in utils.h that
were only used during unittests. Two other basic functions were replaced
by the same function already present in base/ (StringHasSuffix and
StringHasPrefix). The functions in test_utils.h now have their own
namespace chromeos_update_engine::test_utils so is clear they come
from the test_utils file, in the same way the ones from utils are
in their own namespace.

Some othe minor linter fixes included here.

BUG=chromium:351429
TEST=Unittest still pass.

Change-Id: I73ab72a14158cb21c8e1f404cbc728423bc8f34f
Reviewed-on: https://chromium-review.googlesource.com/229021
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
f329b933db41d26644a97afef928eb1b319d6d99 30-Oct-2014 Alex Deymo <deymo@chromium.org> update_engine: Fix all the "using" declaration usage.

This patch removes unused "using" declarations, that is, declarations
included in a .cc file at a global scope such that "using foo::bar"
that later don't use the identifier "bar" at all.

This also unifies the usage of these identifiers in the .cc files
in favor of using the short name defined by the using declaration.
For example, in several cases the .h refer to a type like
"std::string" because using declarations are forbidden in header
files while the .cc includes "using std::string;" with the purpose
of just writting "string" in the .cc file. Very rarely, the full
identifier is used when a local name ocludes it, for example,
StringVectorToGStrv() and StringVectorToString() in utils.cc named
its argument just "vector" need to refer to std::vector with the
full name. This patch renames those arguments instead.

Finally, it also sorts a few lists of using declarations that weren't
in order.

BUG=None
TEST=FEATURES=test emerge-link update_engine

Change-Id: I30f6b9510ecb7e03640f1951c48d5bb106309840
Reviewed-on: https://chromium-review.googlesource.com/226423
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
88b591f24cb3f94f982d7024c2e8ed25c2cc26a2 29-Aug-2014 Alex Vakulenko <avakulenko@chromium.org> update_engine: Replace NULL with nullptr

Replaced the usage of NULL with nullptr. This also makes it possible to
use standard gtest macros to compare pointers in Update Manager's unit tests.
So, there is no need in custom UMTEST_... macros which are replaced with the
gtest macros (see change in update_engine/update_manager/umtest_utils.h):

UMTEST_ASSERT_NULL(p) => ASSERT_EQ(nullptr, p)
UMTEST_ASSERT_NOT_NULL(p) => ASSERT_NE(nullptr, p)
UMTEST_EXPECT_NULL(p) => EXPECT_EQ(nullptr, p)
UMTEST_EXPECT_NOT_NULL(p) => EXPECT_NE(nullptr, p)

BUG=None
TEST=FEATURES=test emerge-link update_engine
USE="clang asan" FEATURES=test emerge-link update_engine

Change-Id: I77a42a1e9ce992bb2f9f263db5cf75fe6110a4ec
Reviewed-on: https://chromium-review.googlesource.com/215136
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
4906c1c6dac2ef7916bfa7193cc9324e99b4d223 21-Aug-2014 Alex Vakulenko <avakulenko@chromium.org> update_engine: Replace googe::protobuf::Closure with base::Closure

In order to clean up libchromeos, remove chromeos/callback.h and
switch update_engine from using
googe::protobuf::Closure/NewPermanentCallback to base::Closure/Bind

BUG=chromium:406060
TEST=FEATURES=test emerge-link update_engine

Change-Id: I02d222c3f6c9a5bf5177d58e3a25a596348619bf
Reviewed-on: https://chromium-review.googlesource.com/213675
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Reviewed-by: David Zeuthen <zeuthen@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
d2779df63aaad8b65fc5d4badee7dbc9bed7f2b6 16-Jun-2014 Alex Vakulenko <avakulenko@chromium.org> update_engine: fixed warnings from cpplint

Fixed all the cpplint warnings in update engine.

BUG=None
TEST=Unit tests still pass.

Change-Id: I285ae858eec8abe0b26ff203b99a42a200ceb71c
Reviewed-on: https://chromium-review.googlesource.com/204027
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
77f79e876a77796fc248d099b6574f05bd23c954 03-Jun-2014 Chris Sosa <sosa@chromium.org> Only disable update downloads over expensive connections, not all transfers.

Previous to this CL, all network traffic was disabled from the update engine
if an update wasn't allowed. Instead, in this CL, we switch to only disable
the update application portion (downloading and applying the payload). This
is done by moving the detection logic from the mechanism (the network fetcher)
to the user of said mechanism i.e. the omaha request/response actions.

I have done a little refactoring of the unittests to make this CL easier to
test and found 1 bug in the http_fetcher_unittests where we weren't sending
either the right URL or handling a server not existing correctly.

This CL comes with one caveat:
Technically in the previous impl if a download started over wifi then got
disconnected / retried and bounced to a different connection type that
we couldn't update over, we'd stop the update. This would no longer be true
here if the http_fetcher recovered seamlessly.

BUG=chromium:379832
TEST=Unittests

Change-Id: I6b1a76e4c030d4e384e8ba0b519424a35406aafb
Reviewed-on: https://chromium-review.googlesource.com/202435
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: David Zeuthen <zeuthen@chromium.org>
Commit-Queue: Chris Sosa <sosa@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
c4acdf40033fc7bedba3ec6449b7a750b1fe5f24 29-May-2014 Alex Deymo <deymo@chromium.org> Fix memory leak on HttpFetcher and ProxyResolver.

The current HttpFetcher and ProxyResolver code uses
google::protobuf::Closure callbacks created with NewCallback. These
callbacks will self-delete them when you call Run(), leaking the
callback if that doesn't happens.

This patch replaces all the NewCallback() calls by
NewPermanentCallback(), which won't delete the callback after running
it. It then adds a new utils::GlibDestroyClosure() function to use in
conjunction with the existing utils::GlibRunClosure() to schedule
callbacks from the glib main loop without leaking them.

Finally, this patch fixes a use-after-free on the
AbortingHttpFetcherTestDelegate class only affecting unit tests.

Other minor linting errors fixed.

BUG=chromium:378548
TEST=`FEATURES="test" USE="clang asan" emerge-link update_engine` doesn't complain about HttpFetcher.

Change-Id: Ica3265aca42f07811b7dff6131f9a43ab06269aa
Reviewed-on: https://chromium-review.googlesource.com/202062
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
7d623ebf9eee346fdeee71f1ccb32ebaf4bd3318 14-May-2014 Nam T. Nguyen <namnguyen@chromium.org> update_engine: Remove GPIO support.

We have not used GPIO since Jan 2013. This CL removes the GPIO handling code.
As a side effect, it also removes dependency on libudev.

BUG=chromium:221725
CQ-DEPEND=CL:199683
TEST=unittest
TEST=AU end2end test on real device from older version to a version that
has CL applied, then from that to another newer version (need not have
this CL applied).

Change-Id: I4352488ec360b44a44b137c40a3ae4ec35c6fe9d
Reviewed-on: https://chromium-review.googlesource.com/199626
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Nam Nguyen <namnguyen@chromium.org>
Tested-by: Nam Nguyen <namnguyen@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
7984bf0ef36c92fa5cd3b23e307e1c1656497950 03-Apr-2014 Alex Deymo <deymo@chromium.org> Remove IsOfficialBuild() from other singleton interfaces.

IsOfficialBuild() is replicated on other singleton interfaces as a
shortcut for accessing the new HardwareInterface. These shortcuts
were used for testing when it wasn't possible to fake out this value
in a more standard way.

This patch removes the IsOfficialBuild() method from all the
singleton interfaces and uses HardwareInterface directly, which
can be faked via the default FakeHardware on MockSystemState.
This helps reduce the actual dependencies on the
UpdateCheckScheduler before migrate it to the PolicyManager.

Some minor linter issues are also solved on this patch.

BUG=chromium:358269
TEST=Unittests still pass.

Change-Id: I19d5add04b8cdc679e918cbc7fe27f688e8da64e
Reviewed-on: https://chromium-review.googlesource.com/192974
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Alex Deymo <deymo@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
75039d7397f03dff77bdf4e26398049ff88edc4c 25-Mar-2014 Alex Vakulenko <avakulenko@chromium.org> update_engine: update to libbase 242728

Made update_engine link with libchrome-242728 and fixed compile
issues due to changes in namespaces and include file layout.

Also removed some of suppressed compiler warnings and fixed the
issues masked by them (e.g. mismatched printf-like specifiers).

Added -Wextra compiler option to enable additional useful warnings
to ensure more strict checking... Had to disable "unused-parameter"
though since we have a lot of functions (mainly in fakes) that do
not use all of their parameters.

BUG=chromium:351593
TEST=Unit tests passed.
CQ-DEPEND=CL:191721

Change-Id: I1aa63a48d5f1f4ea75ba6b00aec7aa5f3bad15c4
Reviewed-on: https://chromium-review.googlesource.com/191510
Reviewed-by: Alex Vakulenko <avakulenko@chromium.org>
Commit-Queue: Alex Vakulenko <avakulenko@chromium.org>
Tested-by: Alex Vakulenko <avakulenko@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
6ae9120dc76ce9a3c8d05f71c05f23e3f4bdc6bd 11-Mar-2014 Alex Deymo <deymo@chromium.org> Treat confirmed Tethered networks as Cellular networks.

The updates are allowed based on the type of connection used to
download it from. For example, by default, no update is made over
Cellular networks even if the device is connected to a VPN over a
Cellular network to prevent huge charges on those connections.

Nevertheless, when the device is connected to a tethered network such
as an Android or iPhone sharing its Cellular connection over Wifi,
the connection type the device sees is a Wifi and thus will allow
the updates by default.

To prevent updates over tethered networks, this patch uses the
Tethering property expossed by shill to avoid those situations. If
the device is connected to a network that shill confirms to be a
tethered network, it will be treated as if the device is connected
to a Cellular network. This means that the updates will be allowed
based on the same settings that govern if the updates are allowed
over Cellular networks.

BUG=chromium:323010
TEST=Unit tests added to verify policy and property parsing.

Change-Id: I3a31c804465c9ed5c76b5d6156adda8e5e4e8a6d
Reviewed-on: https://chromium-review.googlesource.com/189524
Tested-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: Alex Deymo <deymo@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
1b9d6ae00b3c3badf9785584c2c867710f214a62 03-Mar-2014 Gilad Arnold <garnold@chromium.org> Further cleanup around DBus interface classes.

* Removed the "Glib" part from the various DbusGlib class name and added
"Wrapper" instead: only part of the methods here declared have
anything to do with Glib; in essence, this is an interface containing
wrappers for various DBus functions.

* Changed "Dbus" to "DBus", to comply with the rest of the world.

* Renamed the actual implementation "RealDBusWrapper" (instead of
"Concrete").

* Separated out RealDBusWrapper into its own header file. Client code,
which often only cares about the abstract class, does not need to pull
it unless specifically requiring it.

* Cleaned up the includes in these headers.

Aside from all that, also renamed system_state.cc into
real_system_state.cc, as this is what this file contains.

BUG=None
TEST=Unit tests.

Change-Id: I015c407cbc159aba8b5925eb0e916ba604c829cd
Reviewed-on: https://chromium-review.googlesource.com/189373
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
133f892757c10d4e42602bfacd5f8324ea4f6999 07-Nov-2013 Chris Sosa <sosa@chromium.org> Default to allowing updates when we can't determine the network type.

BUG=None
TEST=Manual -- kill shill.

Change-Id: Ibed384b42ef461d0fb728f587c8e3d15c76a6de1
Reviewed-on: https://chromium-review.googlesource.com/176021
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
056b0abc1273a6eebfd73060a1d0344ef02b8696 29-Oct-2013 J. Richard Barnette <jrbarnette@chromium.org> Move IsOfficialBuild() and IsNormalBootMode() into HardwareInterface.

This makes the implementation of the two methods part of the
HardwareInterface, so that unit tests won't end up with meaningless
(and unpredictable) calls to the real functions.

BUG=None
TEST=unit tests

Change-Id: Ia23932634124987c1d6ff0683acb15cf4819bc5e
Reviewed-on: https://chromium-review.googlesource.com/175024
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
34135a9f24c72137aa85378b57e7698162c687f2 06-Aug-2013 David Zeuthen <zeuthen@chromium.org> p2p: Make HTTP downloads fail fast if using p2p to download

Failing fast when downloading via p2p is desirable because if we're
disconnected from the peer we're downloading from, chances are good
that it's not coming back. For example the peer could have gone to
sleep (user shutting the lid) or gone out of range. This is unlike the
non-p2p path where we can assume much better connectivity.

Also introduce new constants instead of hard-coded numbers and move
some existing constants to constants.h.

BUG=chromium:260426
TEST=Unit tests pass
Change-Id: Id2f1d0c60907caec06c4bdff3c70871d9f3eb20d
Reviewed-on: https://chromium-review.googlesource.com/64830
Reviewed-by: David Zeuthen <zeuthen@chromium.org>
Commit-Queue: David Zeuthen <zeuthen@chromium.org>
Tested-by: David Zeuthen <zeuthen@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
59d9e0142388f23dae84f7b480d24f1f71b80cfa 24-Jul-2013 Gilad Arnold <garnold@chromium.org> AU: remove obsolete proxy resolver code

The module chrome_proxy_resolver has been long obsoleted by
chrome_browser_proxy_resolver, only that it kept living in our codebase,
if only for the use of a single static method, which had nothing to do
with proxy resolution and in fact is Curl specific. This CL removes it
along with its build/execution dependencies.

BUG=chromium:208655
TEST=Builds fine and unit tests running

Change-Id: Iff50c2c75451e5f3ddbc27c5a90a8b1a421a5d8d
Reviewed-on: https://gerrit.chromium.org/gerrit/63147
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Alex Deymo <deymo@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
6f6ea00aa8c4cf54b6842be32ca1226854c24f78 14-Dec-2012 Jay Srinivasan <jaysri@chromium.org> Support for processing multiple URLs in update_engine.

Main changes:
1. Added a new PayloadState class which encapsulates all the persisted
state we use for multiple URLs, back-off (TBD), etc.
2. Added support for handling multiple URLs stored in the OmahaResponse in
OmahaRequestAction and OmahaResponseHandlerAction code.
3. Added support for picking the right URL in OmahaResponseHandlerAction
and putting it in the install_plan. This way, the rest of the code that
uses the install_plan is oblivious to the presence of multiple URLs :-)
4. Added support for advancing to next URL when an update fails. The full
error classification is a new work item (chromium-os:37206). Right now,
it's a basic round-robin on every error.
5. Updated the conditions for determining when hash checks are mandatory.
Previously since there was only one URL, if it was HTTPS, the checks were
waived. Now, even if there's one HTTP URL, we make hash checks mandatory
even if other HTTPS URLs are present.

6. Added new unit tests for PayloadState and the new logic added to other
places.

Noisy changes:
1. Instead of passing PrefsInterface to OmahaRequestAction and
OmahaResponseHandlerAction, we're now passing SystemState which will now
contain PrefsInterface and the newly added PayloadState object that these
actions need to do their work.
2. Renamed a bunch of setters/getters to set_x() and x() instead of SetX()
and GetX() methods - this was pending from Gilad's old CR. As I'm
adding new methods in the correct style, I went ahead and fixed it to
avoid the confusing styles.
3. Updated all existing unit tests to reflect these changes.

BUG=chromium-os:36807
TEST=All Single/Multiple URL scenarios work fine on my ZGB as expected.
TEST=Old and new unit tests run fine.

Change-Id: Id31f9ccb220471f3ec3a475f624dc03c16119144
Reviewed-on: https://gerrit.chromium.org/gerrit/39638
Commit-Ready: Jay Srinivasan <jaysri@chromium.org>
Reviewed-by: Jay Srinivasan <jaysri@chromium.org>
Tested-by: Jay Srinivasan <jaysri@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
738fdf37c15284b60dac703408b8de19eef9c6a3 08-Dec-2012 Jay Srinivasan <jaysri@chromium.org> Make hash checks mandatory for HTTP downloads.

Currently we've made all the checks for metadata size, metadata signature
and operation hashes as optional. While they are still optional if we use
HTTPS for downloading the payload, we want to make them mandatory in case
of HTTP, so as to support HTTP downloads.

In this CL, we make these checks mandatory if the Omaha response has a
HTTP URL. This will not affect any scenarios of our test team because they
always use HTTPS URLs for payload URLs. But this would break the dev tools
and our hardware test lab scenarios because they use HTTP URLs and do not
generate the required manifest signature yet. So we waive this requirement
for dev/test images even though they use HTTP.

This CL will not have any effect until we decide to add a HTTP rule in
Omaha, which serves as a safety knob till we are confident with our
testing.

BUG=chromium-os:36808
TEST=Existing unit tests pass. Added new unit tests for most new code.
TEST=Ran manual tests on ZGB for every type of hash failure for HTTP.
TEST=Tested image_to_live to make sure hash checks are waived as expected.

Change-Id: I8c4408e3052635ccf4bee0c848781733c1f8e984
Reviewed-on: https://gerrit.chromium.org/gerrit/39293
Reviewed-by: Gaurav Shah <gauravsh@chromium.org>
Commit-Ready: Jay Srinivasan <jaysri@chromium.org>
Reviewed-by: Jay Srinivasan <jaysri@chromium.org>
Tested-by: Jay Srinivasan <jaysri@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
b3f5540ee27bd53a28e3a3f87e2f5e80fb2a4876 04-Dec-2012 Jay Srinivasan <jaysri@chromium.org> Set libcurl options based on protocol (HTTP or HTTPS)

Today update_engine only supports HTTPS protocol. So we lock down libcurl
to use only HTTPS, set options to validate the peer certificate,
set other SSL options, etc. Moving forward, we want to support payload
downloads via HTTP also. So, this CL sets the old options only if the URL
we're using is HTTPS. If it is HTTP, we just need to lock down the protocol
to HTTP.

BUG=chromium-os:36809
TEST=Existing unit tests pass.
TEST=Manually tested downloading using HTTP and HTTPS URLs on my ZGB.

Change-Id: I5948beb3bb84914dd3a3801f516c2eb15c857c6b
Reviewed-on: https://gerrit.chromium.org/gerrit/39142
Reviewed-by: Gaurav Shah <gauravsh@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Commit-Ready: Jay Srinivasan <jaysri@chromium.org>
Tested-by: Jay Srinivasan <jaysri@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
7c04e76f469b30b0086201baba94db1c15a028e5 23-May-2012 Gilad Arnold <garnold@chromium.org> Enable test mode updates.

* Uses the GPIO module to deduce whether a current update session needs
to be treated as a controlled test, which allows a few relaxations.

* LibcurlHttpFetcher is extended to relax some of its security lock down
provisions.

* Fix: a test mode flag remains persistent throughout an update attempt,
so that it can be delegated to the various HttpFetcher instances used
in the same attempt.

BUG=chromium-os:25397
TEST=Builds and unittests; automated test script works w/ servo
connected Alex

Change-Id: I8a29d1a21a0632912c10f01f69a26d9c659472fd
Reviewed-on: https://gerrit.chromium.org/gerrit/25128
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
4348879380222dc8c74b0976e527bf3da66ba08f 19-Jun-2012 Jay Srinivasan <jaysri@chromium.org> UpdateEngine-side changes to allow updates over 3G based on device policy.

Some enterprise chromebooks have only 3G and hence they need the ability
to update over 3G if the enterprise policy allows that. This CL adds
the support in update_engine to enable that.

BUG=chromium-os:31099
TEST=Tested E2E on 3G, added unit tests and did regression testing.
CQ-DEPEND=I1a55a392f3dc0f12d917eb45dcf0456b57735514
Change-Id: I121bda35e54fa6c35e002a76db198d13b72b650e
Reviewed-on: https://gerrit.chromium.org/gerrit/25470
Commit-Ready: Jay Srinivasan <jaysri@chromium.org>
Reviewed-by: Jay Srinivasan <jaysri@chromium.org>
Tested-by: Jay Srinivasan <jaysri@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
d186e637e06ac0aff98a2ad4f87101037ec61d96 13-Jun-2012 Don Garrett <dgarrett@google.com> Fix update_engine scons building.

To build update_engine without an emerge, you need to run setup_dev_packages
to get the needed packages into the chroot, and then "scons" to do the
actual build.

However, setup_dev_packages has a broken dependancy. That dependancy is needed
to generate the test coverage reports, but not otherwise. Also, the SConstruct
file listed a "BASE_VER" for libchrome dependancies that was outdated and
overridden in the ebuild file.

This change removes the broken package depedancy, and updates the BASE_VER
so that it's relatively easy to build the update engine directly without doing
an emerge. That breaks the script to measure how much unittest coverage we
have, but that script is already broken.

Also, Gilad was right about static int's in one of Jay's previous CLs (and I
was wrong). How he was declaring some constants is causing problems for
commanline compiles, but doesn't for ebuild builds. So, this CL fixes that.

BUG=None
TEST=scons compelted.

Change-Id: I134f83528d8ad6fe3e504d98efb3d1030bfa3865
Reviewed-on: https://gerrit.chromium.org/gerrit/25252
Reviewed-by: Jay Srinivasan <jaysri@chromium.org>
Tested-by: Jay Srinivasan <jaysri@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Commit-Ready: Don Garrett <dgarrett@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
08fce04e5341c99f308ee7462b1f351f5232eacd 08-Jun-2012 Jay Srinivasan <jaysri@chromium.org> Don't scatter during OOBE or user-initiated update checks.

We need to add logic to disable scattering of downloads if we are in OOBE
or if we're doing a manual update check.

Scheduled checks are already disabled during OOBE, but this extra check
will ensure that any scattering policy (there's a pending work item to get
policy during OOBE) during OOBE will have no effect on the update.

Similarly manual (i.e user-initiated) update checks through
update_engine_client or through Chrome UI should not honor scattering.
That way, this can serve as a simple user-friendly workaround in case
there's any bug in scattering logic that bricks the system by any chance.

BUG=chromeos-31563: Don't scatter during OOBE or manual update checks.
TEST=Updated unit tests. Tested all code paths manually on ZGB and Kaen.
Change-Id: Ib631e560c1f620ca53db79ee59dc66efb27ea83c
Reviewed-on: https://gerrit.chromium.org/gerrit/24564
Commit-Ready: Jay Srinivasan <jaysri@chromium.org>
Reviewed-by: Jay Srinivasan <jaysri@chromium.org>
Tested-by: Jay Srinivasan <jaysri@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
32f2357356546d3a4d1184fdbb6b664d620c6af1 05-Jun-2012 Jay Srinivasan <jaysri@chromium.org> Add resiliency in UE against interrupted HTTP downloads.

Currently when there's an interruption in the HTTP transfer, update_engine
attempts to resume the transfer only 3 times. After that it gives up.
For reasons yet to be investigated (31019), this happens quite consistently
on ARM and hence we need to fix update_engine to be more resilient in such
cases.

The fix is to increase the retry count to 20 for post-OOBE case. We'll
still maintain the original limit of 3 retries for OOBE case so as to not
stall OOBE forever.

BUG=chromeos-31511:Add resiliency in UE against interrupted HTTP downloads
TEST=Existing unit tests hit all new code paths and pass. Manually tested
on ZGB and Kaen.
Change-Id: I4e47761dc9b859701023b749c902ec2b1c649b6f
Reviewed-on: https://gerrit.chromium.org/gerrit/24416
Commit-Ready: Jay Srinivasan <jaysri@chromium.org>
Reviewed-by: Jay Srinivasan <jaysri@chromium.org>
Tested-by: Jay Srinivasan <jaysri@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
a2dee1d3a7f13c91b6d4973dca477c5496e9cf53 12-Apr-2012 Gilad Arnold <garnold@chromium.org> Fix AU fallback proxy mechanism for resumed updates.

This fixes a subtle problem where resuming an update in the presence of
a failing proxy would consistently fail, without resorting to
alternative proxies or a direct connection.

BUG=chromium-os:29042
TEST=Unittests; tested resume update with bogus proxy configuration.

Change-Id: I81eab8cf65451c891569091eda84ce9a18a7f8ab
Reviewed-on: https://gerrit.chromium.org/gerrit/20119
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Darin Petkov <petkov@chromium.org>
Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
fbaee24d80b2da982b81deca03702350e656f4b5 05-Apr-2012 Gilad Arnold <garnold@chromium.org> AU: fix proxy logging

BUG=chromium-os:28966
TEST=Passes unit tests; update logs correctly show proxy settings

Change-Id: I5fed3e0bacb12e085b7d4e53638ee7a4096cf9a5
Reviewed-on: https://gerrit.chromium.org/gerrit/19613
Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
9dd1e7c6bbc605c2150800f763c596ff0a4ad1c1 16-Feb-2012 Gilad Arnold <garnold@chromium.org> Update engine properly sets Content-Type to text/xml.

Since the payload sent to Omaha is XML encoded, update engine should
properly set the HTTP Content-Type header to text/xml, instead of using
the libcurl default (application/x-www-form-urlencoded).

BUG=chromium-os:7613
TEST=Tested against devserver, ensuring that Content-Type is set
correctly

Change-Id: I9766e8dd67ffd387634a0ab4ef83c2990b16b537
Reviewed-on: https://gerrit.chromium.org/gerrit/16051
Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
34bf1ee4ff22ece3dfc5c207e7e1f46aaff73a24 10-Feb-2012 Gilad Arnold <garnold@chromium.org> Reduce number of resume attempts and delay between attempts.

This reduces the number of resume attempts from 20 to 3 (total
4 attempts), and the delay between a failure and the subsequent attempt
from 60 to 20 seconds. In total, the delay induced due to resume
attempts will not exceed 1 minute. This appears long enough to overcome
transient connectivity issues such as a user moving between access
points, a temporary Ethernet cable disconnect, or a switch restarting.
It is necessary for reducing the amount of time a user needs to wait in
the case of interrupted network connectivity during OOBE update.

This requires a corresponding fix to unittests, as the assumption on the
number of retries has changed.

BUG=chrome-os-partner:7961
TEST=Passed unittests; verifying OOBE behavior pending at test lab.

Change-Id: I669ac6e2a1264c1d154b23a06c66763f155ab946
Reviewed-on: https://gerrit.chromium.org/gerrit/15639
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
e4ad2508de4d69d7a90d3ce441efe2c82c55bd1d 30-Dec-2011 Gilad Arnold <garnold@chromium.org> AU multi-range fetcher requests properly closed ranges when their length
is known.

* HttpFetcher allows to set the length of data to be fetched.
LibcurlHttpFetcher uses this value in applying the appropriate libcurl
option (CURLOPT_RANGE). MultiHttpFetcher sets the desired payload
length in the underlying fetcher accordingly.

* Improved functionality of test_http_server: (a) correctly parses
closed range intervals; (b) generalized response header generation;
(c) unified and generalized get handling for both stable and flaky
cases.

* Small scale refactoring, improved logging and readability.

BUG=chromium-os:24666
TEST=unit tests

Change-Id: I1727710ca747088c67a68305f355da683b07b6a3
Reviewed-on: https://gerrit.chromium.org/gerrit/13594
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
9bedeb51f80c9547269eef6c2ec09596033bb818 18-Nov-2011 Gilad Arnold <garnold@chromium.org> Fetcher tries all proxies when a secondary chunk download error occurs.

This is a fix to issue 18143:

* New test cases for asserting the desired behavior: if a transfer of
a secondary chunk within a multi-chunk fetch fails, then the fetcher
needs to retry with other available proxies; it will only fail when no
additional proxies are available. The tests ensure both success (one
of the proxies eventually succeeds) and failure (all proxies fail)
cases.

* Small fix to LibcurlHttpFetcher to retry with other proxies upon
failure (error value) of a secondary chunk.

Other changes applied in the course of this fix:

* Massive refactoring of http_fetcher_unittest: substituted template
specialization in typed test setup with proper subclassing, resulting
in a safer and more maintainable infrastructure; extended URLs to
include all (most) parameters pertaining to test workload, such as
download size, flakiness, etc.

* Respective changes to test_http_server: it is now much more
independent of particular kind of tests, and more easily
parametrizable. Also, generalized several internal methods for better
readability and extensibility, such as writing of arbitrary payloads,
parsing headers,

* Migrated common definitions into http_common.{h,cc} (universal
HTTP-related stuff) and http_fetcher_unittest.h (shared definitions
pertaining to unit tests).

* Extended direct proxy resolver to generate a list of (non-) proxies,
so we can unit test proxy failure. Also, better logging to improve
testability.

* Some renaming of classes for better consistency.

BUG=chromium-os:18143
TEST=unit tests

Change-Id: Ib90b53394d7e47184d9953df8fc80348921e8af0
Reviewed-on: https://gerrit.chromium.org/gerrit/12092
Commit-Ready: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
48085ba58516e94f045d3ab7e26c8f36e6a6936f 16-Nov-2011 Gilad Arnold <garnold@google.com> Updater avoids download in case of an error HTTP response.

(a) LibcurlHttpFetcher avoids download if the HTTP reponse indicates an
error; corresponding change to unit test code and test HTTP server. (b)
Added a method for returning the total bytes downloaded to HttpFetcher
and all subclasses, needed for unit testing. (c) Generalized check for
successful HTTP response code in LibcurlHttpFetcher.

BUG=chromium-os:9648
TEST=unit tests

Change-Id: I46d72fbde0ecfb53823b0705ce17f9547515ee61
Reviewed-on: https://gerrit.chromium.org/gerrit/11773
Tested-by: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
7f9aea2748370b0a26e1d5c36db7d8bbf3ba1245 12-Sep-2011 Bruno Rocha <bpontes@google.com> Add UMA metrics for detecting and reporting update certificate changes.

LibcurlHttpFetcher checks if the update server certificate has changed
since last update, and stores an eventual report in prefs. UpdateCheckScheduler
submits to UMA reports from the previous update.

BUG=chromium-os:19842
TEST=Included unittest for the new class and tested locally on an Alex device,
since we need to verify against the actual server certificates.

Change-Id: I5bee5d648982cd7618db09b67d5bff377eaa1fc1
Reviewed-on: http://gerrit.chromium.org/gerrit/7565
Reviewed-by: Bruno Pontes Soares Rocha <bpontes@chromium.org>
Tested-by: Bruno Pontes Soares Rocha <bpontes@chromium.org>
Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
Reviewed-by: Darin Petkov <petkov@chromium.org>
/system/update_engine/libcurl_http_fetcher.cc
e72f9c028fe41052cc18c95411fc1cd21d5029a3 20-Apr-2011 Andrew de los Reyes <adlr@chromium.org> AU: Set connection timeout to 30 seconds.

A slow connect doesn't count toward the timeouts we had, so they could
drag on for many minutes. This sets it to 30 seconds.

BUG=chromium-os:14072
TEST=tested on device with captive portal

Change-Id: I83ae9880e9434b95582f7f93321db9afba6e7585

Review URL: http://codereview.chromium.org/6883076
/system/update_engine/libcurl_http_fetcher.cc
2a0fd7439504216a842587887b6e62f04296cae7 06-Apr-2011 Andrew de los Reyes <adlr@chromium.org> AU: Lower HTTP timeout on official builds.

We grew the timeout from 90s to 3 min a while back to give developers
more time to let the dev server build images. Unfortunately, this
means official builds take longer to time out when they have a bad
HTTP connection. This CL restores the 90s timeout on official builds.

BUG=chromium-os:13793
TEST=on device testing

Change-Id: Ia7a6474331770b3db3425147921d24077452c097

Review URL: http://codereview.chromium.org/6804001
/system/update_engine/libcurl_http_fetcher.cc
f3ed8e7d4ad456a3c841d8a530f308b6fcb2a4cc 16-Feb-2011 Andrew de los Reyes <adlr@chromium.org> AU: Make proxy resolution asynchronous.

This doesn't change proxy resolution overall (we still use settings
stored in the session manager), but it changes the implementation in
the updater to be asynchronous. The clients of the proxy resolver now
give a callback to be called when the proxies are known.

This is anticipation of a switch to using Chrome to resolve proxies,
which will need to be asynchronous.

BUG=chromium-os:12079
TEST=unittests; tested update on device w/ and w/o proxy settings

Review URL: http://codereview.chromium.org/6516026

Change-Id: Icc5c08e3abf4381be55d8d555020d4c630a07fd6
/system/update_engine/libcurl_http_fetcher.cc
819fef2e0fa08984cf31e848e704442c500ea4f9 17-Dec-2010 Andrew de los Reyes <adlr@chromium.org> AU: MultiHttpFetcher cleanup/rewrite

This is the first of many CLs to cleanup/refactor/unfork the
HttpFetcher classes.

This CL changes MultiHttpFetcher to MultiRangeHTTPFetcher, makes it
work with a single base fetcher, and un-templatizes it.

Also, fix a (new?) bug in SConstruct w/ setting CCFLAGS.

TEST=unittests, tested an interrupted/resumed update on device.
BUG=10395

Review URL: http://codereview.chromium.org/5835004

Change-Id: I8422358a6d425233987dd799c5ee7c87135d85fd
/system/update_engine/libcurl_http_fetcher.cc
3367619e28a22301a6074862028cefd322012dd9 30-Nov-2010 Andrew de los Reyes <adlr@chromium.org> AU: Disallow Anonymous Diffie Hellman ciphers

BUG=9062
TEST=tested that things didn't break on device/unittest

Review URL: http://codereview.chromium.org/5406001

Change-Id: Ic5832cbef0fbfc3dcf61f7d24b468e77f61d9870
/system/update_engine/libcurl_http_fetcher.cc
5d0783d4a9dc230aebdf89923d31b51fe1def5f1 30-Nov-2010 Andrew de los Reyes <adlr@chromium.org> AU: Always retry on failed connections, even manual checks.

This is a temporary workaround for a bug discovered w/ httpdate and
the oobe.

BUG=9736
TEST=unittests

Change-Id: Ic7bbc867be0298d01686a9ea56ce9b012def5f7b

Review URL: http://codereview.chromium.org/5395004
/system/update_engine/libcurl_http_fetcher.cc
a092955cdf763260563a774f6a68e1bfb38c5a8b 29-Nov-2010 Darin Petkov <petkov@chromium.org> AU: Retry up to 3 times 30 seconds apart on HTTP response code 0.

This is a temporary way to avoid failing to do an update check on resume by
allowing at least 1.5 minutes for the network to go online.

BUG=9705
TEST=tested on device through suspend/resume; unit tests

Change-Id: I291a1c31ce87c17d5dce0e30488d454d7690ddbc

Review URL: http://codereview.chromium.org/5260004
/system/update_engine/libcurl_http_fetcher.cc
4516810fe41a39c0c55d2095679898787259ae38 22-Nov-2010 Andrew de los Reyes <adlr@chromium.org> AU: Manual proxy support

Utilize the ChromeProxyResolver to resolve proxies in our network
requests. This means the following changes:

- HttpFetcher classes take a ProxyResolver* in their ctor. Also, a few
useful functions in HttpFetcher to allow subclasses to iterate
through the proxies.

- LibcurlHttpFetcher support for using the ProxyResolver. It will
attempt to use each proxy in the order specified. If any data comes
in from any proxy, it won't continue down the list and will continue
to use that proxy for its lifetime.

- UpdateAttempter can choose, for a given update session, whether or
not to use the ChromeProxyResolver or DirectProxyResolver. For now,
the logic is: for automatic checks, 80% of the time use
ChromeProxyResolver, 20% DirectProxyResolver. For manual checks, the
first 19 manual checks in a row use Chrome, then once it uses
Direct, then starts over again. The idea is that the updater doesn't
necessarily trust Chrome, so some requests should skip it. If a
manual check is performed, the user likely wants her proxy settings
honored, so use them, but don't allow frequent manual checks to
starve out usage of the DirectProxyResolver.

- Updates to tests

BUG=3167
TEST=unittests, tested on device

Review URL: http://codereview.chromium.org/5205002

Change-Id: Iee0f589e5b28d4b804afe1f5b6729ba066d48d62
/system/update_engine/libcurl_http_fetcher.cc
b2bf1228d03c094427ab05f19b73735a737b0bfc 19-Nov-2010 Ken Mixter <kmixter@chromium.org> update_engine: Move CA certs to separate ebuild

Change-Id: If8ffe94e01ceeaeb6d0677d7bdf3019f9a247db0

BUG=8938
TEST=Manually verified same installation point

Review URL: http://codereview.chromium.org/5231001
/system/update_engine/libcurl_http_fetcher.cc
9ce452b6acd95899e9adb17c2935012625163dc6 17-Nov-2010 Darin Petkov <petkov@chromium.org> AU: Fix potential issues with premature destruction of HTTP fetchers.

This patch adds a new TransferTerminated callback to the
HttpFetcher class. It fixes two potential memory corruption
issues with premature destruction of HttpFetcher instances:

1. When MultiHttpFetcher completes a range, it terminates
the current fetcher and starts the next one, if any. Change
so that the next fetcher is started when the
TransferTerminated callback is received from the current
fetcher. This prevents the multi fetcher from sending a
TransferComplete/TransferTerminated callbacks before the
underlying fetcher is cleaned up, which may lead to the
fetchers being destroyed prematurely.

2. If the download action fails due to a failed write,
terminate the transfer and then wait for the transfer
terminated callback before notifying the action processor
that the action is complete. Otherwise, the action may get
destroyed before the transfer is actually terminated
possibly leading to memory corruption, etc.

Hopefully these changes fix crosbug.com/8798.

BUG=8798
TEST=unit tests, tested on device with write errors

Change-Id: If416b95625ab31662f2e1308df6bdd1757a2ad78

Review URL: http://codereview.chromium.org/5009009
/system/update_engine/libcurl_http_fetcher.cc
e237d19031bbea506931f60708200e71008151e1 16-Nov-2010 Darin Petkov <petkov@chromium.org> AU: Restrict SSL ciphers to HIGH in official builds.

BUG=6407
TEST=unit tests, tested on device with the update server

Change-Id: I72de989003c9177f23b1afddde051d9b8d8efa68

Review URL: http://codereview.chromium.org/5085002
/system/update_engine/libcurl_http_fetcher.cc
60e1415b31b58bc17017d29c8e06f6e5452fdf4e 28-Oct-2010 Darin Petkov <petkov@chromium.org> AU: Watch for writes on write fds only, and for reads on read fds.

This reduces the CPU load from 100% to 0% when staying idle waiting for a server
response.

BUG=8200
TEST=unit tests, tested on device

Change-Id: I07216076753b4b0dea9672470f1a6d572482f9a2

Review URL: http://codereview.chromium.org/4190009
/system/update_engine/libcurl_http_fetcher.cc
fc7a0cec794ab43e66b6781ddb7f498b0ce257b9 25-Oct-2010 Darin Petkov <petkov@chromium.org> AU: Restrict to HTTPS for official builds.

Also, fix multi http fetcher unit tests to predictably force non-expensive
connections.

BUG=7945
TEST=unit tests; tested on device with dev server as well as with
no /root/.dev_mode, dev server, omaha update

Change-Id: Iacc0188b464ec018fc0dbbc8d7d447386113ceb7

Review URL: http://codereview.chromium.org/4004004
/system/update_engine/libcurl_http_fetcher.cc
d57d1474a7bb152adf10d8c31d7ef6d40d27f7cc 21-Oct-2010 Andrew de los Reyes <adlr@chromium.org> AU: Don't use network on expensive connection types

Specifically:

- FlimFlam proxy class to query the current network status and find
out if it's expensive.
- Dbus Interface, so dbus can be mocked.
- Libcurl change to redirect the URL if we are on an expensive
connection. This may seem hacky, but the reason I avoided retooling
the whole class is that we may decide that some network usage is
okay on pricy connections. Perhaps it's okay to throttle. So, for
now this is a more minimal change.

BUG=chromium-os:2397
TEST=Unit tests, tested that flimflam proxy works on the device.

Review URL: http://codereview.chromium.org/4029002

Change-Id: Ic4dcde1ca863bda890bc46a55c552e2b32d9433d
/system/update_engine/libcurl_http_fetcher.cc
3fd5d30433509859bfdcc0b650e242981410c6a7 08-Oct-2010 Andrew de los Reyes <adlr@chromium.org> AU: MultiHttpFetcher, an HttpFetcher for specific byte ranges

MultiHttpFetcher takes an HttpFetcher class via template parameter,
and a collection of byte ranges. It hits up the URL multiple times,
once per range specified. For each time, it uses a new HttpFetcher of
the type specified and fast-forwards to the offset requested, and
aborting after enough bytes have been downloaded. Any range many
specify a length of -1, which means until the end of the file (as
dictated by the server). Thus, a single range of [0, -1] makes
MultiHttpFetcher a pass-through.

HttpFetcher change: ability to supply an offset.

LibcurlHttpFetcher changes: offset support (from HttpFetcher API),
ability to be terminted in a write-callback.

test_http_fetcher: support for failures in write() on the socket (at
least in the /big url case).

BUG=7391
TEST=unittests

Review URL: http://codereview.chromium.org/3591018
/system/update_engine/libcurl_http_fetcher.cc
3a4016aefdf84b55ff04571fd005c7967efb243f 28-Sep-2010 Darin Petkov <petkov@chromium.org> AU: Restrict the CA certificates to a smaller trusted set.

BUG=1969
TEST=unit tests, gmerged on device, updated successfully from
https://tools.google.com/service/update2; removed certs from directory
and update failed as expected

Change-Id: I18a04b0222a29249347aae56315bc35170063626

Review URL: http://codereview.chromium.org/3475026
/system/update_engine/libcurl_http_fetcher.cc
cb466212cccca21dfe0dcd01afbbb06e005309e2 26-Aug-2010 Darin Petkov <petkov@chromium.org> AU: Expose the server's HTTP response code in HttpFetcher.

A step towards resolving 2394 -- we'll need to exponentially back
off on 500 and 503 as well as 502 if possible.

BUG=2394
TEST=unit tests, gmerged and made sure updates can happen

Change-Id: I7928e3af37f23ce1ba197315ec52ab0b2ed0dc4c

Review URL: http://codereview.chromium.org/3106038
/system/update_engine/libcurl_http_fetcher.cc
41c2fcf4525a3b4e8960c4b71dfe4f4f11c8224f 25-Aug-2010 Darin Petkov <petkov@chromium.org> AU: Support redirects.

BUG=5692
TEST=unit tests, gmerged on device, making sure update engine still works

Change-Id: If593f6efbd39452aa98b453f4d5489ce7d7d7fb9

Review URL: http://codereview.chromium.org/3161041
/system/update_engine/libcurl_http_fetcher.cc
b83371f6d7fc12a66d33eac614c0fae3aa0978ba 17-Aug-2010 Darin Petkov <petkov@chromium.org> Increase retry timeout to 1 minute.

Also, reduce the retry timeout in unit tests -- this speeds them up by
~80 seconds. Some cleanup of the libcurl perform timeout.

BUG=5576
TEST=unit tests, gmerged on device, ran update, looked at logs

Change-Id: Ifd554913cf437c43c481950897e7a5067fadee01

Review URL: http://codereview.chromium.org/3187005
/system/update_engine/libcurl_http_fetcher.cc
9b11165c7bb64843118fc9e1c9c4746137271107 16-Aug-2010 Darin Petkov <petkov@chromium.org> AU: Switch GLib's timeout sources from milliseconds to seconds.

There's no need for the extra resolution and seconds is preferred.

BUG=5577
TEST=unit tests, gmerged on device and saw checks at 2 and 30 minutes.

Change-Id: I6d64774d5a0e8961fbb9868edc42e62eb794ba11

Review URL: http://codereview.chromium.org/3183004
/system/update_engine/libcurl_http_fetcher.cc
790e62e2d8114647b0785574bea361cf931c482b 12-Aug-2010 Chris Masone <cmasone@chromium.org> [update_engine] Update to use not-hideously-old logging API from chrome

BUG=None
TEST=compile

Change-Id: Ieed8952b6806a72b59e16c1579f7293b6a98157a

Review URL: http://codereview.chromium.org/3170010
/system/update_engine/libcurl_http_fetcher.cc
192ced474a4226f1921fcdaa6a763f5f2d7e1b49 24-Jul-2010 Darin Petkov <petkov@chromium.org> Increase HTTP request timeout to make dev server happy.

BUG=none
TEST=unit tested, gmerged and tested with dev server

Review URL: http://codereview.chromium.org/3035024
/system/update_engine/libcurl_http_fetcher.cc
cb319330c529b0394f6efb416dbe7b03bf38b19b 19-Jul-2010 Andrew de los Reyes <adlr@chromium.org> AU: minor fixes requested in http://codereview.chromium.org/3010009

These were too minor to fix at the time, since the CL was needed
urgently.

Review URL: http://codereview.chromium.org/3021003
/system/update_engine/libcurl_http_fetcher.cc
fb4ad7d5e8c5946fc475f3ee289b38287f015326 19-Jul-2010 Andrew de los Reyes <adlr@chromium.org> AU: HTTP success is not just 200, but anything in the 2xx range

BUG=None
TEST=attached unittest

Review URL: http://codereview.chromium.org/3036005
/system/update_engine/libcurl_http_fetcher.cc
9bbd18757660a09fb8831147b17916df8a3212e5 16-Jul-2010 Andrew de los Reyes <adlr@chromium.org> AU: When server dies, don't retry forever

BUG=4871
TEST=attached unittests

Review URL: http://codereview.chromium.org/3010009
/system/update_engine/libcurl_http_fetcher.cc
3270f7411f55b872db385d0edffdfed18a684121 16-Jul-2010 Andrew de los Reyes <adlr@chromium.org> AU: Changes for deltas on traditional bios machines.

BUG=None
TEST=Attached unittests/tested on image

- Fix uninitialized variable err in action processor unittest

- Let Omaha dictate if an update is a delta or full update

- Bug fix in delta generator for differently-sized images

- More logging when applying delta updates

- Fix infinite loop in http fetcher unittest

- log each HTTP connection to know when a dropped connection is
reestablished.

- Detect when speed goes below a threshold and reestablish HTTP
connection (currently < 10bytes/sec for 90 contiguous seconds).

- Fix stack overflow in libcurl http fetcher.

- optimize out a lot of needless CPU usage in libcurl http fetcher
(turns out adding a glib main loop source uses a lot of CPU).

- subprocess: pass PATH, log stdout/stderr

- postinstall runner: support for ext3 and ext4 target filesystems.

Review URL: http://codereview.chromium.org/2805027
/system/update_engine/libcurl_http_fetcher.cc
c98a7edf648aad88b3f66df3b5a7d43d6a6d7fa9 04-Dec-2009 adlr@google.com <adlr@google.com@06c00378-0e64-4dae-be16-12b19f9950a1> AU: Beginnings of delta support

- proto file for delta files; still needs hardlink support

- code to generate a delta update from two directory trees (old, new).

- code to parse delta update

- Actions: postinst-runner, install, bootable flag setter, filesystem
copier, Omaha response handler, Omaha request preparer,

- misc utility functions, like StringHasSuffix(), templatized Action
classes to feed/collect an object from another action.

- FilesystemIterator: iterates a directory tree with optional
exclusion path. Tolerates deleting of files during iteration.

- Subprocess class: support for synchronously or asynchronously
running an external command. Doesn't pass any env variables.

- Integration test that strings many Actions together and tests using
actual Omaha/Lorry. Currently only tests full updates.

- New simple HTTP server for unittest that supports fake flaky
connections.

- Some refactoring.

Review URL: http://codereview.chromium.org/466036


git-svn-id: svn://chrome-svn/chromeos/trunk@334 06c00378-0e64-4dae-be16-12b19f9950a1
/system/update_engine/libcurl_http_fetcher.cc
d2e4ccc22cd4dc1df69bb242321edde4af933e20 05-Nov-2009 seanparent@google.com <seanparent@google.com@06c00378-0e64-4dae-be16-12b19f9950a1> Die glog, die!

Review URL: http://chromereview.prom.corp.google.com/1181050

git-svn-id: svn://chrome-svn/chromeos/trunk@171 06c00378-0e64-4dae-be16-12b19f9950a1
/system/update_engine/libcurl_http_fetcher.cc
49fdf1889b965be25f929eeebc5b60cd40b90435 10-Oct-2009 rspangler@google.com <rspangler@google.com@06c00378-0e64-4dae-be16-12b19f9950a1> Almost there...



git-svn-id: svn://chrome-svn/chromeos/trunk@24 06c00378-0e64-4dae-be16-12b19f9950a1
/system/update_engine/libcurl_http_fetcher.cc