History log of /frameworks/base/cmds/statsd/Android.mk
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
876889cb760e99221c2fd77c7d67d5409bda0bc7 02-May-2018 Yao Chen <yaochen@google.com> Add cmd to let statsd print all logs it received for debugging.

It only works on eng build. And all code is behind a build flag, so the
code will be stripped out in production builds.

Bug: 78239479
Test: manual
Change-Id: I20ee51822d18e6c77ca324a5327712cbed09593e
/frameworks/base/cmds/statsd/Android.mk
9def8e3995b1034d943a0fd22b6e512bfacdab77 17-Apr-2018 Yangster-mac <yanglu@google.com> Reduce statsd log data size.

1. Hash the strings in metric dimensions.
2. Optimize the timestamp encoding in bucket.
Use bucket num for full bucket and millis for
partial bucket.
3. Encode the dimension path per metric and avoid
deduping it across dimensons.

Test: statsd test
Change-Id: I18f69654de85edb21a9c835c73edead756295e05
BUG: b/77813755
/frameworks/base/cmds/statsd/Android.mk
49954cd681fc4c1013c39cd502408af770286730 18-Apr-2018 Yao Chen <yaochen@google.com> Add socket listener to statsd. And remove Davey atom.

+ The socket listener is behind a flag. It's disabled until we get sepolicy changes in.
+ Data parsing code is from logd, because we use the same format.

+ Removed Davey from JankTracker because it violates our new sepolicy

Test: manually
Bug: 78239479

Change-Id: Ib17729fbc362cdb13385f780e2d636a95adf9bc3
/frameworks/base/cmds/statsd/Android.mk
15f6bbc24f4d7a3d4481d90a18df33d402ddacba 08-Apr-2018 Yangster-mac <yanglu@google.com> Flush the bucket when creating the metric producer.
Use int64 for value field.
E2e test for gauge/value metric.

BUG: b/74445671

Test: statsd test.
Change-Id: I823a0bade8f89834bdfb9cf48864852a47d7b63b
/frameworks/base/cmds/statsd/Android.mk
bd12527c90d55eefa657e6a71cfdd287ecdb4ab3 05-Apr-2018 David Chen <dwchen@google.com> Fix uid map to be simpler and fix partial bucket.

The previous scheme captured periodic snapshots for each config with
complex logic that's unnecessary and wasted memory. We actually don't
need to store any snapshots since we just convert the current state
into a snapshot and also include the deltas (change events) since the
previous report until now.

To make the system more robust, we also include up to 100 of the
deleted apps in the uid map.

Also, fix the wiring of the partial buckets so the metric producers
form partial buckets on both app upgrade and removal, but not on
installation of a new app.

Also, we update StatsCompanionService to also include disabled apps.

Bug: 77607583
Test: Verified unit-tests pass and added new e2e tests.
Change-Id: I98e1f544d6e6571545ae1581c4cebab807596f51
/frameworks/base/cmds/statsd/Android.mk
b142cc8add29c8c97f6134d35873d23db666027c 31-Mar-2018 Yangster-mac <yanglu@google.com> Statsd config TTL

Roughly check the config every hour to see whether the ttl expired.
If so, read the config from disk and recreate the metric manager.

Test: statsd test

BUG: b/77274363

Change-Id: I16838afe5bbe966c3a0f638869751f9b59a5a259
/frameworks/base/cmds/statsd/Android.mk
684d195227bf9eaeb21698cbdcf865afb570c454 25-Mar-2018 Yangster-mac <yanglu@google.com> E2e test for periodic alarm.

Test: new test

BUG: b/76281156
Change-Id: I60cb28baaeec6996e946a7cb3358ec8e0aca80e5
/frameworks/base/cmds/statsd/Android.mk
cdc9d9008bc2ccbf35d345557b64c3f0d2f184eb 27-Mar-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Flush the past buckets in anomaly tracker when time jumps forward" into pi-dev
be10ddfe46ba6371bcd02cb57a06782e6b18d371 13-Mar-2018 Yangster-mac <yanglu@google.com> Flush the past buckets in anomaly tracker when time jumps forward

E2e test for count/duration anomaly trackers.

Test: new statsd tests.

BUG: b/74446029

Change-Id: Ia9be0240ba5021d44c1e1f096d67563e9138bb59
/frameworks/base/cmds/statsd/Android.mk
c8ab224b2a7bea493eb3235b10a961fd1c6d548d 24-Mar-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Fix StatsCompanionService sometimes can be null" into pi-dev
aa5b2010152c4042786b558a44a89d25229d8962 21-Mar-2018 Chenjie Yu <cjyu@google.com> Fix StatsCompanionService sometimes can be null

Bug: 75970648
Test: will add cts test for puller alarms
Change-Id: I51b7d13f855d3c8ded8325d7cf0f614531eceea5
/frameworks/base/cmds/statsd/Android.mk
461a75f26998667bb3226dfd74235eb3fd55c024 22-Mar-2018 Yao Chen <yaochen@google.com> Remove unsigned-integer-overflow signed-integer-overflow from sanitizer.

Sanitizer is picking on the protobuf generated code. Need to figure out why
Disable them to unblock development first.

Bug: 76117312
Test: manual
Change-Id: I3f3a39dfe933f4c1b1cf4063d621392b66f0adea
/frameworks/base/cmds/statsd/Android.mk
dc5677336c35c44e26592dfeefe160a4a9c4bbe0 21-Mar-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Enable sanitizer on statsd eng build." into pi-dev
cd7f90053dab8d50120d09949b758978c5dbc37a 21-Mar-2018 Yao Chen <yaochen@google.com> Enable sanitizer on statsd eng build.

Bug: 74946342

Test: manual
Change-Id: I287def435d077670814db50e4d8398ecee539b74
/frameworks/base/cmds/statsd/Android.mk
9fdd40302e0f4409b2fd4e5a418037c07b42bbe5 20-Mar-2018 David Chen <dwchen@google.com> Allow statsd to be given empty config.

Statsd clients may want to set an empty config temporarily, so it's
more convenient to allow them to set an empty config instead of
having to use the removeConfig and then having to remember to call
StatsManager#setDataFetchOperation.

Test: Added unit-tests and check they pass on marlin-eng.
Bug: 74997752
Change-Id: I2e762e5ec01e5a2c9a3469fb330b53fefbd734d6
/frameworks/base/cmds/statsd/Android.mk
f384b900498b552c776931c8873e4bd03af36fe5 15-Mar-2018 David Chen <dwchen@google.com> Removes stats_log proto from uid map in statsd.

We don't need to parse the proto of uid map, so we use the
ProtoOutputStreame class to generate the binary form of the proto
output that's needed for parsing the uid map data.

Test: Verified unit-tests still pass.
Bug: 74010813
Change-Id: Ia2f7572f3b78bb6f7b60e8b14cf5d65428469ab6
/frameworks/base/cmds/statsd/Android.mk
c7ce1a09783e7b1d32576d381e9bfe312c57c089 14-Mar-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Add wall clock timestamp for ConfigMetricsReport and gauge atoms." into pi-dev
3fa5d7fb236f263125bc6364ea693e5e9f122976 11-Mar-2018 Yangster-mac <yanglu@google.com> Add wall clock timestamp for ConfigMetricsReport and gauge atoms.

Fix the bug when serializing multiple atoms in gauge metric

BUG: b/74159560

Test: new test for ALL_CONDITION_CHANGES sampling method.
Change-Id: I6d33c1efbac92b6e13be2d64c323e090cb1f84aa
/frameworks/base/cmds/statsd/Android.mk
b19425e01113f3341c48f5283affd386658281b0 08-Mar-2018 Yao Chen <yaochen@google.com> Remove unused shared lib from statsd

Test: build statsd, and statsd_test

Bug: 72129300
Change-Id: I0ebff977dabe796799a0d41c64adb2c2f2e9035e
/frameworks/base/cmds/statsd/Android.mk
13fb7e4eeaf7aee408821afe7ee55a5167e49e59 08-Mar-2018 Yangster <yanglu@google.com> Statsd cpu optimizaton

- cache the changed dimensions in condition tracker.
- avoid query condition wizard when unnecessary.
- avoid copy dimension keys in condition key generation.

Test: statsd tests.

BUG: b/73959649
Change-Id: I17d68e2a82643de3f421309841e75f84c6fd8f43
/frameworks/base/cmds/statsd/Android.mk
ec67661bdfe1814039f625575176678219acc7b1 07-Mar-2018 Chenjie Yu <cjyu@google.com> Refactor cpu stats pullers

use same cpu stats readers with BatteryStats so that
1) both throttle to avoid too frequent pulls
2) cached value is served within throttle interval to avoid double
pulling by both statsd and BatteryStats

To run unit tests:
bit
FrameworksCoreTests:com.android.internal.os.KernelUidCpuFreqTimeReaderTest
bit
FrameworksCoreTests:com.android.internal.os.KernelUidCpuClusterTimeReaderTest
bit
FrameworksCoreTests:com.android.internal.os.KernelUidCpuActiveTimeReaderTest

make -j56 statsd_test && adb sync data && adb shell
/data/nativetest64/statsd_test/statsd_test

Test: cts test, unit test
Bug: 73745189
Bug: 73780619
Bug: 73360959

Merged-In: I10a9bc91ca67fa812f4cd71c4fbd73c1a5ba580e

Change-Id: I10a9bc91ca67fa812f4cd71c4fbd73c1a5ba580e
/frameworks/base/cmds/statsd/Android.mk
644e15d5bd476009178bdfb16e0a4ed94255f3e6 02-Mar-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Reduce statsd binary size from 730KB-> 664KB"
580ea321b16c71ddec515553761d6f37359bd3a0 26-Feb-2018 Yao Chen <yaochen@google.com> Add StateTracker.

StateTracker is a special condition tracker that's based on a state atom.
State atoms are annotated in atoms.proto.

The rules for StateTracker:
1. must not have "stop". must have "dimension"
2. must be based on a state atom.
3. it must have the all primary fields and the exclusive state field in its dimension.

For example UidProcessStateTracker, will have output dimension {uid, state}.

Test: unit tests added.
Change-Id: I6b77e58e9fabe61f7326daf929577d8b2cfbf27b
/frameworks/base/cmds/statsd/Android.mk
20e9e6231a1aba79b4e5ae47f3ccfb066920e60f 28-Feb-2018 Yao Chen <yaochen@google.com> Reduce statsd binary size from 730KB-> 664KB

1. StatsdStats does not use the proto object in memory anymore.
2. lite_static -> lite
3. don't use sstream

Bug: 72129300
Test: statsd_test

Change-Id: I8a5adaf222d4d5034e8bf115215fb6dd5f042cac
/frameworks/base/cmds/statsd/Android.mk
03b91d77c4eccd2d088a12e6fe747240a09b573d 28-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Alarm: wakes up statsd and notifies the subscribers."
932ececa1674c59a8da9f3e32d2651e781b86fc4 01-Feb-2018 Yangster-mac <yanglu@google.com> Alarm: wakes up statsd and notifies the subscribers.

Test: manually tested it.
Change-Id: Id796a68976aeb1611183023ba4e9c6a8b8c44bb8
/frameworks/base/cmds/statsd/Android.mk
dcd6ae8bc8f54050a150c18f7a9923fb5d4d19cf 27-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Atom: TemperatureReported"
4029831e36ef1e4a017bf55e1ec8f8b64849c5a3 16-Feb-2018 Tej Singh <singhtejinder@google.com> Atom: TemperatureReported

Makes the temperature reported atom pulled, and adds CPU, GPU, and SKIN
temperatures. Pulls information from the thermal hal.

Test: CTS test on cl in this topic
Change-Id: I0a8e2d1135bdd77e1cc510f24ff5214ce9e14ead
/frameworks/base/cmds/statsd/Android.mk
3b3d38d99645dbbfd55ab160acf2718d8245efaf 24-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Avoid creating temporary objects in FiterValue()."
f520492279ce5c0a029c36b5251031a66f1413fb 23-Feb-2018 Yangster-mac <yanglu@google.com> Avoid creating temporary objects in FiterValue().

It reduces the cpu time from 1000ns to 750ns

Test: statsd test.
Change-Id: Ifa7e98e3368f8d55f85c7b09d05a6c416482981d
/frameworks/base/cmds/statsd/Android.mk
9c1debe330006b2b4d5c08a55905789c57369609 19-Feb-2018 Yao Chen <yaochen@google.com> Add annotation to atoms that represent a state change in atoms.proto

+ A state change atom can have one exclusive state field, and any
number of primary key fields.

When there is primary key in the atom, it means the state belongs to the primary key.
For example,
message UidProcessStateChanged {
optional int32 uid = 1 [(stateFieldOption).option = PRIMARY];
optional android.app.ProcessStateEnum state = 2 [(stateFieldOption).option = EXCLUSIVE];
}

When there is no primary key fields in the atom, the state is global.
For example,
message ScreenStateChanged {
optional android.view.DisplayStateEnum state = 1 [(stateFieldOption).option = EXCLUSIVE];
}

+ The annotation is consumed by stats_log_api_gen to generate a static map from the state
atoms to its primary fields, and exclusive fields

+ stats_log.proto is splitted into 2 proto files, because statsd needs proto lite, and c++
lite proto library cannot properly ignore the field options which requires full proto.

This CL doesn't change any logic in the statsd yet. A separate CL will use the field option
information to correctly track the state.

Test: added unit tests in stats_log_api_gen_test. and statsd_test pases.
Change-Id: I9e8a979fe81ba60efd4d854bb7087ce4b2b147ec
/frameworks/base/cmds/statsd/Android.mk
4c959cb99eb7e71e5417a61f5429c5fa0073e826 13-Feb-2018 Yao Chen <yaochen@google.com> Remove unused variables in statsd, and make more warnings show.

Test: statsd_test

Change-Id: I2c7b674cb615f22c5de90c2de5f2d58108ab2e7f
/frameworks/base/cmds/statsd/Android.mk
e8cf6110aed233019a41fddd4fdca0cc93bb8433 12-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Statsd CPU optimization."
8a8d16ceea1e5b7a2f8c41e17b5d993035f50f5d 08-Feb-2018 Yao Chen <yaochen@google.com> Statsd CPU optimization.

The key change is to revamp how we parse/store/match a log event, especially how we match repeated
field and attribution nodes, and how we construct dimensions and compare them.

+ We use a integer to encode the field of a log element. And also encode the FieldMatcher into an
integer and a bit mask. The log matching becomes 2 integer operations.

+ Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then
becoming checking if the underlying integers are equal. The integers are stored contiguously
in memory, so it's much faster than previous tree structure.

Start review from FieldValue.h

Test: statsd_test + new unit tests

Bug: 72659059

Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
/frameworks/base/cmds/statsd/Android.mk
8a1bacaf91039d7f174fcd1760016ad1134c9821 10-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Benchmark: stats_write"
80f9112aecf08845ef0b2b132d721b7ce850970f 01-Feb-2018 Chenjie Yu <cjyu@google.com> Pullers merge isolated process with host

Test: unit test
Change-Id: Idcb4d19e1f5182554b503c5a35fbde4da146835c
Fix: 72230210
/frameworks/base/cmds/statsd/Android.mk
afb36062d5e6d36700147226b5776b4ca8abf922 01-Feb-2018 Yi Jin <jinyithu@google.com> Allow statsd_config be able to set destination value AUTO/EXPLICIT.

Also move incidentd integration to its own files.
clang-format -style=file -i src/anomaly/* and src/subscribers/Incidentd*

Bug: 70239380
Test: manual
Change-Id: I7bfe14d704d9e86d925365a8a21ffed726723e60
/frameworks/base/cmds/statsd/Android.mk
3410634f2c6454626b1ec4b11f5b6f036a1141e2 03-Feb-2018 Tej Singh <singhtejinder@google.com> Benchmark: stats_write

benchmarks how long stats_write takes for boot sequence atom
results: I ran it a few times, and the times ranged from ~18-20us on
marlin

Test: ran the benchmark test
Change-Id: I900ef26ce219301a6d43999fe7be5e4875ae5b8a
/frameworks/base/cmds/statsd/Android.mk
ad73f4132971bae88fec975bdf41ebd3103aa1e6 02-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Add command to clear puller cache"
4d41f291273710cfe0f33ed04c962c964308c5b4 01-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Atom: Remaining and full battery capacity"
e72252b6886096eef308164b830fe84dd3c6c01d 01-Feb-2018 Chenjie Yu <cjyu@google.com> Add command to clear puller cache

Test: manual test
Change-Id: Id1fa48b5a6f2200d22f4a746763f65d0c8d65f72
/frameworks/base/cmds/statsd/Android.mk
bf972d99a9ee383b984137c49ad45a17df0c73d2 11-Jan-2018 Tej Singh <singhtejinder@google.com> Atom: Remaining and full battery capacity

Add pulled remaining battery capacity and full battery capacity atoms.

Test: cts test accompanies
Change-Id: I0f09e9459b56e25cc3ac21e1a5e781daadea0a01
/frameworks/base/cmds/statsd/Android.mk
9369446f0b04945d6674550728ae81196d6fb5c2 23-Jan-2018 Yangster-mac <yanglu@google.com> Support dimension in condition in metric producers.

Test: added e2e tests for count/duration metrics sliced by fields in condition and with/without links.

Change-Id: Ie34deba68e6780abdde458be3f0ce5284e76a1a2
/frameworks/base/cmds/statsd/Android.mk
9b97976e7619f79b37e5e1ab62274862eccefea5 30-Jan-2018 yro <yro@google.com> Re-enable statsd from droidfooders by default

Test: the team-wide bug bash to capture any issues in advance but did
not find any statsd crash

Change-Id: I253e0067a710c37bf10667e3a8dbde99d0b289c7
/frameworks/base/cmds/statsd/Android.mk
181c835f818df011cf829763fffff9f9c1d3e35d 25-Jan-2018 yro <yro@google.com> Disable statsd that was accidentally enabled in ag/3480570

Test: manual test

Change-Id: I57176203a1ca7d0b5639119642335e3e9a1e4e4c
/frameworks/base/cmds/statsd/Android.mk
d0c260ff418f0d1c9d1536eb1420373391116f88 25-Jan-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Adding guardrails on writing to disk from statsd"
73a331f52490f648d0347cd8856c1196d418a600 25-Jan-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Statsd broadcast subscriber"
540aa206e2a7e4a4897b4b92a8cca9e0ddb78d24 25-Jan-2018 Howard Ro <yro@google.com> Merge "Revert "Revert "Revert "Enable init.rc for statsd to start statsd by default in boot time""""
62ac686168337c223f8a63b2ceba2e5c756929a1 25-Jan-2018 Howard Ro <yro@google.com> Revert "Revert "Revert "Enable init.rc for statsd to start statsd by default in boot time"""

This reverts commit b00cce84d9ddba04a3ebd5ed45457c1e154be535.

Reason for revert: b/72459151

Change-Id: I71593b4b323d69edb548dcb1bb0e1f4eb7cddbc9
/frameworks/base/cmds/statsd/Android.mk
c697797d43b6932fc7cca1c8d7d850c3fb4d1452 17-Jan-2018 Bookatz <bookatz@google.com> Statsd broadcast subscriber

Allows a uid that uploads a statsd config to additionally
register a BroadcastSubscriber with statsd. If statsd
detects an anomaly (according to the config's Alert),
statsd can inform a BroadcastSubscriber provided in the config.
The config uses a subscriberId (just an int) to identify the
BroadcastSubscriber. It then uses StatsManager.setBroadcastSubscriber
to associate that subscriberId with a given PendingIntent.
Then, when the anomaly is detected, statsd sends a broadcast
using that PendingIntent, alerting whoever was specified by
the config/setBroadcastSubscriber.

Bug: 70356901
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.alert.BroadcastSubscriberTests
Change-Id: I4d9ea9a6c8a85e61fadfd99c1513c55abbadd5e9
/frameworks/base/cmds/statsd/Android.mk
98a28501fe8ab53a490ec353c8a1f74f2e329cc5 19-Jan-2018 yro <yro@google.com> Adding guardrails on writing to disk from statsd

- Limit total number of files to 1000
- Limit total size of files to 5MB
- Remove idle files to be deleted after 30 days

Bug: 69854160
Test: manual testing, statsd, statsd_test
Change-Id: I33148a3b7ca11d413ec2495d5c0659f1ba4485c3
/frameworks/base/cmds/statsd/Android.mk
48d75189ccbfc06f24f315ff5b8a1a42cf8207c5 23-Jan-2018 Yao Chen <yaochen@google.com> Add statsd microbenchmark and fix a crash in LogEvent

+ We should benchmark the core functions in the critical path.
1. LogEvent parsing
2. Log matching
3. Event processing in various metrics
....

+ Using microbenchmark, we can quantify the optimization we do in the future.

---------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------
BM_LogEventCreation 4772 ns 4705 ns 145738

Test: make -j64 statsd_benchmark

Bug: 72242322

Change-Id: I43aa704ffdc7cd21e02ef9038eff66ca7022dbfb
/frameworks/base/cmds/statsd/Android.mk
a92805d2cf6e18b56352b15b9ef7538741ed30f3 23-Jan-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Revert "Revert "Enable init.rc for statsd to start statsd by default in boot time"""
b00cce84d9ddba04a3ebd5ed45457c1e154be535 22-Jan-2018 Yao Chen <yaochen@google.com> Revert "Revert "Enable init.rc for statsd to start statsd by default in boot time""

This reverts commit 5f3dc878dc0a0fddd4f7b3bc80e1277ffde22c51.

Change-Id: I8d59dcd075d9b84ad44187e7744d125e79400c4c
/frameworks/base/cmds/statsd/Android.mk
e4d44919aa90f16285a9966dcc632c128b932374 10-Jan-2018 Primiano Tucci <primiano@google.com> Statsd -> Perfetto integration

This CL invokes the perfetto client utility to start the
collection of a trace when an anomaly that subscribed to
Perfetto is detected. The code simply spawns the
/system/bin/perfetto client and passes the trace config via
stdin. The client takes care of the dropbox upload.
The CollectPerfettoTraceAndUploadToDropbox() function does
NOT wait for the full trace collection (in order to avoid
blocking statsd) and instead returns immediately after having
spawned perfetto.

Change-Id: I4f02067bad7a46ede7b6e4841cdcf381c1a4e2a7
Bug: 71795552
/frameworks/base/cmds/statsd/Android.mk
d03c598b937ea078108cd81cb1b61775adb1315e 19-Jan-2018 Yao Chen <yaochen@google.com> Merge "Revert "Enable init.rc for statsd to start statsd by default in boot time""
5f3dc878dc0a0fddd4f7b3bc80e1277ffde22c51 19-Jan-2018 Yao Chen <yaochen@google.com> Revert "Enable init.rc for statsd to start statsd by default in boot time"

This reverts commit 56967528d7bcc7a1342d68f19e0ffa79f2a1ed59.

Change-Id: I4b76bf5783f0df43603e82168009f33296d1f840
/frameworks/base/cmds/statsd/Android.mk
c8b7f2277c7d05dc0a1225dbed13ee85f18a6031 12-Jan-2018 Chenjie Yu <cjyu@google.com> consolidate subsystem sleep state atoms

Test: cts test
Change-Id: Ib9ff746eaa17b6c2c8f9fe5860dd0b480ec85186
/frameworks/base/cmds/statsd/Android.mk
004e782ee50e0afff9a1b287900946bb4dc0b389 17-Jan-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Add a statsd proto static lib for cts and others."
87718e283a62660eaa0e39fd780c97a290da988f 12-Jan-2018 Yangster-mac <yanglu@google.com> Gauge metric e2e test

Test: new test passed
Change-Id: I6a8bceb43fbb6e7b82f47951e71b5620779f2ceb
/frameworks/base/cmds/statsd/Android.mk
459dee6c51adc82d6dbcdcb47b5a8f1490d59868 11-Jan-2018 Yao Chen <yaochen@google.com> Add a statsd proto static lib for cts and others.

Test: build
Change-Id: I0119320cba093dc0f758a890ffb9e595b39324ec
/frameworks/base/cmds/statsd/Android.mk
b5bc74122fb95c81bedad625160fb8d144a863cd 07-Jan-2018 Yangster-mac <yanglu@google.com> E2e test: attribution matching and slicing by attribution.

Test: statsd unit tests passed.
Change-Id: I05875b80a9a1ab5f87a478d1f63543f9798254a5
/frameworks/base/cmds/statsd/Android.mk
b038b709566c191164bd23539175df8aadeb2f94 19-Dec-2017 Chenjie Yu <cjyu@google.com> pullers now cache data to throttle frequent pull requests.
all pullers have a default 1s cool down before next pull.
We can adjust these later.
Also add puller stats in StatsdStats

Test: unit test
Change-Id: I71894a24c41e059d841591312dbb852f54387b7d
/frameworks/base/cmds/statsd/Android.mk
d19bcae42df300c080415a1a91df53c4e28ea4d5 05-Jan-2018 Yangster-mac <yanglu@google.com> Move Field proto out of stats_log.proto to keep it clean.

Test: statsd unit test passed.

Change-Id: I31ffea40cf1b82c16bf37c706969bfb159639c7b
/frameworks/base/cmds/statsd/Android.mk
2087716f2bdca90c7c3034d556ac12911bd8018e 23-Dec-2017 Yangster-mac <yanglu@google.com> 1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.

Test: statsd unit test passed.

Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
/frameworks/base/cmds/statsd/Android.mk
857aaa520804d846a050e1eeb85d82977c983666 20-Dec-2017 Bookatz <bookatz@google.com> Splits AnomalyTracker into two files

Splits out DurationAnomalyTracker-specific functions into their own
subclass.

Test: the unit tests and CTS tests
Change-Id: Id6eb74d232b4a9c3a932d805d1ba3f0ba43a88b1
/frameworks/base/cmds/statsd/Android.mk
d5aa01b3716e07463019fa772f07a40613f9e39e 20-Dec-2017 Yao Chen <yaochen@google.com> Remove the hacky HashableDimensionKey.

+ Add a real HashableDimensionKey as a wrapper of the dimension.
So we can get rid of the maps that we kept.

Pay down technical debt and reduce memory usage.

Test: statsd_test & manual
Change-Id: I233280cf1e2ce93da6a8cd4e8514abb066f4016d
/frameworks/base/cmds/statsd/Android.mk
d9dfda76e0aa8b8aa4d2562a7ff4259f4500e355 12-Dec-2017 Chenjie Yu <cjyu@google.com> add feature: GaugeMetricProducer now takes repeated list of fields
bug fix: GaugeMetricProducer now works better with pulled events.
unit test also includes GaugeMetricProducer_test

Test: unit test
Change-Id: Ic60f09342d14cfb107be2130d445b323a56909e0
/frameworks/base/cmds/statsd/Android.mk
c9f1a9e3dd214b167a49ac4560382e2e98084ff6 14-Dec-2017 Joe Onorato <joeo@google.com> Merge "Fix how we build the statsd protos."
62c220b20b3fc431c27feac29a71d040fc8e2626 19-Nov-2017 Joe Onorato <joeo@google.com> Fix how we build the statsd protos.

This lets us include frameworks protos, and use the constants
and messages from them.

Change-Id: I609d6e524f780e6a5beea543a68561bede47813e
Test: make
/frameworks/base/cmds/statsd/Android.mk
2794da220fa853c109905586a87202eff361c175 14-Dec-2017 Yao Chen <yaochen@google.com> Remove dead code in statsd

Remove dead code and shared lib.
And fix a bug in ValueMetricProducer

Test: statsd_test & manual
Change-Id: Ie76bfc02e14ae6a0fa9f8933751fd06397de9411
/frameworks/base/cmds/statsd/Android.mk
56967528d7bcc7a1342d68f19e0ffa79f2a1ed59 12-Dec-2017 yro <yro@google.com> Enable init.rc for statsd to start statsd by default in boot time

Test: manual testing

Change-Id: I1fea9c8a89216df72a9fec6f2f8589170561215a
/frameworks/base/cmds/statsd/Android.mk
d9269e2ee70b01cad3dda14c786e0e4434192248 05-Dec-2017 David Chen <dwchen@google.com> Adds rate limit to checking byte size.

Since there is a separate guardrail for memory used by uid map, we
no longer add the memory from uid map with the memory per each
config's metrics. We also prevent the byte size check from happening
too frequently. In order to mock the MetricsManager, we refactor
some of the existing methods.

Test: Added unit-tests and verified they all pass on marlin.
Change-Id: I15cf105f7d95f4016fdb0443b0a33eebe862cafb
/frameworks/base/cmds/statsd/Android.mk
f2bee6fec965fd42ab223f1a3aa705f07ba79aea 29-Nov-2017 Yangster <yanglu@google.com> 1/ Only expose thread-safe interfaces in metric producer.
2/ Simplify lock logic.
3/ Add test for duration metric producer.

Test: all unit test passsed.
Change-Id: If6ee2e69a17f12406f4b3ea3553b14642cd636d6
/frameworks/base/cmds/statsd/Android.mk
b356151e63140085cb96fa16804ee18b3862a4fc 22-Nov-2017 Yao Chen <yaochen@google.com> Add StatsdStats and guardrail.

+ StatsdStats is the global class that tracks the stats about statsd.

+ Added guardrail for classes that have a map which could potentially grow
unboundedly with the number of logs.

TODO: add unit tests & CTS for StatsdStats, and guardrail
add stats for pulled atoms.

Test: statsd_test

Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
/frameworks/base/cmds/statsd/Android.mk
947fbce521d9e8377df03e3c1c31884ed5577f32 16-Nov-2017 yro <yro@google.com> Captures metrics on disk when devices reboot and shutdown. Specifically,

1. Create intent receiver in StatsCompanionService to listen to shutdown
events.
2. Create StatsWriter class to handle disk writes and deleting files.
3. Update StatsLogProcessor, ConfigManager, and StatsService to handle
files on disk using StatsWriter.
4. Add a wrapper for ConfigMetricsReport.

Still TODO is to be able to add a guardrail to prevent accumulating
excessive amount files on disk, which will be followed up by another
change.

Test: statsd, statsd_test
Change-Id: Ia0b3af315af545daa8b0078b3700c600aa7c285f
/frameworks/base/cmds/statsd/Android.mk
52c4cf38ae980a431175c9d4354134537177d3fc 22-Nov-2017 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Statsd test app."
1ff4f4325882d812ded4f75ee6c2b6d352a85c9e 17-Nov-2017 Yao Chen <yaochen@google.com> Statsd test app.

Features:
+ Push baseline config to statsd
+ Have buttons to write events to StatsLog (plug/unplug, screen on/off, wake locks)
+ Dump stats log report, and display it on screen (currently support DurationMetricData display).

Test: manual

Change-Id: Ia58d215d30ed7570c0aa1b0627d3b88f8355d425
/frameworks/base/cmds/statsd/Android.mk
d215a23179c90eb9f953704fa3ee9c5e9d6dc1df 22-Nov-2017 TreeHugger Robot <treehugger-gerrit@google.com> Merge "The first step towards adding a guardrail."
e2cd6d509b17894b95d14523ae3e7c4c7a9a74e3 10-Nov-2017 Yangster-mac <yanglu@google.com> 1/ Duration anomaly tracker with alarm.
2/ Init anomaly from config based on the public language.
3/ Unit tests for anomaly detection in count/gauge producer.
4/ Revisit the duration tracker logic.

Test: unit test passed.
Change-Id: I2423c0e0f05b1e37626954de9e749303423963f2
/frameworks/base/cmds/statsd/Android.mk
29f69e8f0442f78dba468a15e290fe94729263a5 19-Nov-2017 yro <yro@google.com> Remove IStatsCallback aidl from master

Test: statsd, statsd_test

Change-Id: Idd0fb945b62f3a139a0db07f1014a3c4a4e2f5f8
/frameworks/base/cmds/statsd/Android.mk
8d9989bb376f3937d9c8fef07c9cc65ef78cbcae 19-Nov-2017 Yao Chen <yaochen@google.com> The first step towards adding a guardrail.

Added a malloc debug cmd to find memory leak and where
the most heap allocations are with backtrace.

More info: malloc_debug/README.md

The util class is borrowed from audio server.

Usage:

adb shell stop
adb shell setprop libc.debug.malloc.program statsd
adb shell setprop libc.debug.malloc.options backtrace
adb shell start
adb shell cmd stats meminfo

Also,

adb shell dumpsys meminfo [statsd_pid]

also gives you the overall summary of the memory usage.

Test: manual

Change-Id: I4eb4167466a748ea0c3d9fd39e45d9900bc00fc2
/frameworks/base/cmds/statsd/Android.mk
ae2df01aae2ee7076af875c04e991140f685f733 14-Nov-2017 Stefan Lafon <stlafon@google.com> Rename "stats_events" as "atom".

Also make atom WakelockStateChanged use the "Type" enum instead of int32.

Test: Unittests are passing. Statsd is working.

Change-Id: I0909e2d97297f78996a81366d66aae62d5bf5ce1
/frameworks/base/cmds/statsd/Android.mk
6736c893a73c567d3794d88ae0d17abab34b7a6f 09-Nov-2017 Chenjie Yu <cjyu@google.com> Unit tests for ValueMetricProducer
StatsPullerManager is refactored so that we can mock it.
It may need more refactor pass to make is safer for longer runs.

Test: unit test
Change-Id: Ief0c99710e4d06e1454678f8b749c9599467d114
/frameworks/base/cmds/statsd/Android.mk
e33bc3b967aef6f10df82e5b374e4b701ce6ca69 07-Nov-2017 Chenjie Yu <cjyu@google.com> add CpuTimePerUidPuller and CpuTimePerUidFreqPuller
Note: these pullers return monotonically increasing numbers.
The uids could include isolated uids that needs further translation in
statsd.

Test: manually tested on device and check print outs
Change-Id: I7097aec0417a3ac567b9b86149b757ac27cd58d8
/frameworks/base/cmds/statsd/Android.mk
967b2051fe17fa3c38d248a556f8c4ff21a573a7 08-Nov-2017 Yao Chen <yaochen@google.com> Make SimpleCondition satisfactorily complicated.

+ Support nested counting
+ Support StopAll
+ Added default_condition to SimpleCondition config
+ Some refactoring/clean up
+ Added unit tests

Test: Added unit tests, statsd_test
Change-Id: I6564ac2e068ce6810e8090c0818064c625c7847a
/frameworks/base/cmds/statsd/Android.mk
3eba62186592382ed3d97cecca0c547487e4b2e4 26-Oct-2017 Yang Lu <yanglu@google.com> Update CountAnomalyTracker to handle slicing.

Re-added CountAnomalyTracker (under the name DiscreteAnomalyDetector).
It is now able to handle dimensions.

Test: unit test passed.

Change-Id: I133fcf6db99b9e305756f7ee5891c25f0f276348
/frameworks/base/cmds/statsd/Android.mk
93fe3a34a02c673eaee4a2d18565ba8df20685cb 02-Nov-2017 Yao Chen <yaochen@google.com> Add unit tests for CountMetricProducer, EventMetricProducer

And other miscellaneous fixes.
+ clang-format
+ 2 bug fixes, one in dump-report command, one in ResourcePowerManagerPuller

Test: statsd_test

Change-Id: Ibd164d948ad62adcc529d813df1210781e38be47
/frameworks/base/cmds/statsd/Android.mk
1d4d686001b220e114df737154f143ffabc212f1 31-Oct-2017 Yangster <yanglu@google.com> Gauge metric producer.

Test: manual tests passed for pushed device temperature.
Change-Id: I7592a4c04666606b745cdb41db8f9d8a96a966da
/frameworks/base/cmds/statsd/Android.mk
5305e1ddc22bfe9ad941cb2550987aaba8de234c 31-Oct-2017 Chenjie Yu <cjyu@google.com> pull resource power manager stats into statsd

Test: tested on device and check output
Change-Id: I65932a5527d6099e12546b6928a24f3616e54e16
/frameworks/base/cmds/statsd/Android.mk
b3dda41a5a48ddfa44dc8fa939ab495926a0195d 24-Oct-2017 Chenjie Yu <cjyu@google.com> reretry ValueMetric implementation and pulling mechanism

Note:
This is for value metric. The default operations is sum the diffs.
The test uses kernel wake lock, which also needs dimension by kernel
wake lock name.

The test is a bit cumbersome as it needs StatsCompanionService to do
the alarm, which is not exact alarm.

The internal state of a slice of bucket would look something like this:

4:ipc0000005e_727_android.hardwar
0 0
4:SensorService_wakelock
40 64
4:ipc0000005c_727_android.hardwar
...

Test: manual test on device.
Change-Id: I2ed0ac7d3c5fcba8b7611d46f38a38ffd8bdc92a
/frameworks/base/cmds/statsd/Android.mk
53f801163c3df707d846de8875cf92974d5400a8 01-Nov-2017 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Adding multi layer aggregation in DurationMetric"
5154a379303ab90a2b2914676a4441917a329b5d 31-Oct-2017 Yao Chen <yaochen@google.com> Adding multi layer aggregation in DurationMetric

Newly supported metrics examples:

1) Compute [Total|Max] duration of [an app] holding [ANY] wake lock while [this app] is in
[background] and [screen off], bucket size 30seconds, and slice output by uid.

2) Compute [Total|Max] duration of [ANY app] holding [ANY] wake lock while [this app] is in
[background] and [screen off], bucket size 30 seconds.

+ DurationMetric proto has a "what" which is a SimpleCondition. It defines the atom level start
and stop of the duration timer, and it has its atom dimension. e.g., for wake locks, the atom
dimensions wil be uid and wl name.

+ Now dimension is explicitly specified in SimpleCondition proto instead of inferred from the "link"

+ Added support for "Or" and "Max" through 2 layers of aggregation.

TODO: (1) The way we track slicedCondition in duration metric is not efficient. optimize!
(2) The output dimension should all use int32 instead of KeyMatcher. Fix in a future cl.

Test: Added some unit tests using gmock. Will add more unit tests.

Change-Id: I58a827624f01f9a54fcb80709c4de4ff94a8bc67
/frameworks/base/cmds/statsd/Android.mk
d6896898912eb1fc2c8976dce4340ab8d4e8d22f 25-Oct-2017 David Chen <dwchen@google.com> Updates uidmap to update snapshots and upload.

We send a snapshot of all installed apps with their uids every time
a user is added or removed and when statsd is started.
We keep track of the latest timestamp when a config key has retrieved
the UID map data. This allows us to remove older data when we're
guaranteed that all config sources have retrieved the old data.

Test: Added more unit tests to UidMap_test and passed on marlin-eng.
Change-Id: I34a3d61e75eedec44b98d896d7f6db0bc383f46a
/frameworks/base/cmds/statsd/Android.mk
c3787a8a83034e2d416baa1ce13b6ebeaf639f7e 25-Oct-2017 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Adds aidl definitions and their implementations for binder transfer of statsd entries to clients. This change only includes changes on statds side and does not include java library for clients to import. Java library will be a separate change as it requires system api review."
31eb67b3498d326659b2b164ff367a01a793d641 24-Oct-2017 yro <yro@google.com> Adds aidl definitions and their implementations for binder transfer of
statsd entries to clients. This change only includes changes on statds
side and does not include java library for clients to import. Java
library will be a separate change as it requires system api review.

Test: statsd, statsd_test
Change-Id: I306c6e9687801668cc0145b12d38406bfe634775
/frameworks/base/cmds/statsd/Android.mk
5110bedd787835d4dbc809b7977285cac364bdef 23-Oct-2017 Yao Chen <yaochen@google.com> Add EventMetricProducer

+ Started to use ProtoOutputStream in EventMetricProducer.
[TODO]: We need to auto-generate fieldIds for StatsLogReport, XXXMetricData, etc.
[TODO]: We need to add Enum type to liblog, otherwise we cannot reconstruct a proto containing
an enum

+ Some refactor in metric initialization code. There are still boiler plate code, because Metrics
are similar but with subtle differences.

Test: statsd_test

Change-Id: Id7e3212566249a8139b9680f04238c455d50c1b8
/frameworks/base/cmds/statsd/Android.mk
810a6c53b387491c40ec50ac229c07987673be17 23-Oct-2017 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Refactor MetricProducer so that a big chunk of boiler plate code can be removed."
b704177d401de895c3b00d258885dc4243a7b3a7 21-Oct-2017 Yao Chen <yaochen@google.com> Refactor MetricProducer so that a big chunk of boiler plate code can be removed.

Test: statsd_test
Change-Id: I62489b94c6dfceb0b08ce4147c331a8a785eabb4
/frameworks/base/cmds/statsd/Android.mk
1481fe142d36d5f0b36eeebc358d5a8aef7bf28a 16-Oct-2017 David Chen <dwchen@google.com> Changes pulled data to use Parcel objects.

Previously, pulled data was returned as a string. We instead
return the data as an array of StatsLogEventWrapper, which encodes
using the binary-encoded format liblog uses. StatsD uses the same
parsing as for pushed events to convert these. This CL also fixes
the parsing of log_msg since the strings were previously emptied
before we had a chance to read the values.

Note that the cpp-aidl can't support List of Parcelable, so we
have to return the results as an array.

Test: Manual using the new command in StatsService to print results.
Also created a new unit-test by creating a dummy pull code of -1,
but this test is deleted since it required creating a fake output in
StatsCompanionService.

Change-Id: I1cfb9ea081a59292a60e934e8527adc40982ed80
/frameworks/base/cmds/statsd/Android.mk
729093df0dd7d7038ad43b16ecdb59a1404f3b50 16-Oct-2017 Yao Chen <yaochen@google.com> Add support for dimension, and link with condition and added DurationMetric

Now we support following metrics:
<Duration> of [app holding a wake lock], while [*this app*] is [in background] [AND] [screen is off]
[Slice] the output by [app name, wake lock name], with bucket size [30sec]

+ Also added onDumpReport() api in MetricsManager, it can be called from client to fetch the data from
statsd

+ Also added command line tool to dump the StatsLogReport from all metrics for debugging.

+ Synced proto from google3. with a pending cl (cr/172359050)

TODO: We need to add tons of tests to test the Metrics. I will work on it after this CL so people
can be unblocked.

I locally test the duration metric with wake lock with an app that generates StatsLog events.

Test: statsd_test

and manual test, and run:

adb shell cmd stats dump-report
We have a default config, which contains a metrics to count PROCESS_START event sliced by
package name.

Change-Id: I4838cc6cf025c143b7e84f43040703a78121fd25
/frameworks/base/cmds/statsd/Android.mk
c4dfae56c10a1dd571baa78c750f2e68c919d74f 18-Oct-2017 Joe Onorato <joeo@google.com> Introduce a new wrapper for log_msg -- LogEvent

It stores all of the parsed fields in a single vector, and
provides accessor methods to get at fields by index with
their correct type.

Test: statsd_test
Change-Id: I4fa94e4ce52db3ac87f19b62f9c85398de6e8145
/frameworks/base/cmds/statsd/Android.mk
9fc9edf95a308f5884bf541cac81ce1f41aba0ba 16-Oct-2017 Joe Onorato <joeo@google.com> Clean up how we handle configurations, and other assorted cleanup

- Add a ConfigManager class that tracks the configurations
that have been passed to us. Configurations are now
tracked by tuples of (uid,tag), where the tag is an
app-defined string, in case a single uid has multiple
configurations.
- Move all of the initialization into StatsService.
- Get rid of the ability to have multiple LogListeners. Raw
events are now pushed directly into StatsService, which
can distribute them to the interested parties (and will
eventually be able to do the proper locking).
- Add Log.h, which sets our LOG_TAG correctly.
- Move some of the related files that I expect will grow some
into their own subdirectories.

Test: statsd_test
Test: adb shell cmd stats config ...
Test: adb shell dumpsys stats
Change-Id: I79487603003d8a842d5bd319741f1ecbf72063d1
/frameworks/base/cmds/statsd/Android.mk
70b986376189bf64706924a614d2d44e5702ed9c 17-Oct-2017 Yao Chen <yaochen@google.com> Fix build error.

Test: builds successfully.

Change-Id: I9ffefa28f37830319864e7b98d32340f59df0a62
/frameworks/base/cmds/statsd/Android.mk
d54f9dd6251c7bd7033630b7afa770190d41c409 17-Oct-2017 Yao Chen <yaochen@google.com> Revert "Revert "Start auto-generating the stats log API.""

Test: builds successfully

This reverts commit 931945399859ab91545ba2c2a914f044092d5e2e.

Change-Id: I22bca4a32adf86040b9d72ad5b45999aba28f586
/frameworks/base/cmds/statsd/Android.mk
931945399859ab91545ba2c2a914f044092d5e2e 17-Oct-2017 Joe Onorato <joeo@google.com> Revert "Start auto-generating the stats log API."

This reverts commit b81d1a7b9a38dcb4d356ae3435a82fb52ba7d585.

Change-Id: I189684d8913ffffca42d9514ac88ea5cc4a44f05
/frameworks/base/cmds/statsd/Android.mk
b81d1a7b9a38dcb4d356ae3435a82fb52ba7d585 16-Oct-2017 Joe Onorato <joeo@google.com> Start auto-generating the stats log API.

Both native and java bindings.

TODOs:
- Finish WorkSources.
- Clean up the package names for the protos.
- Put the protos in a more suitable location.

Test: stats-log-api-gen-test
Change-Id: Idf4022225e2be05106dbcf7de8e97a3337fc63e2
/frameworks/base/cmds/statsd/Android.mk
ece5f705d59c6b73005edc7eeaa6953482f7c6f0 09-Oct-2017 Bookatz <bookatz@google.com> AnomalyMonitor can get alarms based on timestamp

When StatsCompanion informs statsd that an anomaly alarm has fired,
statsd will need to check to see whether an anomaly has indeed occurred.
This will require determining which anomaly alarms have times in the
past (since that indicates that the anomaly did indeed occur). This
capability is now added.

Test: adb shell data/nativetest64/statsd_test/statsd_test
Change-Id: I5464c94634df70832d2723d8c4718277776d58aa
/frameworks/base/cmds/statsd/Android.mk
de70169109c57787a23c732ec4b361ade2e9850e 05-Oct-2017 David Chen <dwchen@google.com> UID mapping to provide app name and version.

The UID map is updated by StatsCompanionService, which listens to broadcast
updates indicating that an app was updated/installed or removed. Also,
the entire map is updated when statsd first connects to the companion
service. Also, there is a way for metrics producers to subscribe to
updates, so that they can know when an app was upgraded.

Test: Created new unit-test for mapping and manually tested for install
and remove. Did not manually test the app upgrade.

Change-Id: I6676ae5c93b75c72d9badabb36aa9c40006db07d
/frameworks/base/cmds/statsd/Android.mk
caf339d004fad667748b68912c254df4e75cdc5a 07-Oct-2017 Yao Chen <yaochen@google.com> More complete implementation for condition and log matchers in statsd.

+ also synced proto from google3 to fix the LogEntryMatcher proto

+ MetricsManager represents StatsdConfig, it's responsible for initializing and managing all
LogEntryMatcher, Condition, and Metrics. Start review from here.

+ Added more complete StatsdConfig initialization, including building the map for:
LogEntryMatcher -> Metrics
LogEntryMatcher -> Condition
Condition -> Metrics.

All the maps use index(int). The extra amount of memory for storing mappings help us
quickly process log events.

The StatsdConfig initialization process detects malformed config
- Circle dependency
- Missing definition
etc.

And once we detect ANY error, statsd will reject the config. And the resources related to this
config will be released.

Test: Added unit tests
Change-Id: I2c4aefdbf3e2aa1701eacbb2fb5e653819ec1fbb
/frameworks/base/cmds/statsd/Android.mk
16e9b10aa236d293845924bf0e9d877efb6dd9d2 09-Oct-2017 Chenjie Yu <cjyu@google.com> Merge "refactor statspuller"
4f99ad1e7b192af27da2ddd99b92fd9651dbecb5 09-Oct-2017 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Statsd Anomaly tracking for CountMetricProducer"
1a317baedaa79f015ad637353c81abb337890733 06-Oct-2017 Chenjie Yu <cjyu@google.com> refactor statspuller

Test: manual test on device
Change-Id: Ibdec6a821e47cd2b2e7435002219c0b2e3f4c5d2
/frameworks/base/cmds/statsd/Android.mk
a4bc9c4a1e39574ca83a56cf6cfa75c40059b4cb 04-Oct-2017 Bookatz <bookatz@google.com> Statsd Anomaly tracking for CountMetricProducer

CountMetricProducer now has a CountAnomalyTracker which stores past
bucket information. Anomalies can be determined by seeing if the
information from the past and current buckets exeeds a threshold.

Test: manual
Change-Id: I35103c01dd32dcc31cb155f5685161cbaf969d03
/frameworks/base/cmds/statsd/Android.mk
6214ad9f5f02d21026a1c9e132f9e95a825359fc 02-Oct-2017 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Statsd can pull kernel wakelock data"
c68a9d21b47f4c2fb65139c10d646145a9ab9a73 27-Sep-2017 Bookatz <bookatz@google.com> Statsd can pull kernel wakelock data

When statsd is told that it is time to poll data, it asks
StatsCompanionService to pull kernel wakelock data, receives the result
(as a string), and outputs it to screen.

Still to do:
1. don't use a string; use a parcel instead
2. don't output it to screen; do something useful instead
3. do more than just kernel wakelocks
4. pull data on demand, in addition to just on periodic pulling

Test: added setPollingAlarms to statsd.main and confirmed that kernel
wakelock information was written to screen
Change-Id: I35f5164420699dea1a00c9e530b938904f1d3055
/frameworks/base/cmds/statsd/Android.mk
44cf27c14880005df093f372491c593e1a9a3c58 15-Sep-2017 Yao Chen <yaochen@google.com> Add metric computation skeleton to statsd.

This cl is to let statsd understand statsd_config, and compute metrics
defined in the config.

+ StatsLogProcessor is given a StatsdConfig (hard coded right now).
We construct a MetricProducer for each of the metric, and the metrics
share Condition and LogEntryMatchers

+ Added the CountMetricProducer type for CountMetric.

We can now count times of SCREEN_ON events given a config.

TODO: 1) conditions are not implemented.
2) slicings are not implemented in CountMetric
3) move the interaction to dropbox to a separate thread
4) decide how the in memory metrics would be used by anomaly detection

Test: manual test.

$ adb shell /system/bin/statsd

$ cat config_file.dat | adb shell cmd stats config

Change-Id: I38f4059c0dc5a827c338131d4a6fa7d4cbe865db
/frameworks/base/cmds/statsd/Android.mk
cdb1a0ed959818b0c3822e114c0456833769bade 28-Sep-2017 Stefan Lafon <stlafon@google.com> Check in new protos and constants.
Test: Started statsd and verified it outputs data. Also ran statsd tests.

Change-Id: I2a438b2ddfcb1576e21acb6159bea607fed7caaa
/frameworks/base/cmds/statsd/Android.mk
dd89694f5e843963b992a09317aa58fa4a9d7a8d 26-Sep-2017 David Chen <dwchen@google.com> Adds new utility functions for evaluating log entry matching.
Includes matching for both simple and compound matchers.

Change-Id: Id913d2625d205ad9a529019e98ae805f730f4d48
Test: Added new unit-tests and checked on marlin device.
/frameworks/base/cmds/statsd/Android.mk
b487b5533eba8635232009c7f32a54a0380a532d 18-Sep-2017 Bookatz <bookatz@google.com> statsd & statscompanion communication more robust

If statsd or statsdcompanion crashes, or if one loads
before the other, the other will be able to accomodate.

When statsd loads, it will attempt to tell statscompanion that it's
alive, and then get on to its business, while assuming that
statscompanion is not alive. Only when statscompanion tells statsd
that it is alive, statsd will then start to use it.

When statscompanion loads, it will attempt to tell statsd that it's
alive and then do nothing (since it has nothing to do). When statsd
tells statscompanion that statsd is alive, statscompanion will respond,
telling statsd that it is alive and, if that binder call returns, will
get to work.

This way, if statsd loads first, it can work unobstructed until
statscompanion informs statsd that it is alive, at which point they
shake hands and work. Conversely, if statscompanion loads first, it will
do nothing until statsd contacts it, at which point they will shake
hands and work.

Test: manual
Change-Id: I969ad47fb8060e27814d05ad37433a02711cfa6a
/frameworks/base/cmds/statsd/Android.mk
43b272e20c031f48b0f0d9775723ae962f7d7472 20-Sep-2017 Adam Bookatz <bookatz@google.com> Merge changes I82fe220e,Iff324c73,I3bd73acd

* changes:
indexed priority queue for AnomalyMonitor
StatsCompanionService connection to Statsd
Set up StatsCompanionService.java
00698daf3c6c184bb20eb8e9d2c17eb92b567062 15-Sep-2017 yro <yro@google.com> Translate Android log entries to stats_log proto
Test: tested on local device

Change-Id: If9a779a96d31cefaffb1e4424629c14b08e1fc57
/frameworks/base/cmds/statsd/Android.mk
0656b7a158f6f71989e76ba55423217e3e75d8b4 14-Sep-2017 David Chen <dwchen@google.com> Adding ability to add configs via adb command-line. The input
must be in serialized binary format. Also fixes small issue
in build rule related to proto lib.

Test: Manually tested that ADB command works without crashing.

Change-Id: Iba2e677561ff500adb601a598f73e8a7b32540e5
/frameworks/base/cmds/statsd/Android.mk
0e95909ef0b2aac44f305551ea2aa03209b1eec0 08-Sep-2017 Bookatz <bookatz@google.com> indexed priority queue for AnomalyMonitor

Created a new class, indexed_priority_queue, which is a priority queue
that allows the removal of elements (other than just the top element).
This is required for AnomalyMonitor, which will henceforth use it.

Some tests for this new class are included.

I have only implemented the methods in indexed_priority_queue that are
currently needed. For example, pop_top() has not been written as
AnomalyMonitor does not need it.

Test: adb shell data/nativetest64/statsd_test/statsd_test
Change-Id: I82fe220ee5a879189b0cfa03b551c829cfdd05f0
/frameworks/base/cmds/statsd/Android.mk
486d1cf358a5589263ac509ca513a30251bf20d6 01-Sep-2017 Bookatz <bookatz@google.com> StatsCompanionService connection to Statsd

Created AnomalyMonitor in statsd, which can monitor statsd's internal
anomaly alarms and register the soonest one with the
StatsCompanionService.

This cl introduces the AnomalyMonitor and allows it to register alarms.
It does not yet allow for the removal of internal alarms.

Test: manually added alarms and ensured they were correctly registered
in StatsCompanionService
Change-Id: Iff324c73751c4f43b1507dc64649b50ac388adef
/frameworks/base/cmds/statsd/Android.mk
482d272d7c94201206518d6b37d32647838b15d7 12-Sep-2017 Yao Chen <yaochen@google.com> Add a cmd line tool to StatsService to parse log files from Dropbox

Test: adb shell cmd stats all-logs
Change-Id: I7803c9c021a971619f60fbf6bdfabd33d2f476ef
/frameworks/base/cmds/statsd/Android.mk
ab273e2e13f87d57f66395c52df424710b8066e1 06-Sep-2017 Yao Chen <yaochen@google.com> Add a DropboxWriter in statsd.

+ The DropboxWriter keeps data in cache, and flush to files once the
size exceeds the maximum value.

+ Different components should create their owner DropboxWriter with
different tags, e.g., anomly detection, experiment metrics, etc.

+ Copied stats_log related protos from g3

Test: run statsd, and adb shell dumpsys dropbox
Will add unit tests.

Change-Id: If06e9a9953be32082252b340a97124d732656b40
/frameworks/base/cmds/statsd/Android.mk
5dcbc6c015fd56db9381cb7aff58506e8ebcc150 30-Aug-2017 Joe Onorato <joeo@google.com> Add statsd.

It doesn't start yet by default. When you start it manually, it sets
itself up as a binder system service and starts a thread to read the
event log.

Test: Run statsd, observe output. also run stats_test
Change-Id: If435d6a80fef3c1d957aedb61699bf5e9aae7e56
/frameworks/base/cmds/statsd/Android.mk