History log of /frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
892f3d32293b8358c72da4a94633827abbda640b 02-May-2018 Yangster-mac <yanglu@google.com> Reset statsd and correctly record the dump reason when system
server restarts/crashes.

Test: statsd test
BUG: b/79161505
Change-Id: I0646c764964f6eafde91f9ae0179a1c837af320d
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
163d2602dbc79133096b3dec7920ee157ff1a88b 10-Apr-2018 Yao Chen <yaochen@google.com> Handle logd reconnect.

When statsd reconnects to logd, statsd will read all logs from buffer again. To prevent us from
reprocessing old events, we do the following:

1. At any given moment, record the largest timestamp(T_max) and last timestamp (check point) that
we've seen before.
2. When reconnection happens, we look for the check point until we see a new log with a timestamp
larger than T_max.
-> If we found the CP, resume after the CP. Success
-> If we can't find CP, there is definitely log loss. We reset all configs.

Note:
1. Logd has an API to read logs after a certain timestamp. But this api is vulnerable to
time changes from Settings. So we cannot rely on it.

2. If logd inserts a new log (with older timestamp) before CP, we cannot detect it. It's not
possible to detect it without record all timestamps we have seen.

Test: statsd_test
Bug: 77813113

Change-Id: Ic3fdb47230807606ab11dc994cb162194adb8448
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
15f6bbc24f4d7a3d4481d90a18df33d402ddacba 08-Apr-2018 Yangster-mac <yanglu@google.com> Flush the bucket when creating the metric producer.
Use int64 for value field.
E2e test for gauge/value metric.

BUG: b/74445671

Test: statsd test.
Change-Id: I823a0bade8f89834bdfb9cf48864852a47d7b63b
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
e68f3a5811209eeab71976bc583c6075d9a5979a 04-Apr-2018 Yangster-mac <yanglu@google.com> Flush the partial bucket when startd shuts down or config updated.

Test: statsd test

BUG: b/77556036
Change-Id: Ie4a04ace55e07c4529cdff5906ba874f8815f620
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
203bbbf942e8675338b88ab67d49481d2bfcf904 06-Apr-2018 David Chen <dwchen@google.com> Merge "Fix uid map to be simpler and fix partial bucket." into pi-dev
bd12527c90d55eefa657e6a71cfdd287ecdb4ab3 05-Apr-2018 David Chen <dwchen@google.com> Fix uid map to be simpler and fix partial bucket.

The previous scheme captured periodic snapshots for each config with
complex logic that's unnecessary and wasted memory. We actually don't
need to store any snapshots since we just convert the current state
into a snapshot and also include the deltas (change events) since the
previous report until now.

To make the system more robust, we also include up to 100 of the
deleted apps in the uid map.

Also, fix the wiring of the partial buckets so the metric producers
form partial buckets on both app upgrade and removal, but not on
installation of a new app.

Also, we update StatsCompanionService to also include disabled apps.

Bug: 77607583
Test: Verified unit-tests pass and added new e2e tests.
Change-Id: I98e1f544d6e6571545ae1581c4cebab807596f51
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
164a73e0cfa880f26acdc04e0c35255cf8cd671d 05-Apr-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Retry logs write when it fails. Report skipped event in statsd." into pi-dev
b8382a10a0dca8ff50b0afc35071e47440ecf86c 04-Apr-2018 Yangster-mac <yanglu@google.com> Retry logs write when it fails.
Report skipped event in statsd.

Test: manual test
BUG: b/77222120
Change-Id: I257f5e76d557893c4eb4a8e8a13396d8b5d1afc0
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
5260eea5e48f7c54fa28c2254547c588c344d26d 05-Apr-2018 Tej Singh <singhtejinder@google.com> BugFix: Config TTL

In statsStats, we were writing the config TTL reset time to the wrong
proto field number. This fixes that.

Bug: b/77604625
Test: unit tests, pending CTS test
Change-Id: I9f6e502bf13785bfa60d68805d46b9a5c2286f44
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
b142cc8add29c8c97f6134d35873d23db666027c 31-Mar-2018 Yangster-mac <yanglu@google.com> Statsd config TTL

Roughly check the config every hour to see whether the ttl expired.
If so, read the config from disk and recreate the metric manager.

Test: statsd test

BUG: b/77274363

Change-Id: I16838afe5bbe966c3a0f638869751f9b59a5a259
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
faa1af535bf6ede5fcab2e0bad5f6c16908916a9 31-Mar-2018 David Chen <dwchen@google.com> Includes annotations with statsd reports.

It's tricky to determine the source of the metrics on a device
currently since we can take the union of multiple configs and send
only one giant statsd_config into statsd. We will use the int64 field
to track the sub config id's and the int32 field to track the version
for each sub config, but the fields are named more generically as
annotations.

The annotations are available in both the reports and metadata.

Test: Check that all unit-tests pass on marlin-eng
Bug: 77327261
Change-Id: Ic37c549c8b2991676f69948c515156765c9f5108
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
306ccc2d6bbb396e53f1a250da32003989a24b42 24-Mar-2018 Yangster-mac <yanglu@google.com> Guardrail for dimension in condition in duration tracker.

Test: statsd test

BUG: b/74437017
Change-Id: I349528e419ede817904f1e3884260c06651c6d0b
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
c58750578fdf6e54500e9deabe7d5062afbd6e09 09-Mar-2018 Chenjie Yu <cjyu@google.com> Up the dimension key limit for CpuTimePerUidFreq

This atom logs time spent on cpu per frequency per uid.
On marlin, there are 27 frequency steps per uid (should be 50+ if
flatten to both cores)
On walleye, there are 52 frequency steps per uid.
So it easily goes to 6k data.

Soft limit set to 6k now.
Hard limit set to 10k to accommodate future processors

Memory impact: on marlin, start memory no config is 2,346K
after using this atom in one guage metric, it is 3,067K

Bug: 72505991
Test: cts test
Change-Id: I067a32e54e4a457fdf9a25911aa16030e893ef4d
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
665208d74345d779bed8724862e8d0d1c46a7a76 14-Mar-2018 yro <yro@google.com> Add stats from storage manager to dumpsys result

Bug: 74601313
Test: manual test, statsd_test
Change-Id: Ifdde6bb10d29f3983b0c27a66a6e30f1f40308ee
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
e05c8e777646c51a8fdbda8e2d5ccd35a991d03d 08-Mar-2018 Chenjie Yu <cjyu@google.com> Remove kPullerCooldownMap from code

remove dead code

Bug: 74032852
Test: manual test
Change-Id: I0da74be21bab5842a89ede2a272094ba5ee33d80
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
5ee0787024cc446a21008ff5710dec19c6afc834 06-Mar-2018 Yi Jin <jinyithu@google.com> Use uint64_t instead of long long as API type for consistent reason.

Bug: 74118023
Test: manual
Change-Id: Icd5f506c76d3a008a79cb6c9d2061962ca7fdd40
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
20e9e6231a1aba79b4e5ae47f3ccfb066920e60f 28-Feb-2018 Yao Chen <yaochen@google.com> Reduce statsd binary size from 730KB-> 664KB

1. StatsdStats does not use the proto object in memory anymore.
2. lite_static -> lite
3. don't use sstream

Bug: 72129300
Test: statsd_test

Change-Id: I8a5adaf222d4d5034e8bf115215fb6dd5f042cac
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
03b91d77c4eccd2d088a12e6fe747240a09b573d 28-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Alarm: wakes up statsd and notifies the subscribers."
932ececa1674c59a8da9f3e32d2651e781b86fc4 01-Feb-2018 Yangster-mac <yanglu@google.com> Alarm: wakes up statsd and notifies the subscribers.

Test: manually tested it.
Change-Id: Id796a68976aeb1611183023ba4e9c6a8b8c44bb8
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
255f72e73e23f79157faaf28fcea482c0fa2f5bd 27-Feb-2018 yro <yro@google.com> Return when invalid config ID was provided through adb command and
change the separator of config_uid and config_id to underscore from dash
to disambiguate negative config ids

Bug: 73896814
Test: statsd_test
Change-Id: Ib0604e9f4c104560d570a64208a9e94d7526f8d6
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
6189807c12e9cf32a36d32557725561c44b8aa5a 14-Feb-2018 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Remove unused variables in statsd, and make more warnings show."
330af58f2b8582b855085655fae553cdfaf44e6c 09-Feb-2018 Yangster-mac <yanglu@google.com> Use elapsed realtime instead of times based on wall clock, which can jump around and go backwards.

Test: statsd unit test passed

Change-Id: Ib541df99231e171b3be2a24f75632693e36da90e
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
4c959cb99eb7e71e5417a61f5429c5fa0073e826 13-Feb-2018 Yao Chen <yaochen@google.com> Remove unused variables in statsd, and make more warnings show.

Test: statsd_test

Change-Id: I2c7b674cb615f22c5de90c2de5f2d58108ab2e7f
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
484524a246ffe453f8cd89b698a279c23b0bde1f 02-Feb-2018 Tej Singh <singhtejinder@google.com> Turn off debug logging in statsd

Sets DEBUG to false everywhere and replaces all ALOGD with VLOG so they
do not print with DEBUG false. Leaves all ALOGI, ALOGW and ALOGE as is.

Test: ran all CTS tests and checked "adb logcat -s statsd" to make sure
it wasn't spammy

Change-Id: Iaa8eb3a0a63723ffe40f94f2815f94df877fd432
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
884c8c130fde0d02ada1316f7c27f0f55e7e48b9 26-Jan-2018 Yao Chen <yaochen@google.com> Add more statsd's debugging info to dumpsys.

+ Bugreport will use the non-verbose mode
+ Reuse the log_msg object in LogReader
+ Add logd errors to StatsdStats

Bug: 72383073

Test: manual + statsd_test

Change-Id: Id5a8b103074d034f5ece3c9831c740d44a5df9cd
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
67462296b1a18fc7aa65e0b7ae0e973e49e0a3ae 18-Jan-2018 Yao Chen <yaochen@google.com> Merge "Add StatsdStats in statsd's dumpsys"
f5acabe9466af3571a4318852cdb26856e5ba100 17-Jan-2018 Yao Chen <yaochen@google.com> Add StatsdStats in statsd's dumpsys

Bug: 72123656
Test: manual + statsd_test

Change-Id: I8ad9bb3abfa94ee099aa5fc3f28ad9b6fb7340f7
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
c8b7f2277c7d05dc0a1225dbed13ee85f18a6031 12-Jan-2018 Chenjie Yu <cjyu@google.com> consolidate subsystem sleep state atoms

Test: cts test
Change-Id: Ib9ff746eaa17b6c2c8f9fe5860dd0b480ec85186
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
f6723dfcf08e388adaa9b12a2e6ea8ee54b74d14 09-Jan-2018 Yao Chen <yaochen@google.com> Cap the size of ice box in StatsdStats.

+ only keep the last 20 old config stats.

Test: statsd_test
Change-Id: I5bff68299bb9e4def7f7b54aa1d347133bc7a768
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
b038b709566c191164bd23539175df8aadeb2f94 19-Dec-2017 Chenjie Yu <cjyu@google.com> pullers now cache data to throttle frequent pull requests.
all pullers have a default 1s cool down before next pull.
We can adjust these later.
Also add puller stats in StatsdStats

Test: unit test
Change-Id: I71894a24c41e059d841591312dbb852f54387b7d
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
94e197cceb2ba7df13ff8de04f60bfeec64015d9 03-Jan-2018 Yangster-mac <yanglu@google.com> 1/ Change all "name" to id in statsD.
2/ Handle Subscription for alert.
3/ Support no_report_metric

Bug: 69522276
Test: all statsd unit tests passed.
Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
e1d143a0589f22876f17fcb2f892fd9a39a3ebd9 14-Dec-2017 Bookatz <bookatz@google.com> StatsdStats alerts -rename alerted_times

Renaming declared_times to alerted_times.

Test: still compiles and passes tests
Change-Id: Ib7c890a21bb91a7185c6d3a098b1bec8bd868ad5
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
8f2f3d82053693b0dac828e848e2bb238e1db2d2 07-Dec-2017 Bookatz <bookatz@google.com> Anomaly Alert declarations in StatsdStats

StatsdStats now tracks the number of times an anomaly is detected (per
config, per alert name).

Also adds a configKey to AnomalyTracker, which is needed, not only for
statsdstats, but also (in the future) for reporting the header
information to incidentd.

Bug: 67978682
Test: adb shell data/nativetest64/statsd_test/statsd_test
Change-Id: Ib254db7e1edb4f0f193f4772d17f14934cdf7e30
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
1d0136d322899cd9db4416c22873ee7ed01d843a 01-Dec-2017 Bookatz <bookatz@google.com> Anomaly detection statsdstats

Adds statsdstats for tracking the number of anomaly alarms registered.

Test: adb shell data/nativetest64/statsd_test/statsd_test
Bug: 67978682
Change-Id: I4aa0539a13c60ccb3b7fd2a4f41c4ad867f3a4ea
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
c136f45aee2c193e954c107f35109ccc8b1146b3 27-Nov-2017 David Chen <dwchen@google.com> Adds guardrail for memory usage for statsd uid map.

Checks if current memory usage of uid map is above a configured limit
and if so, we start deleting snapshots. If there are no more
snapshots, we begin deleting two of the deltas. Also records stats
in the guardrail StatsdStats. Also fixes an edge case where a config
is added after the snapshots are added. We request a snapshot of all
installed uid's at that moment. Finally, adds the uid map memory size
when determining if we should send a broadcast to trigger collection.

Test: Added unit-tests and check they pass on marlin.
Change-Id: Id5d86378bd1efe12a06b409164c777c0c6f4e3ab
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
0fac5b17e6ec11461a35859740212daa13d2c5cd 29-Nov-2017 Yao Chen <yaochen@google.com> Add no more than 20 timestamps for a config in StatsdStats.

To avoid the array to grow unboundedly.

Test: added unit tests.
Change-Id: I3c4823b2a89bb69428d81b9fe348e7ea9aba24c7
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
69f1baf7dd3d6e595abdae000adf0f197558d0ea 28-Nov-2017 Yao Chen <yaochen@google.com> Some fixes in StatsdStats, and add some unit tests

+ Add timestamp for when metric data is reported.

Test: statsd_test

Change-Id: Ief5ec5172feed4ec74b7422b77cf69ec8361ef2f
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp
b356151e63140085cb96fa16804ee18b3862a4fc 22-Nov-2017 Yao Chen <yaochen@google.com> Add StatsdStats and guardrail.

+ StatsdStats is the global class that tracks the stats about statsd.

+ Added guardrail for classes that have a map which could potentially grow
unboundedly with the number of logs.

TODO: add unit tests & CTS for StatsdStats, and guardrail
add stats for pulled atoms.

Test: statsd_test

Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
/frameworks/base/cmds/statsd/src/guardrail/StatsdStats.cpp