b85e5616dfa153eea1e61e99427164c5102dbbd3 |
|
02-May-2018 |
android-build-team Robot <android-build-team-robot@google.com> |
Merge "Reduce statsd log data size." into pi-dev
|
9def8e3995b1034d943a0fd22b6e512bfacdab77 |
|
17-Apr-2018 |
Yangster-mac <yanglu@google.com> |
Reduce statsd log data size. 1. Hash the strings in metric dimensions. 2. Optimize the timestamp encoding in bucket. Use bucket num for full bucket and millis for partial bucket. 3. Encode the dimension path per metric and avoid deduping it across dimensons. Test: statsd test Change-Id: I18f69654de85edb21a9c835c73edead756295e05 BUG: b/77813755
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
a802d73da625ac72f6a0211aab017f365139112d |
|
24-Apr-2018 |
Yangster-mac <yanglu@google.com> |
Clear the past buckets for no-report metrics Test: statsd tests. BUG: b/78503732 Change-Id: I3de855929ad05677769cffaae4fea2db0d2f32da
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
e68f3a5811209eeab71976bc583c6075d9a5979a |
|
04-Apr-2018 |
Yangster-mac <yanglu@google.com> |
Flush the partial bucket when startd shuts down or config updated. Test: statsd test BUG: b/77556036 Change-Id: Ie4a04ace55e07c4529cdff5906ba874f8815f620
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
b142cc8add29c8c97f6134d35873d23db666027c |
|
31-Mar-2018 |
Yangster-mac <yanglu@google.com> |
Statsd config TTL Roughly check the config every hour to see whether the ttl expired. If so, read the config from disk and recreate the metric manager. Test: statsd test BUG: b/77274363 Change-Id: I16838afe5bbe966c3a0f638869751f9b59a5a259
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
427d372552490a2c5ac4041fe345b15f69451f57 |
|
22-Mar-2018 |
Yao Chen <yaochen@google.com> |
Support sliced condition change in GaugeMetric TODO: We need CTS to verify the behavior. Bug: 73958484 Test: statsd_test Change-Id: I56406983ddede12bc6a2e12188693a0c51ccae5c
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
06dba5d79c096b02d3ba6fb73e64451ff12e388d |
|
26-Jan-2018 |
Yao Chen <yaochen@google.com> |
Add API to let metrics directly drop data without writing to an output. + Metrics will do flushIfNeeded() to correctly move the clock and informing AnomalyTracker the past bucket info, and then clear past buckets. + We will still keep the current bucket data for the validity of the future metrics. Bug: 70571383 Test: statsd_test Change-Id: Ib13c45574974e7b4e82bd8f305091dc93bda76f5
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
e8cf6110aed233019a41fddd4fdca0cc93bb8433 |
|
12-Feb-2018 |
TreeHugger Robot <treehugger-gerrit@google.com> |
Merge "Statsd CPU optimization."
|
8a8d16ceea1e5b7a2f8c41e17b5d993035f50f5d |
|
08-Feb-2018 |
Yao Chen <yaochen@google.com> |
Statsd CPU optimization. The key change is to revamp how we parse/store/match a log event, especially how we match repeated field and attribution nodes, and how we construct dimensions and compare them. + We use a integer to encode the field of a log element. And also encode the FieldMatcher into an integer and a bit mask. The log matching becomes 2 integer operations. + Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then becoming checking if the underlying integers are equal. The integers are stored contiguously in memory, so it's much faster than previous tree structure. Start review from FieldValue.h Test: statsd_test + new unit tests Bug: 72659059 Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
7f9e63b63269baa41a36fc31acd6fc071309c26c |
|
03-Feb-2018 |
Yi Jin <jinyithu@google.com> |
Add clear method to ProtoOutputStream so it is reusable by just rewinding its internal pointer. Bug: 72319159 Test: atest incidentd_test and atest statsd_test Change-Id: Id046b987a7460d68300e5d4d46ae56aec1eba2d0
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
9369446f0b04945d6674550728ae81196d6fb5c2 |
|
23-Jan-2018 |
Yangster-mac <yanglu@google.com> |
Support dimension in condition in metric producers. Test: added e2e tests for count/duration metrics sliced by fields in condition and with/without links. Change-Id: Ie34deba68e6780abdde458be3f0ce5284e76a1a2
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
884c8c130fde0d02ada1316f7c27f0f55e7e48b9 |
|
26-Jan-2018 |
Yao Chen <yaochen@google.com> |
Add more statsd's debugging info to dumpsys. + Bugreport will use the non-verbose mode + Reuse the log_msg object in LogReader + Add logd errors to StatsdStats Bug: 72383073 Test: manual + statsd_test Change-Id: Id5a8b103074d034f5ece3c9831c740d44a5df9cd
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
2087716f2bdca90c7c3034d556ac12911bd8018e |
|
23-Dec-2017 |
Yangster-mac <yanglu@google.com> |
1/ Support nested message and repeated fields in statsd. 2/ Filter gauge fields by FieldMatcher. 3/ Wire up wakelock attribution chain. 4/ e2e test: wakelock duration metric with aggregated predicate dimensions. 5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate. Test: statsd unit test passed. Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
d10f7b1c7bdb1c66aa04148945cae9733ee4cadf |
|
18-Dec-2017 |
Yao Chen <yaochen@google.com> |
Add log source filtering in statsd to filter out spams. + Add log source whitelist in StatsdConfig + Some changes in UidMap API. Listener needs to be wp instead of sp. + Update dogfood app config to have log source + Increase the stats service thread pool size to 10 (9+1). TODO: add unit tests(b/70805664). This unit test takes some time to write. Test: statsd_test & manual Change-Id: I129b1cc13db5114db7417580962bd7cc4438519d
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
f09569f848ca0b81a21a74e9f4dd8bd9a886151a |
|
14-Dec-2017 |
Yao Chen <yaochen@google.com> |
Further reduce statsd memory usage. + Remove the protobuf *Metric object from MetricProducers -- This saves ~150 bytes per metric. + Remove the StatsdConfig from ConfigManager -- This saves us xKB per config. + Also remove alerts from fake config to avoid crash (Bug: 70627390) + Other misc fixes too. Test: statsd_test & manual Change-Id: Ied4eb3fa31c50599817b3a5e1caf5077c487fad2
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
a7259abde4e89fd91404b14b4845113cd313d1ec |
|
10-Dec-2017 |
Chenjie Yu <cjyu@google.com> |
use running sum for ValueMetricProducer bucket simplify ValueMetricProducer logic for pulled data Test: unit test Change-Id: Ic0a21a543166cc5c34c1fa505dba08d1fc2f510a
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
288c60001330a5a924a47c0eebd6097ae3ee5d67 |
|
12-Dec-2017 |
Yao Chen <yaochen@google.com> |
Only create ProtoOutputStream when onGetData() is called. The exception is EventMetricProducer. Each EventMetricProducer will still have a ProtoOutputStream Because LogEvent comes as a fixed 4K, it's more memory efficient to have an 8k ProtoOutputStream for storing the events. Also removed finish() api in MetricProducer, which was intended to use with Dropbox. Test: statsd_test & dogfood app Bug: 70393808 Change-Id: I2efe4ecc76a88060a9aa5eb49d1fa6ea60bc5da8
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
3accca05ddcad9d0b1b313eae49f273e39121d3c |
|
20-Sep-2013 |
Dianne Hackborn <hackbod@google.com> |
Add major version code to platform. It turns the version code into almost a 64-bit integer, with the new major part being the upper 32 bits. The only tricky part about this is the backup manager, since it stored 32-bit version codes in its backup data sets. This is dealt with by, when the major version code is not 0, writing MIN_INT as the version code and following that by the full long version code, which we can detect when reading. Note that this makes backup sets containing apps with major version codes incompatible with older versions of the platform. Bug: 64459786 Test: Added in Change-Id: Iab8a682b62103babd6c16a56b8dc1e97d7078658 Change-Id: Ibfffe235bbfcf358b3741abd3f7197fdb063d3f3
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
f2bee6fec965fd42ab223f1a3aa705f07ba79aea |
|
29-Nov-2017 |
Yangster <yanglu@google.com> |
1/ Only expose thread-safe interfaces in metric producer. 2/ Simplify lock logic. 3/ Add test for duration metric producer. Test: all unit test passsed. Change-Id: If6ee2e69a17f12406f4b3ea3553b14642cd636d6
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
b356151e63140085cb96fa16804ee18b3862a4fc |
|
22-Nov-2017 |
Yao Chen <yaochen@google.com> |
Add StatsdStats and guardrail. + StatsdStats is the global class that tracks the stats about statsd. + Added guardrail for classes that have a map which could potentially grow unboundedly with the number of logs. TODO: add unit tests & CTS for StatsdStats, and guardrail add stats for pulled atoms. Test: statsd_test Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
7c334a129e93e405a72e8299a1cd928af079d14f |
|
22-Nov-2017 |
Yangster <yanglu@google.com> |
Make member function as const whenever possible. Test: unit tests passed. Change-Id: I751cabf305a4b5aa2095853cc951837da0df4c78
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
e2cd6d509b17894b95d14523ae3e7c4c7a9a74e3 |
|
10-Nov-2017 |
Yangster-mac <yanglu@google.com> |
1/ Duration anomaly tracker with alarm. 2/ Init anomaly from config based on the public language. 3/ Unit tests for anomaly detection in count/gauge producer. 4/ Revisit the duration tracker logic. Test: unit test passed. Change-Id: I2423c0e0f05b1e37626954de9e749303423963f2
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
75f0a4d3941813c8b3f1a4361531202fa741e0bf |
|
18-Nov-2017 |
yro <yro@google.com> |
Use ProtoOutputStream's bytesWritten to properly estimate the size of EventMetricsData on memory Test: statsd, statsd_test Change-Id: I6fc90c5d9303fe6cab582afb03c480969a6dc5a5
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
b0378b093d9b937fe9a731a601d670d50f263078 |
|
10-Nov-2017 |
yro <yro@google.com> |
1. Add FIELD_COUNT_REPEATED bit masks to repeated fields that were previously missing 2. Resolve segfault when serializing proto into vector of uint8_t 3. Add a local byte size variable for EventMetric data as we cannot call size() of ProtoOutputStream before we finish writing. 4. Replace hardcoded conversion of second to nano second with NS_PER_SEC Test: statsd, statsd_test Change-Id: I47a2ce2b05e6191c18596489682118edcb41e945
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
17adac9cf3e03ad95372eb8657b26909d0414ac0 |
|
09-Nov-2017 |
yro <yro@google.com> |
Finish migrating to use ProtoOutputStream. This change will take the outputs of each MetricProducer's and merge the results into ConfigMetricsReport which holds ConfigKey, repeated field of StatsLogReport, and UidMap. The data will be represented as vector<uint8_t> which can be passed down to binder call for clients to pick up. Also, all unnecessary dependencies to stats_log proto have been removed. Test: statsd, statsd_test Change-Id: Ia69137cbc8613644a892e6be1e87b4858bd39fe3
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
2b0f88678b2877a8e9f83cea60f097322b078367 |
|
06-Nov-2017 |
yro <yro@google.com> |
Migrate all remaining MetricProducers to use ProtoOutputStream Test: statsd, statsd_test Change-Id: I1087e1c1ffb372ca288dfc575cb7a372b11ce8c5
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
93fe3a34a02c673eaee4a2d18565ba8df20685cb |
|
02-Nov-2017 |
Yao Chen <yaochen@google.com> |
Add unit tests for CountMetricProducer, EventMetricProducer And other miscellaneous fixes. + clang-format + 2 bug fixes, one in dump-report command, one in ResourcePowerManagerPuller Test: statsd_test Change-Id: Ibd164d948ad62adcc529d813df1210781e38be47
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
b3dda41a5a48ddfa44dc8fa939ab495926a0195d |
|
24-Oct-2017 |
Chenjie Yu <cjyu@google.com> |
reretry ValueMetric implementation and pulling mechanism Note: This is for value metric. The default operations is sum the diffs. The test uses kernel wake lock, which also needs dimension by kernel wake lock name. The test is a bit cumbersome as it needs StatsCompanionService to do the alarm, which is not exact alarm. The internal state of a slice of bucket would look something like this: 4:ipc0000005e_727_android.hardwar 0 0 4:SensorService_wakelock 40 64 4:ipc0000005c_727_android.hardwar ... Test: manual test on device. Change-Id: I2ed0ac7d3c5fcba8b7611d46f38a38ffd8bdc92a
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
53f801163c3df707d846de8875cf92974d5400a8 |
|
01-Nov-2017 |
TreeHugger Robot <treehugger-gerrit@google.com> |
Merge "Adding multi layer aggregation in DurationMetric"
|
5154a379303ab90a2b2914676a4441917a329b5d |
|
31-Oct-2017 |
Yao Chen <yaochen@google.com> |
Adding multi layer aggregation in DurationMetric Newly supported metrics examples: 1) Compute [Total|Max] duration of [an app] holding [ANY] wake lock while [this app] is in [background] and [screen off], bucket size 30seconds, and slice output by uid. 2) Compute [Total|Max] duration of [ANY app] holding [ANY] wake lock while [this app] is in [background] and [screen off], bucket size 30 seconds. + DurationMetric proto has a "what" which is a SimpleCondition. It defines the atom level start and stop of the duration timer, and it has its atom dimension. e.g., for wake locks, the atom dimensions wil be uid and wl name. + Now dimension is explicitly specified in SimpleCondition proto instead of inferred from the "link" + Added support for "Or" and "Max" through 2 layers of aggregation. TODO: (1) The way we track slicedCondition in duration metric is not efficient. optimize! (2) The output dimension should all use int32 instead of KeyMatcher. Fix in a future cl. Test: Added some unit tests using gmock. Will add more unit tests. Change-Id: I58a827624f01f9a54fcb80709c4de4ff94a8bc67
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
d6896898912eb1fc2c8976dce4340ab8d4e8d22f |
|
25-Oct-2017 |
David Chen <dwchen@google.com> |
Updates uidmap to update snapshots and upload. We send a snapshot of all installed apps with their uids every time a user is added or removed and when statsd is started. We keep track of the latest timestamp when a config key has retrieved the UID map data. This allows us to remove older data when we're guaranteed that all config sources have retrieved the old data. Test: Added more unit tests to UidMap_test and passed on marlin-eng. Change-Id: I34a3d61e75eedec44b98d896d7f6db0bc383f46a
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
69007c82625dd8b19bd68d1031191e2ff83b5f49 |
|
27-Oct-2017 |
yro <yro@google.com> |
Implement a logic to rate limitted flush statsd with by checking the size of each MetricsProducer's. The implementation of byteSize() method is still TBD as it depends on migration to ProtoOutputStream. Test: statsd, statsd_test Change-Id: I966606044d7cb814dabe94192bacecad91f28177
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|
5110bedd787835d4dbc809b7977285cac364bdef |
|
23-Oct-2017 |
Yao Chen <yaochen@google.com> |
Add EventMetricProducer + Started to use ProtoOutputStream in EventMetricProducer. [TODO]: We need to auto-generate fieldIds for StatsLogReport, XXXMetricData, etc. [TODO]: We need to add Enum type to liblog, otherwise we cannot reconstruct a proto containing an enum + Some refactor in metric initialization code. There are still boiler plate code, because Metrics are similar but with subtle differences. Test: statsd_test Change-Id: Id7e3212566249a8139b9680f04238c455d50c1b8
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.h
|