| b85e5616dfa153eea1e61e99427164c5102dbbd3 |
|
02-May-2018 |
android-build-team Robot <android-build-team-robot@google.com> |
Merge "Reduce statsd log data size." into pi-dev
|
| 9def8e3995b1034d943a0fd22b6e512bfacdab77 |
|
17-Apr-2018 |
Yangster-mac <yanglu@google.com> |
Reduce statsd log data size.
1. Hash the strings in metric dimensions.
2. Optimize the timestamp encoding in bucket.
Use bucket num for full bucket and millis for
partial bucket.
3. Encode the dimension path per metric and avoid
deduping it across dimensons.
Test: statsd test
Change-Id: I18f69654de85edb21a9c835c73edead756295e05
BUG: b/77813755
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| a802d73da625ac72f6a0211aab017f365139112d |
|
24-Apr-2018 |
Yangster-mac <yanglu@google.com> |
Clear the past buckets for no-report metrics
Test: statsd tests.
BUG: b/78503732
Change-Id: I3de855929ad05677769cffaae4fea2db0d2f32da
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 15f6bbc24f4d7a3d4481d90a18df33d402ddacba |
|
08-Apr-2018 |
Yangster-mac <yanglu@google.com> |
Flush the bucket when creating the metric producer.
Use int64 for value field.
E2e test for gauge/value metric.
BUG: b/74445671
Test: statsd test.
Change-Id: I823a0bade8f89834bdfb9cf48864852a47d7b63b
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| e68f3a5811209eeab71976bc583c6075d9a5979a |
|
04-Apr-2018 |
Yangster-mac <yanglu@google.com> |
Flush the partial bucket when startd shuts down or config updated.
Test: statsd test
BUG: b/77556036
Change-Id: Ie4a04ace55e07c4529cdff5906ba874f8815f620
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| b142cc8add29c8c97f6134d35873d23db666027c |
|
31-Mar-2018 |
Yangster-mac <yanglu@google.com> |
Statsd config TTL
Roughly check the config every hour to see whether the ttl expired.
If so, read the config from disk and recreate the metric manager.
Test: statsd test
BUG: b/77274363
Change-Id: I16838afe5bbe966c3a0f638869751f9b59a5a259
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 427d372552490a2c5ac4041fe345b15f69451f57 |
|
22-Mar-2018 |
Yao Chen <yaochen@google.com> |
Support sliced condition change in GaugeMetric
TODO: We need CTS to verify the behavior.
Bug: 73958484
Test: statsd_test
Change-Id: I56406983ddede12bc6a2e12188693a0c51ccae5c
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| c40a19d2e43d5de3e036e926bf070220c2c865e6 |
|
16-Mar-2018 |
Yao Chen <yaochen@google.com> |
Add uid field annotation in atoms.proto and statd memory usage optimization.
[memory]
statsd binary size from 664k -> 600k
memory usage 1978k -> 1813k (with no configs)
+ Avoid initialize any static map in statslog.h to avoid many copies of the map in each include.
- Do it in cpp so that it is initialized only in places that use them
[Uid annotation]
+ Uid annotation is needed for extracting uid from dimension for UidCpuPuller.
+ After the change, stand-alone uids don't need to be in field 1 anymore.
+ Also added exclusive bit annotation in AppDied
+ Currently only allow one uid field in an Atom. This is to keep things simple until
we find an exception.
Test: statsd_test
Bug: 73958484
Bug: 72129300
Change-Id: I8a916d5c00d5930e24ae7e0825a57dea19c0e744
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 5ee0787024cc446a21008ff5710dec19c6afc834 |
|
06-Mar-2018 |
Yi Jin <jinyithu@google.com> |
Use uint64_t instead of long long as API type for consistent reason.
Bug: 74118023
Test: manual
Change-Id: Icd5f506c76d3a008a79cb6c9d2061962ca7fdd40
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 06dba5d79c096b02d3ba6fb73e64451ff12e388d |
|
26-Jan-2018 |
Yao Chen <yaochen@google.com> |
Add API to let metrics directly drop data without writing to an output.
+ Metrics will do flushIfNeeded() to correctly move the clock and informing
AnomalyTracker the past bucket info, and then clear past buckets.
+ We will still keep the current bucket data for the validity of the future metrics.
Bug: 70571383
Test: statsd_test
Change-Id: Ib13c45574974e7b4e82bd8f305091dc93bda76f5
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| e1faa03b48ce9132371acc52dee2d990df0d6418 |
|
21-Feb-2018 |
Yangster <yanglu@google.com> |
Fix the bug of timestamp encoding in event metric report.
Test: statsd cts tests
Change-Id: Id1d330f5d48d15ba026ae7c7e8cf4e1f6b460fba
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| d5c356214fabec1ad9d5d16d673f000deb49ea30 |
|
02-Feb-2018 |
Yangster-mac <yanglu@google.com> |
Timestamp Truncating whitelist.
Test: manually tested.
Change-Id: I46da375b6c0773ffc611bc06fff12cb4f9a7fc18
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 330af58f2b8582b855085655fae553cdfaf44e6c |
|
09-Feb-2018 |
Yangster-mac <yanglu@google.com> |
Use elapsed realtime instead of times based on wall clock, which can jump around and go backwards.
Test: statsd unit test passed
Change-Id: Ib541df99231e171b3be2a24f75632693e36da90e
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| e8cf6110aed233019a41fddd4fdca0cc93bb8433 |
|
12-Feb-2018 |
TreeHugger Robot <treehugger-gerrit@google.com> |
Merge "Statsd CPU optimization."
|
| 8a8d16ceea1e5b7a2f8c41e17b5d993035f50f5d |
|
08-Feb-2018 |
Yao Chen <yaochen@google.com> |
Statsd CPU optimization.
The key change is to revamp how we parse/store/match a log event, especially how we match repeated
field and attribution nodes, and how we construct dimensions and compare them.
+ We use a integer to encode the field of a log element. And also encode the FieldMatcher into an
integer and a bit mask. The log matching becomes 2 integer operations.
+ Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then
becoming checking if the underlying integers are equal. The integers are stored contiguously
in memory, so it's much faster than previous tree structure.
Start review from FieldValue.h
Test: statsd_test + new unit tests
Bug: 72659059
Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 7f9e63b63269baa41a36fc31acd6fc071309c26c |
|
03-Feb-2018 |
Yi Jin <jinyithu@google.com> |
Add clear method to ProtoOutputStream so it is reusable by just
rewinding its internal pointer.
Bug: 72319159
Test: atest incidentd_test and atest statsd_test
Change-Id: Id046b987a7460d68300e5d4d46ae56aec1eba2d0
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 16049578001364e1fcb0f10d6f94868081e57316 |
|
02-Feb-2018 |
David Chen <dwchen@google.com> |
Modifies statsd output for start and end times.
We include the start of when the last dump occurred and the current
timestamp. These timestamps are shared across all metrics, so
there's no advantage in duplicating these numbers across all metrics.
Also, we should use elapsed realtime instead of times based on wall
clock, which can jump around and go backwards.
Test: Test that statsd can still build and
adb shell cmd stats dump-report doesn't crash.
Change-Id: I819e5643cee75dfa3e78a58f94c9d61ededa78d7
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 27785a8a4a684c831c18f7189a6fa1b98c3573e6 |
|
20-Jan-2018 |
David Chen <dwchen@google.com> |
Partial buckets on app upgrade and fix duration.
Statsd will create partial buckets in all metrics producers when an
app is upgraded so that we can separate metrics between different
versions of an app. By looking at the uid map changes, we can tell
which app versions belong to a bucket; for metrics that are not
affected by an app version, we can instead join the buckets together.
To simplify the logic, the ends of the full buckets are always
aligned to when the metric producers were created. These boundaries
are computed on the fly by using the bucket number and the metric
producers' start times.
We keep the anomaly trackers to only be given full buckets; we buffer
the partial buckets within each metric producer.
Duration metric's MAX_SPARSE is fixed to be implemented as such. In
addition, after further discussion, we find anomaly detection on
MAX_SPARSE to be unnecessary, so this functionality is removed.
Test: Unit-tests added and modified, passed on marlin-eng.
Change-Id: I5ff7a9c7f05c406e9faf400c6a39162970ded102
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 9369446f0b04945d6674550728ae81196d6fb5c2 |
|
23-Jan-2018 |
Yangster-mac <yanglu@google.com> |
Support dimension in condition in metric producers.
Test: added e2e tests for count/duration metrics sliced by fields in condition and with/without links.
Change-Id: Ie34deba68e6780abdde458be3f0ce5284e76a1a2
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 635b4b37210bd61b73aebf19298e38b8fb382e85 |
|
24-Jan-2018 |
Yangster-mac <yanglu@google.com> |
Do not output empty metric reports.
Test: statsd unit test passed.
Change-Id: I296edbe621eaea72ad2e1dfe656698302614c0f3
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 94e197cceb2ba7df13ff8de04f60bfeec64015d9 |
|
03-Jan-2018 |
Yangster-mac <yanglu@google.com> |
1/ Change all "name" to id in statsD.
2/ Handle Subscription for alert.
3/ Support no_report_metric
Bug: 69522276
Test: all statsd unit tests passed.
Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 2087716f2bdca90c7c3034d556ac12911bd8018e |
|
23-Dec-2017 |
Yangster-mac <yanglu@google.com> |
1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.
Test: statsd unit test passed.
Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 3c0b95ceb520c6b23871da90c23f89c55b76b560 |
|
16-Dec-2017 |
Yao Chen <yaochen@google.com> |
Turn off verbose logging, remove the fake config.
Test: statsd_test & manual
Change-Id: I43068c8db006cbbeccacaa9d8426ab4d0394313a
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| f09569f848ca0b81a21a74e9f4dd8bd9a886151a |
|
14-Dec-2017 |
Yao Chen <yaochen@google.com> |
Further reduce statsd memory usage.
+ Remove the protobuf *Metric object from MetricProducers
-- This saves ~150 bytes per metric.
+ Remove the StatsdConfig from ConfigManager
-- This saves us xKB per config.
+ Also remove alerts from fake config to avoid crash (Bug: 70627390)
+ Other misc fixes too.
Test: statsd_test & manual
Change-Id: Ied4eb3fa31c50599817b3a5e1caf5077c487fad2
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| a7259abde4e89fd91404b14b4845113cd313d1ec |
|
10-Dec-2017 |
Chenjie Yu <cjyu@google.com> |
use running sum for ValueMetricProducer bucket
simplify ValueMetricProducer logic for pulled data
Test: unit test
Change-Id: Ic0a21a543166cc5c34c1fa505dba08d1fc2f510a
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 288c60001330a5a924a47c0eebd6097ae3ee5d67 |
|
12-Dec-2017 |
Yao Chen <yaochen@google.com> |
Only create ProtoOutputStream when onGetData() is called.
The exception is EventMetricProducer. Each EventMetricProducer will still have a ProtoOutputStream
Because LogEvent comes as a fixed 4K, it's more memory efficient to have an 8k ProtoOutputStream for
storing the events.
Also removed finish() api in MetricProducer, which was intended to use with Dropbox.
Test: statsd_test & dogfood app
Bug: 70393808
Change-Id: I2efe4ecc76a88060a9aa5eb49d1fa6ea60bc5da8
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| f2bee6fec965fd42ab223f1a3aa705f07ba79aea |
|
29-Nov-2017 |
Yangster <yanglu@google.com> |
1/ Only expose thread-safe interfaces in metric producer.
2/ Simplify lock logic.
3/ Add test for duration metric producer.
Test: all unit test passsed.
Change-Id: If6ee2e69a17f12406f4b3ea3553b14642cd636d6
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 6a8c799d901cbd166aa6463d7dea231bcf594a1e |
|
29-Nov-2017 |
Yao Chen <yaochen@google.com> |
Revert "Thread-safe metric producers."
This reverts commit 8de6939c494da838f6dbbda0631f66425dbbd25b.
Change-Id: Ieae841bfc5339b569f0fca909a6066de72806617
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 8de6939c494da838f6dbbda0631f66425dbbd25b |
|
27-Nov-2017 |
Yangster <yanglu@google.com> |
Thread-safe metric producers.
Test: unit test passed
Change-Id: Ie47404e8649b63ee8ac32e40189a47f6cb7a9def
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| b356151e63140085cb96fa16804ee18b3862a4fc |
|
22-Nov-2017 |
Yao Chen <yaochen@google.com> |
Add StatsdStats and guardrail.
+ StatsdStats is the global class that tracks the stats about statsd.
+ Added guardrail for classes that have a map which could potentially grow
unboundedly with the number of logs.
TODO: add unit tests & CTS for StatsdStats, and guardrail
add stats for pulled atoms.
Test: statsd_test
Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 7c334a129e93e405a72e8299a1cd928af079d14f |
|
22-Nov-2017 |
Yangster <yanglu@google.com> |
Make member function as const whenever possible.
Test: unit tests passed.
Change-Id: I751cabf305a4b5aa2095853cc951837da0df4c78
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 75f0a4d3941813c8b3f1a4361531202fa741e0bf |
|
18-Nov-2017 |
yro <yro@google.com> |
Use ProtoOutputStream's bytesWritten to properly estimate the size of
EventMetricsData on memory
Test: statsd, statsd_test
Change-Id: I6fc90c5d9303fe6cab582afb03c480969a6dc5a5
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 1d7b0cd6b54e3b3ec8efdffd47b40cdd54c4e8d7 |
|
15-Nov-2017 |
David Chen <dwchen@google.com> |
Support StatsD sending broadcasts.
StatsD will send a broadcast when we're 90% of the way to our
allocated memory limit for the configuration. If the memory usage
goes over the limit, we just lose all the data for this config.
Also modifies the adb shell commands to facilitate debugging of the
broadcasts.
Test: Manually tested on marlin-eng with custom gmscore code.
Change-Id: I517a15bd4c959aa221802f84a51f13141a725102
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| ae2df01aae2ee7076af875c04e991140f685f733 |
|
14-Nov-2017 |
Stefan Lafon <stlafon@google.com> |
Rename "stats_events" as "atom".
Also make atom WakelockStateChanged use the "Type" enum instead of int32.
Test: Unittests are passing. Statsd is working.
Change-Id: I0909e2d97297f78996a81366d66aae62d5bf5ce1
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| d1815dc7b82f99259f6c41cfacc44972b867ef2d |
|
14-Nov-2017 |
Yangster-mac <yanglu@google.com> |
Sync statsd to the public proto changes:
1/ use string id/name for all metrics/configs/matchers.
2/ move alerts out of metric proto.
Test: unit test passed.
Change-Id: I08269f47be30f83f6dbadf873e7d62213449932b
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| b0378b093d9b937fe9a731a601d670d50f263078 |
|
10-Nov-2017 |
yro <yro@google.com> |
1. Add FIELD_COUNT_REPEATED bit masks to repeated fields that were previously missing
2. Resolve segfault when serializing proto into vector of uint8_t
3. Add a local byte size variable for EventMetric data as we cannot call
size() of ProtoOutputStream before we finish writing.
4. Replace hardcoded conversion of second to nano second with NS_PER_SEC
Test: statsd, statsd_test
Change-Id: I47a2ce2b05e6191c18596489682118edcb41e945
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 17adac9cf3e03ad95372eb8657b26909d0414ac0 |
|
09-Nov-2017 |
yro <yro@google.com> |
Finish migrating to use ProtoOutputStream. This change will take the
outputs of each MetricProducer's and merge the results into
ConfigMetricsReport which holds ConfigKey, repeated field of
StatsLogReport, and UidMap. The data will be represented as
vector<uint8_t> which can be passed down to binder call for clients to
pick up. Also, all unnecessary dependencies to stats_log proto have been
removed.
Test: statsd, statsd_test
Change-Id: Ia69137cbc8613644a892e6be1e87b4858bd39fe3
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 2b0f88678b2877a8e9f83cea60f097322b078367 |
|
06-Nov-2017 |
yro <yro@google.com> |
Migrate all remaining MetricProducers to use ProtoOutputStream
Test: statsd, statsd_test
Change-Id: I1087e1c1ffb372ca288dfc575cb7a372b11ce8c5
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 93fe3a34a02c673eaee4a2d18565ba8df20685cb |
|
02-Nov-2017 |
Yao Chen <yaochen@google.com> |
Add unit tests for CountMetricProducer, EventMetricProducer
And other miscellaneous fixes.
+ clang-format
+ 2 bug fixes, one in dump-report command, one in ResourcePowerManagerPuller
Test: statsd_test
Change-Id: Ibd164d948ad62adcc529d813df1210781e38be47
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 24809bdb45c28244aeaa9c1795581d685780645c |
|
01-Nov-2017 |
yro <yro@google.com> |
1. Enable FIELD_TYPE IDs defined in ProtoOutputStream
2. Migrate CountMetricProducer to use ProtoOutputStream
Test: statsd, statsd_test
Change-Id: I33a1ea77a49b045818a48923b2263cb594ab0013
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| b3dda41a5a48ddfa44dc8fa939ab495926a0195d |
|
24-Oct-2017 |
Chenjie Yu <cjyu@google.com> |
reretry ValueMetric implementation and pulling mechanism
Note:
This is for value metric. The default operations is sum the diffs.
The test uses kernel wake lock, which also needs dimension by kernel
wake lock name.
The test is a bit cumbersome as it needs StatsCompanionService to do
the alarm, which is not exact alarm.
The internal state of a slice of bucket would look something like this:
4:ipc0000005e_727_android.hardwar
0 0
4:SensorService_wakelock
40 64
4:ipc0000005c_727_android.hardwar
...
Test: manual test on device.
Change-Id: I2ed0ac7d3c5fcba8b7611d46f38a38ffd8bdc92a
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 5154a379303ab90a2b2914676a4441917a329b5d |
|
31-Oct-2017 |
Yao Chen <yaochen@google.com> |
Adding multi layer aggregation in DurationMetric
Newly supported metrics examples:
1) Compute [Total|Max] duration of [an app] holding [ANY] wake lock while [this app] is in
[background] and [screen off], bucket size 30seconds, and slice output by uid.
2) Compute [Total|Max] duration of [ANY app] holding [ANY] wake lock while [this app] is in
[background] and [screen off], bucket size 30 seconds.
+ DurationMetric proto has a "what" which is a SimpleCondition. It defines the atom level start
and stop of the duration timer, and it has its atom dimension. e.g., for wake locks, the atom
dimensions wil be uid and wl name.
+ Now dimension is explicitly specified in SimpleCondition proto instead of inferred from the "link"
+ Added support for "Or" and "Max" through 2 layers of aggregation.
TODO: (1) The way we track slicedCondition in duration metric is not efficient. optimize!
(2) The output dimension should all use int32 instead of KeyMatcher. Fix in a future cl.
Test: Added some unit tests using gmock. Will add more unit tests.
Change-Id: I58a827624f01f9a54fcb80709c4de4ff94a8bc67
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 69007c82625dd8b19bd68d1031191e2ff83b5f49 |
|
27-Oct-2017 |
yro <yro@google.com> |
Implement a logic to rate limitted flush statsd with by checking the
size of each MetricsProducer's. The implementation of byteSize() method
is still TBD as it depends on migration to ProtoOutputStream.
Test: statsd, statsd_test
Change-Id: I966606044d7cb814dabe94192bacecad91f28177
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|
| 5110bedd787835d4dbc809b7977285cac364bdef |
|
23-Oct-2017 |
Yao Chen <yaochen@google.com> |
Add EventMetricProducer
+ Started to use ProtoOutputStream in EventMetricProducer.
[TODO]: We need to auto-generate fieldIds for StatsLogReport, XXXMetricData, etc.
[TODO]: We need to add Enum type to liblog, otherwise we cannot reconstruct a proto containing
an enum
+ Some refactor in metric initialization code. There are still boiler plate code, because Metrics
are similar but with subtle differences.
Test: statsd_test
Change-Id: Id7e3212566249a8139b9680f04238c455d50c1b8
/frameworks/base/cmds/statsd/src/metrics/EventMetricProducer.cpp
|