| bd12527c90d55eefa657e6a71cfdd287ecdb4ab3 |
|
05-Apr-2018 |
David Chen <dwchen@google.com> |
Fix uid map to be simpler and fix partial bucket.
The previous scheme captured periodic snapshots for each config with
complex logic that's unnecessary and wasted memory. We actually don't
need to store any snapshots since we just convert the current state
into a snapshot and also include the deltas (change events) since the
previous report until now.
To make the system more robust, we also include up to 100 of the
deleted apps in the uid map.
Also, fix the wiring of the partial buckets so the metric producers
form partial buckets on both app upgrade and removal, but not on
installation of a new app.
Also, we update StatsCompanionService to also include disabled apps.
Bug: 77607583
Test: Verified unit-tests pass and added new e2e tests.
Change-Id: I98e1f544d6e6571545ae1581c4cebab807596f51
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| d50f2ae4875cea32662454090c308bb8c7b622eb |
|
23-Mar-2018 |
Yao Chen <yaochen@google.com> |
rename neq_all_string to neq_any_string in statsd_config
Bug: 73897465
Test: statsd_test
Change-Id: I1d020de873fc26fbb502f0b3487b85fdb3896753
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 9b1140eecdf1b7c0ce56289e91d945312eceebea |
|
27-Feb-2018 |
Yao Chen <yaochen@google.com> |
Add the option to match a whitelist of strings in FieldValueMatcher.
+ This is useful when we want to build Anomaly detection on wakelocks, but want to whitelist
wakelocks held by some apps that are whitelisted. It reduces the number of matchers needed
in such a config.
+ Also added the ability to match an AID by string name.
Bug: 73897465
Test: unit tests added.
Change-Id: I19315ae4d7d27fc467655d3a29866049cd8c9a2b
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 9c1debe330006b2b4d5c08a55905789c57369609 |
|
19-Feb-2018 |
Yao Chen <yaochen@google.com> |
Add annotation to atoms that represent a state change in atoms.proto
+ A state change atom can have one exclusive state field, and any
number of primary key fields.
When there is primary key in the atom, it means the state belongs to the primary key.
For example,
message UidProcessStateChanged {
optional int32 uid = 1 [(stateFieldOption).option = PRIMARY];
optional android.app.ProcessStateEnum state = 2 [(stateFieldOption).option = EXCLUSIVE];
}
When there is no primary key fields in the atom, the state is global.
For example,
message ScreenStateChanged {
optional android.view.DisplayStateEnum state = 1 [(stateFieldOption).option = EXCLUSIVE];
}
+ The annotation is consumed by stats_log_api_gen to generate a static map from the state
atoms to its primary fields, and exclusive fields
+ stats_log.proto is splitted into 2 proto files, because statsd needs proto lite, and c++
lite proto library cannot properly ignore the field options which requires full proto.
This CL doesn't change any logic in the statsd yet. A separate CL will use the field option
information to correctly track the state.
Test: added unit tests in stats_log_api_gen_test. and statsd_test pases.
Change-Id: I9e8a979fe81ba60efd4d854bb7087ce4b2b147ec
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 8a8d16ceea1e5b7a2f8c41e17b5d993035f50f5d |
|
08-Feb-2018 |
Yao Chen <yaochen@google.com> |
Statsd CPU optimization.
The key change is to revamp how we parse/store/match a log event, especially how we match repeated
field and attribution nodes, and how we construct dimensions and compare them.
+ We use a integer to encode the field of a log element. And also encode the FieldMatcher into an
integer and a bit mask. The log matching becomes 2 integer operations.
+ Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then
becoming checking if the underlying integers are equal. The integers are stored contiguously
in memory, so it's much faster than previous tree structure.
Start review from FieldValue.h
Test: statsd_test + new unit tests
Bug: 72659059
Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 2087716f2bdca90c7c3034d556ac12911bd8018e |
|
23-Dec-2017 |
Yangster-mac <yanglu@google.com> |
1/ Support nested message and repeated fields in statsd.
2/ Filter gauge fields by FieldMatcher.
3/ Wire up wakelock attribution chain.
4/ e2e test: wakelock duration metric with aggregated predicate dimensions.
5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate.
Test: statsd unit test passed.
Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| b8c9aa8c9ccba4e64759c177381b742ca99a5487 |
|
03-Dec-2017 |
Stefan Lafon <stlafon@google.com> |
Rename a few things in statsd_config.proto.
This for frameworks/base/
SimpleLogEntryMatcher -> SimpleAtomMatcher.
simple_log_entry_matcher -> simple_atom_matcher
LogEntryMatcher -> AtomMatcher
log_entry_matcher -> atom_matcher
Test: Ran statsd_test.
Change-Id: I7398758e3fab86235897c162884f5ceab3002510
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 80235403d2a80eee1c23dbe856d18194b1b93843 |
|
14-Nov-2017 |
Yao Chen <yaochen@google.com> |
statsd: parse the new format of stats log
+ Changed how we construct LogEvent, now it's based on the context from log_msg
without making a copy of the list.
+ All stats logs now have the same event tag, the atom id is the first elem in the log.
Test: statsd_test
Change-Id: I4419380f2ee1c2b2155d427b9f2adb78883b337f
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| d41c4220d06875cea2d19a7991c3227bd56ea611 |
|
16-Nov-2017 |
Yao Chen <yaochen@google.com> |
Fix some bugs found in statsd
+ in log matcher, condition tracker and duration metric
Test: added unit test
Change-Id: Id633e856ba5453842487321d7ddc0c64100e4bb8
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 93fe3a34a02c673eaee4a2d18565ba8df20685cb |
|
02-Nov-2017 |
Yao Chen <yaochen@google.com> |
Add unit tests for CountMetricProducer, EventMetricProducer
And other miscellaneous fixes.
+ clang-format
+ 2 bug fixes, one in dump-report command, one in ResourcePowerManagerPuller
Test: statsd_test
Change-Id: Ibd164d948ad62adcc529d813df1210781e38be47
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 1481fe142d36d5f0b36eeebc358d5a8aef7bf28a |
|
16-Oct-2017 |
David Chen <dwchen@google.com> |
Changes pulled data to use Parcel objects.
Previously, pulled data was returned as a string. We instead
return the data as an array of StatsLogEventWrapper, which encodes
using the binary-encoded format liblog uses. StatsD uses the same
parsing as for pushed events to convert these. This CL also fixes
the parsing of log_msg since the strings were previously emptied
before we had a chance to read the values.
Note that the cpp-aidl can't support List of Parcelable, so we
have to return the results as an array.
Test: Manual using the new command in StatsService to print results.
Also created a new unit-test by creating a dummy pull code of -1,
but this test is deleted since it required creating a fake output in
StatsCompanionService.
Change-Id: I1cfb9ea081a59292a60e934e8527adc40982ed80
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 729093df0dd7d7038ad43b16ecdb59a1404f3b50 |
|
16-Oct-2017 |
Yao Chen <yaochen@google.com> |
Add support for dimension, and link with condition and added DurationMetric
Now we support following metrics:
<Duration> of [app holding a wake lock], while [*this app*] is [in background] [AND] [screen is off]
[Slice] the output by [app name, wake lock name], with bucket size [30sec]
+ Also added onDumpReport() api in MetricsManager, it can be called from client to fetch the data from
statsd
+ Also added command line tool to dump the StatsLogReport from all metrics for debugging.
+ Synced proto from google3. with a pending cl (cr/172359050)
TODO: We need to add tons of tests to test the Metrics. I will work on it after this CL so people
can be unblocked.
I locally test the duration metric with wake lock with an app that generates StatsLog events.
Test: statsd_test
and manual test, and run:
adb shell cmd stats dump-report
We have a default config, which contains a metrics to count PROCESS_START event sliced by
package name.
Change-Id: I4838cc6cf025c143b7e84f43040703a78121fd25
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| c4dfae56c10a1dd571baa78c750f2e68c919d74f |
|
18-Oct-2017 |
Joe Onorato <joeo@google.com> |
Introduce a new wrapper for log_msg -- LogEvent
It stores all of the parsed fields in a single vector, and
provides accessor methods to get at fields by index with
their correct type.
Test: statsd_test
Change-Id: I4fa94e4ce52db3ac87f19b62f9c85398de6e8145
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 9fc9edf95a308f5884bf541cac81ce1f41aba0ba |
|
16-Oct-2017 |
Joe Onorato <joeo@google.com> |
Clean up how we handle configurations, and other assorted cleanup
- Add a ConfigManager class that tracks the configurations
that have been passed to us. Configurations are now
tracked by tuples of (uid,tag), where the tag is an
app-defined string, in case a single uid has multiple
configurations.
- Move all of the initialization into StatsService.
- Get rid of the ability to have multiple LogListeners. Raw
events are now pushed directly into StatsService, which
can distribute them to the interested parties (and will
eventually be able to do the proper locking).
- Add Log.h, which sets our LOG_TAG correctly.
- Move some of the related files that I expect will grow some
into their own subdirectories.
Test: statsd_test
Test: adb shell cmd stats config ...
Test: adb shell dumpsys stats
Change-Id: I79487603003d8a842d5bd319741f1ecbf72063d1
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| caf339d004fad667748b68912c254df4e75cdc5a |
|
07-Oct-2017 |
Yao Chen <yaochen@google.com> |
More complete implementation for condition and log matchers in statsd.
+ also synced proto from google3 to fix the LogEntryMatcher proto
+ MetricsManager represents StatsdConfig, it's responsible for initializing and managing all
LogEntryMatcher, Condition, and Metrics. Start review from here.
+ Added more complete StatsdConfig initialization, including building the map for:
LogEntryMatcher -> Metrics
LogEntryMatcher -> Condition
Condition -> Metrics.
All the maps use index(int). The extra amount of memory for storing mappings help us
quickly process log events.
The StatsdConfig initialization process detects malformed config
- Circle dependency
- Missing definition
etc.
And once we detect ANY error, statsd will reject the config. And the resources related to this
config will be released.
Test: Added unit tests
Change-Id: I2c4aefdbf3e2aa1701eacbb2fb5e653819ec1fbb
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| 44cf27c14880005df093f372491c593e1a9a3c58 |
|
15-Sep-2017 |
Yao Chen <yaochen@google.com> |
Add metric computation skeleton to statsd.
This cl is to let statsd understand statsd_config, and compute metrics
defined in the config.
+ StatsLogProcessor is given a StatsdConfig (hard coded right now).
We construct a MetricProducer for each of the metric, and the metrics
share Condition and LogEntryMatchers
+ Added the CountMetricProducer type for CountMetric.
We can now count times of SCREEN_ON events given a config.
TODO: 1) conditions are not implemented.
2) slicings are not implemented in CountMetric
3) move the interaction to dropbox to a separate thread
4) decide how the in memory metrics would be used by anomaly detection
Test: manual test.
$ adb shell /system/bin/statsd
$ cat config_file.dat | adb shell cmd stats config
Change-Id: I38f4059c0dc5a827c338131d4a6fa7d4cbe865db
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| cdb1a0ed959818b0c3822e114c0456833769bade |
|
28-Sep-2017 |
Stefan Lafon <stlafon@google.com> |
Check in new protos and constants.
Test: Started statsd and verified it outputs data. Also ran statsd tests.
Change-Id: I2a438b2ddfcb1576e21acb6159bea607fed7caaa
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|
| dd89694f5e843963b992a09317aa58fa4a9d7a8d |
|
26-Sep-2017 |
David Chen <dwchen@google.com> |
Adds new utility functions for evaluating log entry matching.
Includes matching for both simple and compound matchers.
Change-Id: Id913d2625d205ad9a529019e98ae805f730f4d48
Test: Added new unit-tests and checked on marlin device.
/frameworks/base/cmds/statsd/tests/LogEntryMatcher_test.cpp
|