| a4fef2cb05b02c92928df23d136caf8d2726c0ef |
|
04-May-2017 |
Patrick Bellasi <patrick.bellasi@arm.com> |
utils/trace: remove tasks attribute
The Trace object is tracking a set of tasks which can be used as a default
for other trace analysis modules. However, this API is somehow confusing
since most of the times we still want to specify a different set of tasks
after the trace has been parsed.
This patch removes the usage of the tasks attributes at Trace parsing time
while ensuring that each of the current API clients define which tasks
they want to focus on.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
/external/lisa/libs/utils/test.py
|
| 4beebab43c51a68508d486859c24cd489cc9f9c5 |
|
28-Jun-2017 |
Brendan Jackman <bjackman@users.noreply.github.com> |
Merge pull request #415 from bjackman/lisa-test-iterations-cmdline
Add feature to set test iteration count from command line
|
| 9b99af5876fb3686d772d149419bcfb9a1d66e68 |
|
09-Jun-2017 |
Brendan Jackman <brendan.jackman@arm.com> |
Add feature to set test iteration count from commandline
This is a kludge, but I think doing this properly basically requires
implementing our own Nose test runner, or even our own test-running
framework entire. In the short term we really need this feature.
nosetests doesn't provide a way for the user to pass command-line
arguments to the test. To get around this, this commit adds a
parameter to lisa-test that sets an environment variable, which
LisaTest then reads to set or override the iterations count.
/external/lisa/libs/utils/test.py
|
| 59ad067fd62b8554c9656e635cc1dae172fd96c3 |
|
20-Apr-2017 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Add permitted_fail_pct attribute
This allows specifying a percentage of iterations of a given
test (tuple wload, conf, test method) can be permitted to fail.
If some tests fail but not enough to cross this percentage threshold, a warning
is printed.
/external/lisa/libs/utils/test.py
|
| 8ba281d201a54bd0bd34cc0248830f87af030269 |
|
28-Mar-2017 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Improve running tests with multiple iterations
Currently when you run a LisaTest with experiments_conf['iterations']!=0, the
test assertions are made for every experiment and the test is failed as soon as
any experiment produces a failure. Only the first failure encountered is
displayed.
With this patch all the results for each wload/conf pair are aggregated.
The problem is only solved across iterations: if multiple wloads or confs fail,
only the first wload/conf pair that was found to have failed will be
displayed.
/external/lisa/libs/utils/test.py
|
| 3a01b131174b28d2d5409bd060a58b08be9bf6d6 |
|
20-Apr-2017 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: rename logger to _log
For consistency with other LISA code
/external/lisa/libs/utils/test.py
|
| 1093851c73744730428f9a52afba512c5996c668 |
|
22-Feb-2017 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Replace runTest with overridden __init__
`runTest` is detected and run as a test by nosetests, since it has 'Test' in the
name. We just added it so that we could instantiate LisaTest objects
easily (like `t = FreqInvarianceTest()`). Instead, remove it and override
__init__ with a default `methodName` that doesn't have 'test' in the name.
Fixes https://github.com/ARM-software/lisa/issues/306
/external/lisa/libs/utils/test.py
|
| 968e72e29bfae2fe1329fa77cdc978dd92930b37 |
|
15-Feb-2017 |
Brendan Jackman <bjackman@users.noreply.github.com> |
Merge pull request #295 from derkling/fix-tests
Fix some tests broken by recent executor refactoring
|
| ef8ab219eff42cd7cc817f7a60fd62bbd9ec184b |
|
14-Feb-2017 |
Patrick Bellasi <patrick.bellasi@arm.com> |
utils/test: alias test configuration and workloads
Test configuration and workloads can be specified either using inline dicts
or JSON files. Moreover, they can be part of the same main dictionary or
separate ones. Finally, these information are tracked by different modules.
This patch adds two convenience alias which allows a test to get an easy
access to the set of configurations and workloads used to run the experiments.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
/external/lisa/libs/utils/test.py
|
| cb824ffc8974200583a815e6e5cc28fb7fbc6b2b |
|
14-Feb-2017 |
Patrick Bellasi <patrick.bellasi@arm.com> |
utils/test: fix documentation
A wload configuration can still be specified using a JSON file, let's report
that in the documentation.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
/external/lisa/libs/utils/test.py
|
| 08c88a53db462765141ae3d3771f809e01e41feb |
|
12-Jan-2017 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: add `experiments` alias
/external/lisa/libs/utils/test.py
|
| 07c24985f8c4964a11483db44e383d0ddf602566 |
|
12-Jan-2017 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Add public "runExperiments" method
Having it public makes it useful for using LisaTest outside of the influence of
nosetests, i.e. via notebooks.
/external/lisa/libs/utils/test.py
|
| 10fa32a75d284fcc2965e664407a5cca226430aa |
|
12-Jan-2017 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Fix get_sched_assert method
/external/lisa/libs/utils/test.py
|
| d69f7ab728d780c797440c6e174d73d65d503bf2 |
|
13-Dec-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Add empty runTest method
Defining a default, empty runTest allows test classes to be conveniently
instantiated. Since we run tests via nose, runTest isn't used. Instead
all the public methods with 'test' in the name are treated as test
methods.
/external/lisa/libs/utils/test.py
|
| 15a2cf422f76b8e79d3beb147c73e2b35cafc4fe |
|
23-Nov-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Add get_trace method
It's a good idea to unify the creation of Traces in one place, because
passing a different sent of events will result in different time
indexes due to the time normalization being based on the time of the
first event.
This is also a nice convenience method.
/external/lisa/libs/utils/test.py
|
| c278c2343c7f25083a80cb164b6bdc761d50050b |
|
22-Nov-2016 |
Patrick Bellasi <patrick.bellasi@arm.com> |
libs: use a more configurable logging strategy
This is a big re-factoring which update all the logging statements to make
use of a more sane and consistent configuration:
- each module uses the same logger configuration but refers to a custom
logger which has the same name of the module, this should allows a
fine grained configuration of the loglevels for each module.
- a new libs/utils/conf.py::LisaLogging class allows to load the
configuration and customize the root logger level.
- loggers are configured using a single logfile (logging.conf) which is
provided by default in the LISA's root folder.
Each logger can be configured to specify:
- loglevel: INFO by default for all modules
- output channels: console and logfile (lisa.log by default) for all modules
- string format: the same by default for all modules
If logging.conf does not specify a specific configuration for a logger, than
the default logger configuration applies, which is the one defined by the
"root" logger.
Notebooks and LisaTest are required to call at least one time the
LisaLogging::setup() method to properly intialize logging.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
/external/lisa/libs/utils/test.py
|
| 58d32dc365fa39e96a7875593611702a4035221b |
|
22-Nov-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Flesh out docstrings
/external/lisa/libs/utils/test.py
|
| 41814259b6125af57cdb5c1b26f96b60a636e4b1 |
|
11-Nov-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
LisaTest: Add methods to override for test configuration
These methods allow users of LisaTest to provide a `test_conf` member
which configures the test environment (TestEnv) and an
`experiments_conf` member to configure the experiments to set up the
experiments (Executor).
Users can override _getExperimentsConf with a method that generates the
Executor experiments configuration as a function of target information -
this is called _aftere_ the TestEnv is initialised and it is passed as a parameter.
/external/lisa/libs/utils/test.py
|
| 2bafed53a78810119b683ee1e83eb7ec972b8a96 |
|
17-Nov-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
executor: Rename tests_conf to experiments_conf
tests_conf is easily confused with the test_conf that TestEnv uses.
Also remove the default value since this parameter is required.
/external/lisa/libs/utils/test.py
|
| 324d7fa43f58a074292caca4f08e76a17ea94e75 |
|
11-Nov-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
executor: Take a TestEnv instead of a target_conf
/external/lisa/libs/utils/test.py
|
| 4fb683537195884f4eacf08f7eebf57ca3915dd5 |
|
02-Nov-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
libs/utils/test: Add get_end_time and get_window to LisaTest
/external/lisa/libs/utils/test.py
|
| 0445214a8018fd093f0de203f36bb951220292ce |
|
02-Nov-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
libs/utils/test: Add get_sched_assert method to LisaTest
/external/lisa/libs/utils/test.py
|
| 7487138b3df6377d2ef7c91a7f0028cf28f547a7 |
|
04-Nov-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
libs/utils/tests: Don't look at Executor config in LisaTest
We don't need to validate the config; Executor already does
that. Therefore we don't even need to parse it - just pass it blindly on
to Executor. That way we can use raw dicts instead of JSON files.
/external/lisa/libs/utils/test.py
|
| ad51cdcee3499fc20415ecba8db06dc76f4f7b6a |
|
13-Oct-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
libs/utils/test: Add new utility methods to LisaTest
/external/lisa/libs/utils/test.py
|
| 36b84bc7a7636fb48ab81af023f9bb1c6b6cc622 |
|
30-Sep-2016 |
Brendan Jackman <brendan.jackman@arm.com> |
libs/utils/test: Add experiment_test decorator
It's expected that tests often have to iterate over all the experiments
that were run in order to make assertions about the results of
each. This decorator abstracts that iteration, also providing the
experiment workload's task names for convenience.
/external/lisa/libs/utils/test.py
|
| 19af1b636016d16cda1b117fb6393b86f35e2274 |
|
26-Feb-2016 |
Patrick Bellasi <patrick.bellasi@arm.com> |
libs/utils/test: use cls instead of self for class methods
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
/external/lisa/libs/utils/test.py
|
| df15a12bbd7f5b9cdc798c3a4a952f1d0c50a46d |
|
26-Feb-2016 |
Patrick Bellasi <patrick.bellasi@arm.com> |
libs/utils/test: fix multiple run of the executor
Tests executions and data collection is driven by the Executor module which
should be executed just one time to run all the experiments required by
the following tests.
The current implementation of LisaTest creates the Executor object
and run its from within the ctor method of the base class. This ctor is
called by nosetests before each test runs. This imply that we currently run
data collection multiple times.
This patch fixes this issue by switching to the usage of the setUpClass
class method, which is granted to be called just one time by nosetest.
This mehod is used in each test to call the LisaTest::_init() method
which is now the one in charge to initialized and execute tests.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
/external/lisa/libs/utils/test.py
|
| f80919dc5a5b568b7ec8215d82004fe844b9e679 |
|
12-Feb-2016 |
Patrick Bellasi <patrick.bellasi@arm.com> |
libs/utils/test: add base class for experiments execution
Most of the LISA tests requires the execution of a set of experiments to
collect traces required for the tests. The execution engine provided
by libs/utils/executor provides the minimal code to run a pre-configured
set of experiments, but still we need some glue code to configure the
executor as well as (eventually) post-process their results.
This patch adds a python unittest based class which provides:
1. the glue code to load a tests configuration and check its conformance
with the executor engine
2. execute the tests
This base class adds support also for the definition of a pre-defined set
of testing callbacks. A test writer can thus easily extends a LISA test
by specifying code to be run:
a) before experiments execution (LisaTest::_experimentsInit)
b) after experiments have completed (LisaTest::_experimentsFinalize)
Configuration checking (LisaTest::_checkConf) and experiments execution
(LisaTest::_runExperiments) are also methods which can be eventually
re-defined by a specif test instance.
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
/external/lisa/libs/utils/test.py
|