| 951a1eeb114a6aacc87d6df67b9090bb94afe3bf |
|
17-Apr-2018 |
Mika Raento <mikie@google.com> |
Add test for dimensions unknown at compile time
Adds a test for operands which have one or more unknown dimensions at
compile time but are fully known at execution time. This is meant to be
supported but had a bug earlier.
Bug: 72448000
Test: NeuralNetworksTest_static (emulator)
Change-Id: I7ad5e6b6c29f8fe45098c9291d6a2d4b00c58eb8
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 43d018d1a66872031568857fa369024c9640a085 |
|
20-Mar-2018 |
Miao Wang <miaowang@google.com> |
Enforce validations for ANeuralNetworksExecution_setInput/Output
- Optional output / input must have both nullptr as buffer and 0 as
length.
- If newType is NULL, the operand dimensions must be fully specified.
And ModelArgumentInfo.dimensions will be the same as the operand
dimensions.
- For non-optional input / output, user provided length must match the
operand size.
Bug: 75980924
Test: mm
Test: NeuralNetworksTests
Change-Id: Iaa8f8193c8a6911afd0990fcf0659d0050de833b
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| b74d2837ab1687c1a4f913aa5f90a9838efe0add |
|
21-Feb-2018 |
Miao Wang <miaowang@google.com> |
Add relaxed execution tests
- Extend the test generator and test wrapper to allow relaxed
execution.
- Make the accpeted error range to be 5 ULP of FP16.
- Copy and modify existing V1_0 tests to be V1_1 relaxed tests.
Bug: 63911257
Test: mm
Test: Relaxed tests pass on devices with FP16 support
Change-Id: I8954aeeebcf7312d7d94580992708f1b42e2b32e
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 663155d58ca1e1eb42e01495e236258e0c00d40f |
|
04-Dec-2017 |
David Gross <dgross@google.com> |
Add ::android::nn::graphDump() method for graph visualization.
Call this method to write a representation of a model in Graphviz (.dot)
format.
Test: nn/runtime/test
Change-Id: If999e67c9501ae3ccadb8a62ec0cdc8b1d379253
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 4062279c74f2c995349b7ff04f22feb89eb81e47 |
|
19-Oct-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Slicing a given model up to the given first N operations
Also regenerated all CTS, VTS and unit tests, as a test harness function
is renamed.
Test: test.py and sliced mobilenet_quantized up to first 20 ops
Test: unit tests and VTS on Sailfish
Bug: 67781676
Change-Id: I68dcdbda17f8bedc98b1da573a8f884da0d5722d
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 4107f3201f58a9780e2550a83e90b6ec8a80cb1d |
|
27-Oct-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Modernize the test specs of various outdated tests
These generated test were in an earlier format for test_generator,
and were not re-generated when running generate_test.sh
This CL converts the test spec to a modern test spec. Removed manually
written includes in TestGenerated as well.
Also removed test harness for those outdated tests since it is no
longer used.
Bug: 68383630
Test: NeuralNetworksTest on Sailfish
Change-Id: I994317516794f6198436ee5c476cc2f4dbf764da
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| a1ef918e19d3d66773dfbd6b3ccc6bacfd153a52 |
|
27-Oct-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Modernize the test spec of mobilenet_224_gander_basic_fixed
The generated test was in an earlier format for test_generator in which
weights were defined as automatic variables. That is now considered a
bug.
This CL fixes the problem by converting the test spec to a modern test
spec. Removed manually written includes in TestGenerated as well.
Bug: 68383630
Test: NeuralNetworksTest on Sailfish
Change-Id: I4a4325109bb73ab1d63169735e8f1170fe0eaa83
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| c86cfc7e23d5a1ec0f81dbb049da9256718d56f0 |
|
26-Sep-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Refactor test harness
Fix unaddressed comments in previous CLs on test harness; rewrite macros
using templates.
Bug: 67597726
Test: NeuralNetworksTest on Angler
Change-Id: I19fe804a71554246b953471e2966335cfd9170e5
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 820215d28bed6c90f696cde0f282445d16da432e |
|
05-Oct-2017 |
Miao Wang <miaowang@google.com> |
Allow fine grain verbose logging control through system property.
- Existing DEBUG and INFO change to the new mechanism, existing ERROR and
WARNING unchanged.
- By default, all verbose logging are disabled.
- setprop debug.nn.vlog 1 enables all verbose logging.
- setprop debug.nn.vlog "tag1 tag2 ..." only enable the selected tags.
- Available tags:
all: same as 1.
model: enable logging with tag MODEL.
compilation: enable logging with tag COMPILATION.
execution: enable logging with tag EXECUTION.
cpuexe: enable logging with tag CPUEXE.
manager: enable logging with tag MANAGER.
driver: enable logging with tag DRIVER.
Bug: 63905942
Test: mm
Test: NeuralNetworksTests pass
Test: manually set property to allow individual module to logging
verbosely
Test: tested with debug.nn.partition default -> 0 -> 1 to make
sure getProp still works fine
Change-Id: Iaa0ffa62176dabcdc35058748597df720fd6e47e
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 62cc2758c1c2d303861e209f26bddcf4d7564b73 |
|
28-Sep-2017 |
Jean-Luc Brouillet <jeanluc@google.com> |
Fix problem passing null optional operands.
Allow optional operands to be specified in the model as having no value.
Modify the HAL to allow passing missing operands. Change the validation
of most operations to verify that none of their operands are missing.
Bug: 63905942
Test: Compiled and ran runtime/test.
Change-Id: I378bc51847a8fb30cb2d36c9bf707ebadc79289b
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 01a0489d03adb318894763c80b107366043e7885 |
|
20-Sep-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Downgrade generated test harness for NDK libc++
NDK libc++ does not yet support full C++14. This CL replaced use of
std::get with type as well as auto lambda arguments with equivalent
C++11 constructs.
Test: mm for walleye-eng
Test: NeuralNetworksTest on angler-eng
Bug: 63905942
Change-Id: I602c726b5133ec08e9a65892ec8d962ccee9a8ee
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 1fe4cea7a58045eb715cade0535220b5fb7f658d |
|
08-Sep-2017 |
Yang Ni <yangni@google.com> |
Make each test have its own copy of inputs
Bug: 63905942
To avoid issues across multiple runs of the same stateful OPs such as
RNN cells.
Test: mm and ran NeuralNetworksTest on Marlin
Change-Id: I1e3c20ebc079a70aa5238a3a5ab2bee4d3a5c7e7
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 39dcb0f2cc87b0eba06b314a91d55e50efea5abe |
|
14-Sep-2017 |
David Gross <dgross@google.com> |
Merge changes I5922f13a,I7194e277 into oc-mr1-dev
* changes:
Bring back Event, and wait on Event rather than on Execution.
Make Compilation API synchronous.
|
| 65aa556323f4a054f80a75b6c4c721b2a7ed3298 |
|
14-Sep-2017 |
David Gross <dgross@google.com> |
Make Compilation API synchronous.
Per advice from the API council, the Compilation API no longer exposes
asynchronous behavior to the user -- rather than start() and wait()
APIs, we now have a finish() API.
Also:
- Track whether or not Compilation::finish() has been invoked, for
the sake of API usage checking.
- Changed Model::finish() to return Result not int, for consistency.
- Made Model and Compilation documentation more similar to one another.
- Fixed typos in documentation.
NOTE: It's possible to implement compilation asynchronously behind the
synchronous API -- e.g., Compilation_finish() starts compilation
asynchronously, and we implicitly wait for it to complete at the point of
Request_startCompute(). However, this would defeat one of the purposes of
an explicit Compilation API -- giving the application control over when
to spend the time compiling, to keep it off a critical timing path, such
as repeatedly applying a model to a sequence of frames in real time.
Bug: 63905942
Test: nn/runtime/tests
Change-Id: I7194e277aa6211af66794d917dcc5254c7d81af4
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 276087294775dd6e4cb90a6af943c12ef6d97449 |
|
14-Sep-2017 |
Ray Sung <ijsung@google.com> |
Merge changes I8844fd13,I99a43840 into oc-mr1-dev
* changes:
Test spec for the Basic RNN operation
Support ignored output in test generator
|
| bf15cf2352cd159679210c0e575a26e2e37b770a |
|
08-Sep-2017 |
Yang Ni <yangni@google.com> |
Test spec for the Basic RNN operation
Bug: 63905942
Added test spec for RNN.
Additional changes to make the generated test work:
* Updated auto test harness to use EXPECT_NEAR with an absolute
error of 1.0e-5 for comparison of floating point numbers.
* Made the generate example (inputs and golden outputs) local variable
to avoid issues in the second test run.
Test: run generate_test.sh and run the generated tests.
Change-Id: I8844fd134c6718707d63af0dc4b8447ce5257954
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| ec8e1c71d11f7b5abe306e0c9c6af7da755bd46b |
|
13-Sep-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Support ignored output in test generator
Added a new construct IgnoredOutput for specifying outputs that are
excluded from comparing against golden, required by ops like SVDF.
Also regenerated CTS tests since a test now has one more generated
function bool is_ignored(int);
In addition, updated test generator regression suite accordingly.
Bug: 63905942
Test: NeuralNetworkTest on Angler
Test: VtsHalNeuralnetworksV1_0TargetTest on Angler
Test: tools/test_generator/tests/test.py on host
Change-Id: I99a438400c264cd3b4a12014e1cc473a7d324df6
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 7612f29b31f97f3b15769264131566b36dea9a25 |
|
12-Sep-2017 |
Jean-Luc Brouillet <jeanluc@google.com> |
Remove the initialization and shutdown APIs.
Initialize the list of drivers as needed rather than
requiring an API call. This initialization will be
triggered when compiling the first model.
Bug: 63905942
Test: Compiled and ran the unit tests.
Change-Id: I1893a9f9d96d9d241916de3a7dd3cb7a54e33b06
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 3ced3cfd5b8f22b632c35f24e585c4847383b195 |
|
13-Sep-2017 |
David Gross <dgross@google.com> |
Rename Request to Execution at the API level.
There's a separate notion of Request at the HIDL level;
this is not renamed.
Bug: 63905942
Test: nn/runtime/tests
Change-Id: I671e3b387d735a9091dc5a2d14afbea6b45f444c
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 420cf7bdb689bc53b2a4e8c5f9eca42f7a06c1c0 |
|
11-Sep-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Generated VTS tests and minor tweak on test harness
Changes:
- Export generated models and examples path in Android.bp
- Avoid duplicated definitions in TestHarness.h when included multiple
times
- Add generated VTS models for VTS
- Update test generator to compute scale and offset from fmin and fmax
Bug: 63905942
Test: VTS and CTS unit testcases with sample driver enabled on Angler
Change-Id: I3e7c5a2aa2c9945e7cd8cdf93968ef88f2b3bbfa
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| d756f33874dccaedd99eef78942eaf4ae2389ed9 |
|
12-Sep-2017 |
David Gross <dgross@google.com> |
Merge "More discipline for models and requests." into oc-mr1-dev
|
| 544739620cd7f37d40524d2407c92042e485c73f |
|
11-Sep-2017 |
David Gross <dgross@google.com> |
More discipline for models and requests.
- Must explicitly call new finish() API on model before compiling
or freeing.
- Must not modify a request once start() has been called on it.
Bug: 63905942
Test: nn/runtime/tests, nn/common/operations tests
Change-Id: Ifc6e614bda647d729e8702023a02613e629ca6a0
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 7deb155b081d4c4553a465d5910bdd30fb7b72cf |
|
11-Sep-2017 |
David Gross <dgross@google.com> |
Merge "Add compilation APIs." into oc-mr1-dev
|
| 62acb66f23ddf8fa1b623f9e5950f1cce5dee9de |
|
11-Sep-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Revise generated test harness
Changes:
- Use EXPECT_FLOAT_EQ for floating point comparison
- Replace most macros with templates and lambdas.
Bug: 63905942
Test: NeuralNetworksTest on Angler
Change-Id: I5ec88763b5ace3414e2f495c5093f53bdf127885
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 83e24dc4706a5b7089881a55daf05b3924fab3b7 |
|
10-Sep-2017 |
David Gross <dgross@google.com> |
Add compilation APIs.
For now, compilation is implemented as a synchronous no-op.
Bug: 63905942
Test: nn/runtime/tests, nn/common/operations tests
Change-Id: I1cb96a7e800671d0c70c9cc1b7eab6f8244232cb
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 6a0d306cf902e13ab147c7533b2cb02540ee66d5 |
|
08-Sep-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Support example that has tensors of different types
Operations like LSH_PROJECTION requires tensors of different types. This
CL adds support to the generator and harness (for native tests).
Revised:
- Test generator
- Test harness
- Existing examples
Also regenerated examples from testcases.
Bug: 63905942
Test: tools/test_generator/test.py
Test: NeuralNetworkTest on angler
Test: cts-tradefed run commandAndExit cts -m CtsNNAPITestCases
Change-Id: I000cdac92468659709be18599a425ef82695011a
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| ad36b7a0dd7468bbe6a0433a52cbf89561ff5538 |
|
31-Aug-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Support quantized models and examples in tests
- Revised test_generator
- Added a quantized example (average pool) to
the regression test suite
- Added generated test to NeuralNetworksTest
- Revised test harness
Bug: 63905942
Test: tools/test_generator/test.py
Test: /data/nativetest64/NeuralNetworksTest/NeuralNetworksTest on Angler
Change-Id: I8d0ae3031db5d54ac9f46c7b4d916d0560265730
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 5d58071b9afb2162bcd53525ce1c8610c5ca2cd8 |
|
31-Aug-2017 |
Miao Wang <miaowang@google.com> |
Add basic tests for pooling and Relu[X] operations.
- The tests serves as basic sanity check for FLOAT32 variant of the
following ops:
AVERAGE_POOL
MAX_POOL
L2_POOL
RELU
RELU1
RELU6
Bug: 63905942
Test: mm
Test: All unit tests passing on sailfish
Change-Id: I57ad067cf0aa0219ec144b336386c24b2f521dc4
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| dcd2fbf1da7ebdc1aa1b57c74db4fffd2911e3b8 |
|
14-Jul-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
First implementation of NNAPI test generator.
Please see tools/test_generator/tests/P_full/addfloat.mod.py for how
a test model may looks like.
Bug: 63905942
Test: tests/test.py on host
Test: NeuralNetworksTest --gtest_filter=GeneratedTests.* on Angler
Change-Id: Ia8592da130092248269ab3c8c58dcac184dc29ca
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|
| 242c6dc1f314646f1a87c66140f26d7623cc399a |
|
29-Jul-2017 |
I-Jui (Ray) Sung <ijsung@google.com> |
Test driver, generated models and examples from TFLite
This CL contains models and corresponding examples converted from TFLite
tests (under generated/) and a gtest-based driver (TestGenerated.cpp).
Bug: 63905942
Test: mm
Test: NeuralNetworksTest --gtest_filter=GeneratedTests.* on Angler
Change-Id: I8c062fd9e96585266f7ea9561e79987aa4ddc15c
/frameworks/ml/nn/runtime/test/TestGenerated.cpp
|