| bb255b6e87dc343eb90dec998be1cf153106ab65 |
|
26-Apr-2018 |
Mika Raento <mikie@google.com> |
Unknown dimension tests for partitioning and hvx
More tests for unknown dimensions:
- more extensive tests for partitioning
- change existing unknown dimension test to use quantized tensors so
that it uses the hvx driver when available
Added logging to device selection to make it easier to debug why hvx was
not getting chosen.
TODO: parametrize unknown dimensions test for both floats and quantized
tensors so that it can be used to test other drivers as well
Test: NeuralNetworksTest_static
Bug: 72448000
Bug: 77234888
Change-Id: Id9768cbd871a04d968627411cff90cfe6e73d84a
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 1e9666208595bc251a8958155b1e41eca90b69db |
|
12-Apr-2018 |
Michael Butler <butlermichael@google.com> |
NNAPI: Add execution preference to prepareModel (runtime)
A model can be compiled in different ways to optimize for different
use-cases. This CL propagates the execution preference across the HAL so
that the NN service can better fit the users needs.
Bug: 77864669
Test: mma
Test: NeuralNetworksTest_static
Test: VtsHalNeuralnetworksV1_1TargetTest
Change-Id: I93079b18224ea10cb0a23bfbdc1ed80e302071a8
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 57167f7ec8bfe682139a9a4d60cd8aa913899441 |
|
16-Feb-2018 |
Michael Butler <butlermichael@google.com> |
Simplify lifetime management of ModelBuilder
ExecutionStep fully owns the ModelBuilder it contains, and only provides
references to the subModel. This change simplifies the lifetime
management to reflect the ownership, removing the unnecessary strong
pointer behavior.
Additionally, this CL removes the unnecessary virtual destructor in the
ModelBuilder class, which was likely an artifact of a previous
implementation.
Test: mm
Test: NeuralNetworksTest
Test: VtsHalNeuralnetworksV1_0TargetTest
Change-Id: I62571d9a61fd8bbc193586756267ad4ec8888e27
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| c4c264098a728268ad28084ea6e0263d9c1d7868 |
|
16-Nov-2017 |
David Gross <dgross@google.com> |
Fix for cases where a model output is a submodel input.
Also:
- Added more error checking.
- Added more logging.
- Fixed some comments.
- Simplified some testing interfaces.
- Fixed some indentation.
Test: nn/runtime/test
Bug: 69166603
Change-Id: I38e33f28ff7885958a8146c650a3c59a2372d803
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| def0a14aa77689f12120cfb4f136eea659038cc0 |
|
05-Oct-2017 |
David Gross <dgross@google.com> |
Partitioning interface changes to enable unit testing.
Bug: 63905942
Test: mma (userdebug)
ml/nn/runtime/tests (userdebug)
(no failures, logcat looks plausible)
Change-Id: Ib9a5eb75b9e4e2c3534f392d5fb0e5ba3555197d
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 5e8feed5e8a07bab1ec395e5a01bb8900db00cec |
|
05-Oct-2017 |
David Gross <dgross@google.com> |
Add CPU fallback and remove non-plan driver path.
When we do ExecutionPlan-controlled execution, and
there is a failure at some stage of the execution,
fall back to cpu execution for that one stage (if
possible) or for the entire model (otherwise).
When we don't do ExecutionPlan-controlled execution,
we no longer attempt to find a suitable driver for
full-model execution -- instead, we execute the full
model on the cpu. (Rationale: This avoids having
to implement cpu fallback outside ExecutionPlan-
controlled execution.)
Bug: 63905942
Test: mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 1,
no new failures; hand-insert
errors in the runtime in a few
places to exercise some of the
fallback paths, and verified that
behavior is as expected)
Change-Id: I9ec8fba632f94e04840a786575d338d06fd2dcd3
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 8fb14e90ceb360adfbac0f708d27161b7c5b7fc5 |
|
05-Oct-2017 |
David Gross <dgross@google.com> |
Allocate all inter-partition temporaries to a single Memory object.
Previous behavior was to allocate each inter-partition temporary
to its own Memory object. The Android system limits the number
of shared memory objects, which is what our Memory objects represent;
so as long as all inter-partition temporaries have the same lifetime
(which they do in our present implementation) it is better to put
them all in the same Memory object.
Bug: 63905942
Test: mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 1 and 2;
no new failures)
Change-Id: Ia0ccf4e47aa93d40db0ff5f496662523622caf08
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| a2a03635c8f215cb75be68ff1939bf4dec285ef8 |
|
03-Oct-2017 |
David Gross <dgross@google.com> |
Make partitioning and execution of partitioned plan the default.
Also, rename the debug.nn.partition.test property to
debug.nn.partition, and slightly change the interpretation of its
values:
2: the default: partitioned execution, with a fallback to
non-partitioned execution if there is a partitioning failure
3: a new value: partitioned execution, with NO fallback to
non-partitioned execution if there is a partitioning failure
Bug: 63905942
Test: mma (user)
ml/nn/runtime/tests (user)
(no new failures, logcat looks plausible,
debug.nn.partition 0 and 3 have no effect)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition unspecified, 0, 1, 2, and 3;
no new failures, logcat looks plausible; also tried
manually introducing failures several places in the
partitioning process to ensure that simulation/fallback
behavior works properly for different values of
debug.nn.partition)
Change-Id: I04c71d7009aa0d1c971833a2af5816ff5830ecc9
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 96811e2b1347889a25bd9686f47ca3cbf061fb1b |
|
02-Oct-2017 |
David Gross <dgross@google.com> |
Manage inter-partition temporaries.
This means: Allocate a Memory object for each TEMPORARY in the original
model that is live across partition boundaries; and tell each
StepExecutor about the mapping from a submodel input or output to the
appropriate Memory object.
With these changes, we can now fully execute multiple-partition plans.
Limitations:
- Still under control of debug.nn.partition.test property.
- Cannot handle operands of unknown size, except for model (not
submodel) inputs and outputs.
- Execution is synchronous.
Also: Teach sizeOfData() to work on scalars.
Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 0, 1, and 2;
no new failures, logcat looks plausible,
confirmed that GeneratedTests.mobilenet
runs on multiple partitions)
Change-Id: I58b763bc68bf8fe2e0306610c775b854e9292f76
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 891b10f7048c62a37a74c4b570be220089dfd55e |
|
02-Oct-2017 |
David Gross <dgross@google.com> |
Add ExecutionPlan::Controller support for Compound plans.
This includes proper handling of ExecutionBuilder inputs
and outputs, but not of inter-partition temporaries that
need to become inputs and outputs.
When debug.nn.partitioning.test > 0, always use Controller,
instead of the prior behavior of only using it when
debug.nn.partitioning.test > 1 and the plan is "in scope"
for execution.
If a combination of the partitioning property value and
the plan's shouldBeExecutable() say the plan shouldn't
be given control, then we do simulated execution under
control of the plan before real execution without using
the plan. This allows us to test StepExecutors.
("Simulated execution" means iterating through the
StepExecutors without using them to perform any computation.)
Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 0, 1, and 2;
no new failures, logcat looks plausible)
Change-Id: I3a05f5479e906eb13f65e27b17e7129bfcceac0d
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| b26049114bc4c64e6bea3a5d5d129fcaec8e69b6 |
|
02-Oct-2017 |
David Gross <dgross@google.com> |
Use ExecutionPlan to control (possibly multiple step) execution.
- Split StepExecutor out from ExecutionBuilder. ExecutionBuilder
represents the description of an execution passed in through the
NNAPI; StepExecutor is used to perform a single "step" of that
execution on a single device (or CPU).
- Introduce ExecutionPlan::Controller class. This is a token used
to allow ExecutionBuilder::startCompute() to iterate over a sequence
of steps provided by ExecutionPlan, obtaining a StepExecutor for each
one.
There should be no change from previous behavior; this is code
reorganization in preparation for supporting execution of partitioned
models. While ExecutionPlan is still only used when the property
debug.nn.partition.test is set to 0, and it still controls the
execution only when that property is set to 2, we do use the new
StepExecutor always.
Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 0, 1, and 2;
no new failures, logcat looks plausible)
Change-Id: I17473210bcc5de40603d439ec49374030d1b8f36
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 1f4381539b7e89c42336ee7cd1addb9a4c317b34 |
|
28-Sep-2017 |
David Gross <dgross@google.com> |
Finish implementing single-partition graphs.
This includes wiring up the partitioning algorithm
to the NN API compilation and execution phases under
debug.nn.partition.test=2: When we have a single-
partition graph, we rely on the device selection and
compilation performed by the partitioning algorithm,
rather than doing it at execution time as we were
before.
Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 0, 1, and 2;
no new failures, logcat looks plausible,
compilation happens at the expected times)
Change-Id: I6cd8b53625588d5aca81c87cc8600735e384b8e0
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 8913ae3283de7752aed108c1b26aef1adacb049f |
|
27-Sep-2017 |
David Gross <dgross@google.com> |
Finish constucting submodels for graph partitions.
- Compute submodel outputs.
- Finish each submodel by calling setInputsAndOutputs() and finish().
- Compile each submodel (for testing purposes).
- Fix bugs in device queue management.
- More debugging output.
Bug: 63905942
Test: ml/nn/runtime/tests
(with debug.nn.sample.supported=1
and debug.nn.partition.test 0 and 1;
no new failures, logcat looks plausible)
Change-Id: I5079d3874f183fe06a1660328ae485c01b40170c
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| f4e1c640547a44c7a37209e81ee5f3831b7d0fdc |
|
22-Sep-2017 |
David Gross <dgross@google.com> |
During graph partitioning, add operations and operands to subgraphs.
Bug: 63905942
Test: ml/nn/runtime/tests
(with debug.nn.sample.supported=1
and debug.nn.partition.test 0 and 1;
no new failures, logcat looks plausible)
Change-Id: I54ab08646ffa16608a576e11105be4feb22e347b
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 0b9453e41a544f9c780eaa15ad65136ad4662ccb |
|
23-Sep-2017 |
David Gross <dgross@google.com> |
More debugging tools for graph partitioning.
1) Under property debug.nn.partition.test, run partitioning
algorithm fropm CompilationBuilder::finish(). We don't
use the results -- this is to produce log output and to
verify that we don't crash.
2) Add a little more LOG(DEBUG) output.
Bug: 63905942
Test: mm (userdebug and user)
Change-Id: I8f0f4477cb9a564dcd55536677687c7914a9b86e
/frameworks/ml/nn/runtime/ExecutionPlan.h
|
| 91e8417c4c395e3922d12abfd956b93b71121976 |
|
14-Sep-2017 |
Jean-Luc Brouillet <jeanluc@google.com> |
Partition the requests to run on multiple devices.
Partition the request so that subgraphs can run on the best device.
Also respects the capabilities of a device so that we don't try
to run a device incapable to run a graph.
Bug: 63905942
Test: Compiled and ran unit tests. We'll want more advance tests.
Change-Id: I8662fa719661e5b68c47cd75703edb5937fc8f0a
/frameworks/ml/nn/runtime/ExecutionPlan.h
|