History log of /frameworks/ml/nn/runtime/ExecutionBuilder.h
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
aba8381570d7cc4b5431d741132d6f80519201eb 01-Feb-2018 Michael Butler <butlermichael@google.com> Upgrade NeuralNetworks v1.0 to v1.1 followup CL

This CL addresses some remaining comments from ag/3499102.

Specifically, this CL:
1) Overloads CpuExecutor::run to work with both V1_0::Model and V1_1::Model
2) Documents all VersionedIDevice methods
3) Captures specific variables by reference in lambdas (instead of the general [&])
4) Forward-declares VersionedIDevice instead of including the header.

Bug: 63911257
Test: mm
Test: cts and vts test binaries
Change-Id: I457150346638ea8a2be912046c2bc3c49e95b44e
(cherry picked from commit 8ea26fd92ff697734e5ba690b61f9b0aeda2abf2)
/frameworks/ml/nn/runtime/ExecutionBuilder.h
75886e77f9ca074173a49283b5c0a8c182d98977 23-Jan-2018 Michael Butler <butlermichael@google.com> Upgrade NeuralNetworks v1.0 to v1.1.

For Android P, the NNAPI is getting new operations. To support this
change, the NN runtime needs to handle both v1.0 and v1.1. This change
brings the NN runtime up to v1.1 and adds a VersionedIDevice class which
manages both v1.0 and v1.1 calls.

Test: mm
Test: cts and vts test binaries
Change-Id: Iaec1dcfe00c1712a0ac89954ae142d4ea126989d
/frameworks/ml/nn/runtime/ExecutionBuilder.h
c4c264098a728268ad28084ea6e0263d9c1d7868 16-Nov-2017 David Gross <dgross@google.com> Fix for cases where a model output is a submodel input.

Also:
- Added more error checking.
- Added more logging.
- Fixed some comments.
- Simplified some testing interfaces.
- Fixed some indentation.

Test: nn/runtime/test

Bug: 69166603

Change-Id: I38e33f28ff7885958a8146c650a3c59a2372d803
/frameworks/ml/nn/runtime/ExecutionBuilder.h
c2f1c1198c84f5a75fc2305935155f33b8ff5db2 13-Nov-2017 David Gross <dgross@google.com> Add more testing interfaces to CompilationBuilder.

- Ability to override DeviceManager's partitioning setting.
- Ability to compile against a specified list of Devices.
- Ability to retrieve the ExecutionPlan.

Test: nn/runtime/test with debug.nn.partition unset and set to 0
Change-Id: Iae2ae7fdaf13335acd2856cdc93fa259bbe52bba
/frameworks/ml/nn/runtime/ExecutionBuilder.h
5e8feed5e8a07bab1ec395e5a01bb8900db00cec 05-Oct-2017 David Gross <dgross@google.com> Add CPU fallback and remove non-plan driver path.

When we do ExecutionPlan-controlled execution, and
there is a failure at some stage of the execution,
fall back to cpu execution for that one stage (if
possible) or for the entire model (otherwise).

When we don't do ExecutionPlan-controlled execution,
we no longer attempt to find a suitable driver for
full-model execution -- instead, we execute the full
model on the cpu. (Rationale: This avoids having
to implement cpu fallback outside ExecutionPlan-
controlled execution.)

Bug: 63905942
Test: mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 1,
no new failures; hand-insert
errors in the runtime in a few
places to exercise some of the
fallback paths, and verified that
behavior is as expected)

Change-Id: I9ec8fba632f94e04840a786575d338d06fd2dcd3
/frameworks/ml/nn/runtime/ExecutionBuilder.h
8fb14e90ceb360adfbac0f708d27161b7c5b7fc5 05-Oct-2017 David Gross <dgross@google.com> Allocate all inter-partition temporaries to a single Memory object.

Previous behavior was to allocate each inter-partition temporary
to its own Memory object. The Android system limits the number
of shared memory objects, which is what our Memory objects represent;
so as long as all inter-partition temporaries have the same lifetime
(which they do in our present implementation) it is better to put
them all in the same Memory object.

Bug: 63905942
Test: mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 1 and 2;
no new failures)

Change-Id: Ia0ccf4e47aa93d40db0ff5f496662523622caf08
/frameworks/ml/nn/runtime/ExecutionBuilder.h
e3178825b8686f3300a895572691a2e8c1f0676b 05-Oct-2017 David Gross <dgross@google.com> Asynchronus partitioned execution, plus minor code cleanup.

Partioned execution is now asynchronous end-to-end, although
the individual execution steps are serialized with respect to
one another.

Other changes:
- Minor code cleanup per comments from previous CL
https://googleplex-android-review.git.corp.google.com/#/c/platform/frameworks/ml/+/3000624/
including comments on memory management for
inter-partition temporaries.
- Remove "simulation" form of partitioning, and renumber
values of debug.nn.partition property accordingly.
- Fix android:nn:wrapper::Event -- add missing contructor
and fix comment.

Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(and spot check that debug.nn.partition unspecified,
0, and 1 exhibits the correct control over
partitioning behavior)

Change-Id: I5767d2144c4a42258444a6445866529919d4fe8d
/frameworks/ml/nn/runtime/ExecutionBuilder.h
69d24339cfca4979e4f3af884c455935a9909793 04-Oct-2017 David Gross <dgross@google.com> Merge changes I04c71d70,I58b763bc into oc-mr1-dev

* changes:
Make partitioning and execution of partitioned plan the default.
Manage inter-partition temporaries.
a3bed7887a65e0662080d2630e228954ca578ca3 04-Oct-2017 TreeHugger Robot <treehugger-gerrit@google.com> Merge "Fix problem passing null optional operands." into oc-mr1-dev
96811e2b1347889a25bd9686f47ca3cbf061fb1b 02-Oct-2017 David Gross <dgross@google.com> Manage inter-partition temporaries.

This means: Allocate a Memory object for each TEMPORARY in the original
model that is live across partition boundaries; and tell each
StepExecutor about the mapping from a submodel input or output to the
appropriate Memory object.

With these changes, we can now fully execute multiple-partition plans.

Limitations:
- Still under control of debug.nn.partition.test property.
- Cannot handle operands of unknown size, except for model (not
submodel) inputs and outputs.
- Execution is synchronous.

Also: Teach sizeOfData() to work on scalars.

Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 0, 1, and 2;
no new failures, logcat looks plausible,
confirmed that GeneratedTests.mobilenet
runs on multiple partitions)

Change-Id: I58b763bc68bf8fe2e0306610c775b854e9292f76
/frameworks/ml/nn/runtime/ExecutionBuilder.h
62cc2758c1c2d303861e209f26bddcf4d7564b73 28-Sep-2017 Jean-Luc Brouillet <jeanluc@google.com> Fix problem passing null optional operands.

Allow optional operands to be specified in the model as having no value.
Modify the HAL to allow passing missing operands. Change the validation
of most operations to verify that none of their operands are missing.

Bug: 63905942
Test: Compiled and ran runtime/test.
Change-Id: I378bc51847a8fb30cb2d36c9bf707ebadc79289b
/frameworks/ml/nn/runtime/ExecutionBuilder.h
033b8a6ce8ebd2a01ecccc6bae96d0fff8d4964e 23-Sep-2017 Michael Butler <butlermichael@google.com> NNAPI Runtime: Change IEvent to explicit callbacks

IEvent was a synchronization primitive which caused some confusion
in the interface. Originally the event object was paired with an
asynchronous task, and the asynchronous task would signal this event
when the corresponding output was ready to be used.

In the case of IDevice::prepareModel, the function call would return an
IPreparedModel object that was not guaranteed to be prepared until the
runtime had returned from waiting on the corresponding event object.
The event object has been changed to two explicit callbacks--
IPreparedModelCallback and IExecutionCallback. Now,
IDevice::prepareModel no longer returns an unfinished IPreparedModel;
instead, it will pass the IPreparedModel object to the runtime through
IPreparedModelCallback::notify. When the runtime retreives the
IPreparedModel object, the asynchronous task has already finished
preparing the model.

The two callbacks are used for different purposes. Each has its own
version of notify to pass the data back to the runtime:
* IPreparedModelCallback::notify(ErrorStatus, IPreparedModel)
* IExecutionCallback::notify(ErrorStatus)

Bug: 63905942
Test: mm, vts, ml/nn/runtime/tests
Change-Id: I5cd2734a9fc1377546d55a12955445b6ccdd47b3
/frameworks/ml/nn/runtime/ExecutionBuilder.h
891b10f7048c62a37a74c4b570be220089dfd55e 02-Oct-2017 David Gross <dgross@google.com> Add ExecutionPlan::Controller support for Compound plans.

This includes proper handling of ExecutionBuilder inputs
and outputs, but not of inter-partition temporaries that
need to become inputs and outputs.

When debug.nn.partitioning.test > 0, always use Controller,
instead of the prior behavior of only using it when
debug.nn.partitioning.test > 1 and the plan is "in scope"
for execution.

If a combination of the partitioning property value and
the plan's shouldBeExecutable() say the plan shouldn't
be given control, then we do simulated execution under
control of the plan before real execution without using
the plan. This allows us to test StepExecutors.

("Simulated execution" means iterating through the
StepExecutors without using them to perform any computation.)

Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 0, 1, and 2;
no new failures, logcat looks plausible)

Change-Id: I3a05f5479e906eb13f65e27b17e7129bfcceac0d
/frameworks/ml/nn/runtime/ExecutionBuilder.h
b26049114bc4c64e6bea3a5d5d129fcaec8e69b6 02-Oct-2017 David Gross <dgross@google.com> Use ExecutionPlan to control (possibly multiple step) execution.

- Split StepExecutor out from ExecutionBuilder. ExecutionBuilder
represents the description of an execution passed in through the
NNAPI; StepExecutor is used to perform a single "step" of that
execution on a single device (or CPU).

- Introduce ExecutionPlan::Controller class. This is a token used
to allow ExecutionBuilder::startCompute() to iterate over a sequence
of steps provided by ExecutionPlan, obtaining a StepExecutor for each
one.

There should be no change from previous behavior; this is code
reorganization in preparation for supporting execution of partitioned
models. While ExecutionPlan is still only used when the property
debug.nn.partition.test is set to 0, and it still controls the
execution only when that property is set to 2, we do use the new
StepExecutor always.

Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 0, 1, and 2;
no new failures, logcat looks plausible)

Change-Id: I17473210bcc5de40603d439ec49374030d1b8f36
/frameworks/ml/nn/runtime/ExecutionBuilder.h
c4a34a7ac0b6bd0584ccbee0d846472a23bce657 30-Sep-2017 David Gross <dgross@google.com> Merge "Fix "error: private field 'mPlan' is not used"" into oc-mr1-dev
33afb5bc01f88cedab373c7e2fd55580e44cd126 30-Sep-2017 David Gross <dgross@google.com> Fix "error: private field 'mPlan' is not used"

Bug: 63905942
Test: mma (user)
Change-Id: I4f159c43d1db653f733c577508277868fb31ab55
/frameworks/ml/nn/runtime/ExecutionBuilder.h
0798f3e98982252fb9115ec3369b40b78db87747 30-Sep-2017 David Gross <dgross@google.com> Merge "Minor code cleanup." into oc-mr1-dev
e413eef7e5f790acdb8e5d07cacfee9afab6f7be 29-Sep-2017 David Gross <dgross@google.com> Minor code cleanup.

- Add comments on allocateSharedMemory() usage.
- Move some data from ExecutionBuilder to startComputeOnDevice()
- Simplify initialization of ModelArgumentInfo.
- Remove declaration of undefined function ExecutionBuilder::updateDimensionInfo().
- Fix code idiom.
(See https://googleplex-android-review.git.corp.google.com/#/c/platform/frameworks/ml/+/2970560/3/nn/runtime/ExecutionPlan.cpp@283)

Bug: 63905942
Test: ml/nn/runtime/tests (no new failures)

Change-Id: I5fa7096e425ba9b391a1f64c6f0ad625c36515b5
/frameworks/ml/nn/runtime/ExecutionBuilder.h
47522fce0416bfae1cd26f676f251763cd22c172 29-Sep-2017 David Gross <dgross@google.com> Merge "Finish implementing single-partition graphs." into oc-mr1-dev
1f4381539b7e89c42336ee7cd1addb9a4c317b34 28-Sep-2017 David Gross <dgross@google.com> Finish implementing single-partition graphs.

This includes wiring up the partitioning algorithm
to the NN API compilation and execution phases under
debug.nn.partition.test=2: When we have a single-
partition graph, we rely on the device selection and
compilation performed by the partitioning algorithm,
rather than doing it at execution time as we were
before.

Bug: 63905942
Test: mma (user)
mma (userdebug)
ml/nn/runtime/tests (userdebug)
(with debug.nn.partition.test 0, 1, and 2;
no new failures, logcat looks plausible,
compilation happens at the expected times)

Change-Id: I6cd8b53625588d5aca81c87cc8600735e384b8e0
/frameworks/ml/nn/runtime/ExecutionBuilder.h
e127e49e67b53c96eb79f6a9c58f956ad4761227 28-Sep-2017 Jean-Luc Brouillet <jeanluc@google.com> Moved validation to the specific classes.

It's hard to keep track of what's validated when the validations
are found in two files (NeuralNetworks.cpp + one of the other).
Move all the validations except for the initial nullptr check
to the specific file.

Moved some code from Memory.h into the .cpp file.

Adjusted some type size for setInput/Output/*

Adding new validations will be in the future CLs to make
review easier.

Bug: 63905942
Test: Ran local and VTS tests.
Change-Id: I2c448c9b0d3f2e878278f24eafce5720590403f3
/frameworks/ml/nn/runtime/ExecutionBuilder.h
425b2594c76e934dfdbc93209253e3c189571149 14-Sep-2017 David Gross <dgross@google.com> Bring back Event, and wait on Event rather than on Execution.

This is at the recommendation of the API council.

This undoes change I50b8e155d9589210e14920237878a4f295e7755f

Bug: 63905942
Test: nn/runtime/test

Change-Id: I5922f13ae7070f7ec0af035557a9921c88ca9221
/frameworks/ml/nn/runtime/ExecutionBuilder.h
3ced3cfd5b8f22b632c35f24e585c4847383b195 13-Sep-2017 David Gross <dgross@google.com> Rename Request to Execution at the API level.

There's a separate notion of Request at the HIDL level;
this is not renamed.

Bug: 63905942
Test: nn/runtime/tests

Change-Id: I671e3b387d735a9091dc5a2d14afbea6b45f444c
/frameworks/ml/nn/runtime/ExecutionBuilder.h