| a805116366eddcaa8eb6a602398f8efae076e0b5 |
|
15-Feb-2018 |
Derek Murray <mrry@google.com> |
[tf.data] Return OK and set `*end_of_sequence = true` when repeating an empty dataset.
Returning an error status could lead to situations (like
`empty_ds.repeat(None).interleave(...)`) where the wrong exception was raised.
This change ensures that the proper `OutOfRangeError` is raised in the user
program.
PiperOrigin-RevId: 185900119
/external/tensorflow/tensorflow/core/kernels/data/shuffle_dataset_op.cc
|
| 868627322299a373dae8347d7c4ba9eec5bb6f25 |
|
19-Jan-2018 |
Saurabh Saxena <srbs@google.com> |
Handle case when buffer_size is larger than total dataset size in shuffle dataset op.
Currently ShuffleDatasetOpBase::GetNext throws a "Attempted to repeat an empty dataset infinitely." if the number of outputs produced from the dataset are < buffer_size.
Usually the buffer size is set to <= total dataset size so we never run into this.
PiperOrigin-RevId: 182448427
/external/tensorflow/tensorflow/core/kernels/data/shuffle_dataset_op.cc
|
| dda43c93e0b1cbee0b3215a2ed3fa21afd87e702 |
|
11-Jan-2018 |
Derek Murray <mrry@google.com> |
[tf.data] Change the IteratorBase::RestoreInternal() method to take IteratorContext*.
The IteratorContext type contains all of the state needed to restore
an iterator and it is easier to construct, so this change will make it
possible to control the environment of the restoration more easily
(e.g. when using function overlays on a shared runtime).
PiperOrigin-RevId: 181662977
/external/tensorflow/tensorflow/core/kernels/data/shuffle_dataset_op.cc
|
| a4301f0baa173a347caa34c606c19fd139b055e0 |
|
03-Jan-2018 |
Derek Murray <mrry@google.com> |
[tf.data] Improve error message in `tf.data.FixedLengthRecordDataset`.
The iterator now eagerly returns an error message when the size of an input file
(minus header and footer) is not a multiple of the fixed-length record.
PiperOrigin-RevId: 180612356
/external/tensorflow/tensorflow/core/kernels/data/shuffle_dataset_op.cc
|
| a386aff3dd7383d576bfde3a83b00ca79698d398 |
|
18-Dec-2017 |
Jiri Simsa <jsimsa@google.com> |
Implementing a fused version of shuffle and repeat. The logic is incorporated into the existing `tf.data.Dataset.shuffle_and_repeat` transformation.
Fusing of shuffle and repeat is expected to benefit the performance of pipelines with deep shuffle buffers and/or short epochs as it removes the per epoch initialization of the shuffle buffer.
PiperOrigin-RevId: 179431675
/external/tensorflow/tensorflow/core/kernels/data/shuffle_dataset_op.cc
|
| a5b2a0c9a3335d10c4dd3dfdff96149f74a4d120 |
|
15-Dec-2017 |
Jiri Simsa <jsimsa@google.com> |
Moving tf.data kernels to their own package.
PiperOrigin-RevId: 179112798
/external/tensorflow/tensorflow/core/kernels/data/shuffle_dataset_op.cc
|