art.data_generators
¶
Module defining an interface for data generators and providing concrete implementations for the supported frameworks.
Their purpose is to allow for data loading and batching on the fly, as well as dynamic data augmentation.
The generators can be used with the fit_generator function in the Classifier
interface. Users can define
their own generators following the DataGenerator
interface.
Base Class¶
-
class
art.data_generators.
DataGenerator
(size: Optional[int], batch_size: int)¶ Base class for data generators.
-
property
batch_size
¶ - Returns
Return the batch size.
-
abstract
get_batch
() → tuple¶ Provide the next batch for training in the form of a tuple (x, y). The generator should loop over the data indefinitely.
- Returns
A tuple containing a batch of data (x, y).
-
property
iterator
¶ - Returns
Return the framework’s iterable data generator.
-
property
size
¶ - Returns
Return the dataset size.
-
property
Framework-Specific Data Generators¶
-
class
art.data_generators.
KerasDataGenerator
(iterator: Union[keras.utils.Sequence, tf.keras.utils.Sequence, keras.preprocessing.image.ImageDataGenerator, tf.keras.preprocessing.image.ImageDataGenerator, Generator], size: Optional[int], batch_size: int)¶ Wrapper class on top of the Keras-native data generators. These can either be generator functions, keras.utils.Sequence or Keras-specific data generators (keras.preprocessing.image.ImageDataGenerator).
-
get_batch
() → tuple¶ Provide the next batch for training in the form of a tuple (x, y). The generator should loop over the data indefinitely.
- Returns
A tuple containing a batch of data (x, y).
-
-
class
art.data_generators.
MXDataGenerator
(iterator: mxnet.gluon.data.DataLoader, size: int, batch_size: int)¶ Wrapper class on top of the MXNet/Gluon native data loader
mxnet.gluon.data.DataLoader
.-
get_batch
() → tuple¶ Provide the next batch for training in the form of a tuple (x, y). The generator should loop over the data indefinitely.
- Returns
A tuple containing a batch of data (x, y).
-
-
class
art.data_generators.
PyTorchDataGenerator
(iterator: torch.utils.data.DataLoader, size: int, batch_size: int)¶ Wrapper class on top of the PyTorch native data loader
torch.utils.data.DataLoader
.-
get_batch
() → tuple¶ Provide the next batch for training in the form of a tuple (x, y). The generator should loop over the data indefinitely.
- Returns
A tuple containing a batch of data (x, y).
- Return type
tuple
-
-
class
art.data_generators.
TensorFlowDataGenerator
(sess: tf.Session, iterator: tf.data.Iterator, iterator_type: str, iterator_arg: Union[Dict, Tuple, tf.Operation], size: int, batch_size: int)¶ Wrapper class on top of the TensorFlow native iterators
tf.data.Iterator
.-
get_batch
() → tuple¶ Provide the next batch for training in the form of a tuple (x, y). The generator should loop over the data indefinitely.
- Returns
A tuple containing a batch of data (x, y).
- Raises
ValueError – If the iterator has reached the end.
-
-
class
art.data_generators.
TensorFlowV2DataGenerator
(iterator: tf.data.Dataset, size: int, batch_size: int)¶ Wrapper class on top of the TensorFlow v2 native iterators
tf.data.Iterator
.-
get_batch
() → tuple¶ Provide the next batch for training in the form of a tuple (x, y). The generator should loop over the data indefinitely.
- Returns
A tuple containing a batch of data (x, y).
- Raises
ValueError – If the iterator has reached the end.
-