art.estimators.classification

Classifier API for applying all attacks. Use the Classifier wrapper to be able to apply an attack to a preexisting model.

Mixin Base Class Classifier

class art.estimators.classification.ClassifierMixin(**kwargs)

Mixin abstract base class defining functionality for classifiers.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

Mixin Base Class Class Gradients

class art.estimators.classification.ClassGradientsMixin

Mixin abstract base class defining classifiers providing access to class gradients. A classifier of this type can be combined with certain white-box attacks. This mixin abstract base class has to be mixed in with class Classifier.

abstract class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (np.ndarray or pandas.DataFrame) – Samples.

• label – Index of a specific per-class derivative. If an integer is provided, the gradient of that class output is computed for all samples. If multiple values as provided, the first dimension should match the batch size of x, and each value will be used as target for its corresponding sample in x. If None, then gradients for all classes will be computed for each sample.

Returns:

Gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified.

BlackBox Classifier

class art.estimators.classification.BlackBoxClassifier(predict_fn: Callable | Tuple[ndarray, ndarray], input_shape: Tuple[int, ...], nb_classes: int, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), fuzzy_float_compare: bool = False)

Class for black-box classifiers.

__init__(predict_fn: Callable | Tuple[ndarray, ndarray], input_shape: Tuple[int, ...], nb_classes: int, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), fuzzy_float_compare: bool = False)

Create a Classifier instance for a black-box model.

Parameters:

• predict_fn – Function that takes in an np.ndarray of input data and returns the one-hot encoded matrix of predicted classes or tuple of the form (inputs, labels) containing the predicted labels for each input.

• input_shape – Size of input.

• nb_classes (int) – Number of prediction classes.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

• fuzzy_float_compare (bool) – If predict_fn is a tuple mapping inputs to labels, and this is True, looking up inputs in the table will be done using numpy.isclose. Only set to True if really needed, since this severely affects performance.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → ESTIMATOR_TYPE

Clone estimator for refitting.

compute_loss(x: ndarray, y: Any, **kwargs) → ndarray

Compute the loss of the estimator for samples x.

Parameters:

• x (ndarray) – Input samples.

• y – Target values.

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

fit(*args, **kwargs)

Fit the classifier on the training set (x, y).

Parameters:

• x – Training data.

• y – Labels, one-vs-rest encoding.

• kwargs – Dictionary of framework-specific arguments. These should be parameters supported by the fit_generator function in Keras and will be passed to this function as such. Including the number of epochs or the number of steps per epoch as part of this argument will result in as error.

Raises:

NotImplementedException – This method is not supported for black-box classifiers.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

predict(*args, **kwargs)

Perform prediction for a batch of inputs.

Parameters:

• x – Input samples.

• batch_size – Size of batches.

Returns:

Array of predictions of shape (nb_inputs, nb_classes).

property predict_fn: Callable

Return the prediction function.

Returns:

The prediction function.

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework. For Keras, .h5 format is used.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

Raises:

NotImplementedException – This method is not supported for black-box classifiers.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

BlackBox Classifier NeuralNetwork

class art.estimators.classification.BlackBoxClassifierNeuralNetwork(predict_fn: Callable | Tuple[ndarray, ndarray], input_shape: Tuple[int, ...], nb_classes: int, channels_first: bool = True, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0, 1), fuzzy_float_compare: bool = False)

Class for black-box neural network classifiers.

__init__(predict_fn: Callable | Tuple[ndarray, ndarray], input_shape: Tuple[int, ...], nb_classes: int, channels_first: bool = True, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0, 1), fuzzy_float_compare: bool = False)

Create a Classifier instance for a black-box model.

Parameters:

• predict_fn – Function that takes in an np.ndarray of input data and returns the one-hot encoded matrix of predicted classes or tuple of the form (inputs, labels) containing the predicted labels for each input.

• input_shape – Size of input.

• nb_classes (int) – Number of prediction classes.

• channels_first (bool) – Set channels first or last.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

• fuzzy_float_compare (bool) – If predict_fn is a tuple mapping inputs to labels, and this is True, looking up inputs in the table will be done using numpy.isclose. Only set to True if really needed, since this severely affects performance.

property channels_first: bool

Returns:

Boolean to indicate index of the color channels in the sample x.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → ESTIMATOR_TYPE

Clone estimator for refitting.

compute_loss(x: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for samples x.

Parameters:

• x (ndarray) – Input samples.

• y (ndarray) – Target values.

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

fit(*args, **kwargs)

Fit the model of the estimator on the training data x and y.

Parameters:

• x – Samples of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• y (Format as expected by the model) – Target values.

• batch_size – Batch size.

• nb_epochs – Number of training epochs.

fit_generator(generator: DataGenerator, nb_epochs: int = 20, **kwargs) → None

Fit the estimator using a generator yielding training batches. Implementations can provide framework-specific versions of this function to speed-up computation.

Parameters:

• generator – Batch generator providing (x, y) for each epoch.

• nb_epochs (int) – Number of training epochs.

get_activations(x: ndarray, layer: int | str, batch_size: int, framework: bool = False) → ndarray

Return the output of a specific layer for samples x where layer is the index of the layer between 0 and nb_layers - 1 or the name of the layer. The number of layers can be determined by counting the results returned by calling `layer_names.

Return type:

ndarray

Parameters:

• x (ndarray) – Samples

• layer – Index or name of the layer.

• batch_size (int) – Batch size.

• framework (bool) – If true, return the intermediate tensor representation of the activation.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property layer_names: List[str] | None

Return the names of the hidden layers in the model, if applicable.

Returns:

The names of the hidden layers in the model, input and output layers are ignored.

Warning

layer_names tries to infer the internal structure of the model. This feature comes with no guarantees on the correctness of the result. The intended order of the layers tries to match their order in the model, but this is not guaranteed either.

loss(x: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the neural network for samples x.

Parameters:

• x (ndarray) – Samples of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

Returns:

Loss values.

Return type:

Format as expected by the model

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

predict(*args, **kwargs)

Perform prediction for a batch of inputs.

Parameters:

• x – Test set.

• batch_size – Size of batches.

Returns:

Array of predictions of shape (nb_inputs, nb_classes).

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

Deep Partition Aggregation Classifier

class art.estimators.classification.DeepPartitionEnsemble(classifiers: CLASSIFIER_NEURALNETWORK_TYPE | List[CLASSIFIER_NEURALNETWORK_TYPE], hash_function: Callable | None = None, ensemble_size: int = 50, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0))

Implementation of Deep Partition Aggregation Defense. Training data is partitioned into disjoint buckets based on a hash function and a classifier is trained on each bucket.

Paper link: https://arxiv.org/abs/2006.14768

__init__(classifiers: CLASSIFIER_NEURALNETWORK_TYPE | List[CLASSIFIER_NEURALNETWORK_TYPE], hash_function: Callable | None = None, ensemble_size: int = 50, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0)) → None

Parameters:

• classifiers – The base model definition to use for defining the ensemble. If a list, the list must be the same size as the ensemble size.

• hash_function – The function used to partition the training data. If empty, the hash function will use the sum of the input values modulo the ensemble size for partitioning.

• ensemble_size (int) – The number of models in the ensemble.

• channels_first (bool) – Set channels first or last.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier. Not applicable in this classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one. Not applicable in this classifier.

property channels_first: bool

Returns:

Boolean to indicate index of the color channels in the sample x.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, training_mode: bool = False, raw: bool = False, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• label – Index of a specific per-class derivative. If None, then gradients for all classes will be computed.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

• raw (bool) – Return the individual classifier raw outputs (not aggregated).

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified. If raw=True, an additional dimension is added at the beginning of the array, indexing the different classifiers.

property classifier_weights: ndarray

Return the list of classifier weights to assign to their prediction when aggregating results.

Returns:

The list of classifier weights to assign to their prediction when aggregating results.

property classifiers: List[ClassifierNeuralNetwork]

Return the Classifier instances that are ensembled together.

Returns:

Classifier instances that are ensembled together.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → EnsembleClassifier

Clone classifier for refitting.

compute_loss(x: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the neural network for samples x.

Parameters:

• x (ndarray) – Samples of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

fit(*args, **kwargs)

Fit the classifier on the training set (x, y). Each classifier will be trained with the same parameters unless train_dict is provided. If train_dict is provided, the model id’s specified will use the training parameters in train_dict instead.

Parameters:

• x – Training data.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes).

• batch_size – Size of batches.

• nb_epochs – Number of epochs to use for training.

• train_dict – A dictionary of training args if certain models need specialized arguments. The key should be the model’s partition id and this will override any default training parameters including batch_size and nb_epochs.

• kwargs – Dictionary of framework-specific arguments.

fit_generator(generator: DataGenerator, nb_epochs: int = 20, **kwargs) → None

Fit the classifier using the generator that yields batches as specified. This function is not supported for ensembles.

Parameters:

• generator – Batch generator providing (x, y) for each epoch. If the generator can be used for native training in Keras, it will.

• nb_epochs (int) – Number of epochs to use for trainings.

• kwargs – Dictionary of framework-specific argument.

Raises:

NotImplementedException – This method is not supported for ensembles.

get_activations(x: ndarray, layer: int | str, batch_size: int = 128, framework: bool = False) → ndarray

Return the output of the specified layer for input x. layer is specified by layer index (between 0 and nb_layers - 1) or by name. The number of layers can be determined by counting the results returned by calling layer_names. This function is not supported for ensembles.

Return type:

ndarray

Parameters:

• x (ndarray) – Input for computing the activations.

• layer – Layer for computing the activations.

• batch_size (int) – Size of batches.

• framework (bool) – If true, return the intermediate tensor representation of the activation.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

Raises:

NotImplementedException – This method is not supported for ensembles.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property layer_names: List[str]

Return the hidden layers in the model, if applicable. This function is not supported for ensembles.

Returns:

The hidden layers in the model, input and output layers excluded.

Raises:

NotImplementedException – This method is not supported for ensembles.

loss_gradient(x: ndarray, y: ndarray, training_mode: bool = False, raw: bool = False, **kwargs) → ndarray

Compute the gradient of the loss function w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

• raw (bool) – Return the individual classifier raw outputs (not aggregated).

Returns:

Array of gradients of the same shape as x. If raw=True, shape becomes [nb_classifiers, x.shape].

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

predict(*args, **kwargs)

Perform prediction for a batch of inputs. Aggregation will be performed on the prediction from each classifier if max_aggregate is True. Otherwise, the probabilities will be summed instead. For logits output set max_aggregate=True, as logits are not comparable between models and should not be aggregated using a sum.

Parameters:

• x – Input samples.

• batch_size – Size of batches.

• raw – Return the individual classifier raw outputs (not aggregated).

• max_aggregate – Aggregate the predicted classes of each classifier if True. If false, aggregation is done using a sum. If raw is true, this arg is ignored

Returns:

Array of predictions of shape (nb_inputs, nb_classes), or of shape (nb_classifiers, nb_inputs, nb_classes) if raw=True.

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework. This function is not supported for ensembles.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

Raises:

NotImplementedException – This method is not supported for ensembles.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

Keras Classifier

class art.estimators.classification.KerasClassifier(model: keras.models.Model | tf.keras.models.Model, use_logits: bool = False, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), input_layer: int = 0, output_layer: int = 0)

Wrapper class for importing Keras models.

__init__(model: keras.models.Model | tf.keras.models.Model, use_logits: bool = False, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), input_layer: int = 0, output_layer: int = 0) → None

Create a Classifier instance from a Keras model. Assumes the model passed as argument is compiled.

Parameters:

• model – Keras model, neural network or other.

• use_logits (bool) – True if the output of the model are logits; false for probabilities or any other type of outputs. Logits output should be favored when possible to ensure attack efficiency.

• channels_first (bool) – Set channels first or last.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

• input_layer (int) – The index of the layer to consider as input for models with multiple input layers. The layer with this index will be considered for computing gradients. For models with only one input layer this values is not required.

• output_layer (int) – Which layer to consider as the output when the models has multiple output layers. The layer with this index will be considered for computing gradients. For models with only one output layer this values is not required.

property channels_first: bool

Returns:

Boolean to indicate index of the color channels in the sample x.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, training_mode: bool = False, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• label – Index of a specific per-class derivative. If an integer is provided, the gradient of that class output is computed for all samples. If multiple values are provided, the first dimension should match the batch size of x, and each value will be used as target for its corresponding sample in x. If None, then gradients for all classes will be computed for each sample.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → KerasClassifier

Create a copy of the classifier that can be refit from scratch. Will inherit same architecture, optimizer and initialization as cloned model, but without weights.

Returns:

new classifier

compute_loss(x: ndarray, y: ndarray, reduction: str = 'none', **kwargs) → ndarray

Compute the loss of the neural network for samples x.

Parameters:

• x (ndarray) – Samples of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• reduction (str) – Specifies the reduction to apply to the output: ‘none’ | ‘mean’ | ‘sum’. ‘none’: no reduction will be applied ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed.

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

custom_loss_gradient(nn_function, tensors, input_values, name='default')

Returns the gradient of the nn_function with respect to model input

Parameters:

• nn_function (a Keras tensor) – an intermediate tensor representation of the function to differentiate

• tensors (list) – the tensors or variables to differentiate with respect to

• input_values (list) – the inputs to evaluate the gradient

• name (str) – The name of the function. Functions of the same name are cached

Returns:

the gradient of the function w.r.t vars

Return type:

np.ndarray

fit(*args, **kwargs)

Fit the classifier on the training set (x, y).

Parameters:

• x – Training data.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or index labels of shape (nb_samples,).

• batch_size – Size of batches.

• nb_epochs – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. These should be parameters supported by the fit_generator function in Keras and will be passed to this function as such. Including the number of epochs or the number of steps per epoch as part of this argument will result in as error.

fit_generator(generator: DataGenerator, nb_epochs: int = 20, **kwargs) → None

Fit the classifier using the generator that yields batches as specified.

Parameters:

• generator – Batch generator providing (x, y) for each epoch. If the generator can be used for native training in Keras, it will.

• nb_epochs (int) – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. These should be parameters supported by the fit_generator function in Keras and will be passed to this function as such. Including the number of epochs as part of this argument will result in as error.

get_activations(x: ndarray, layer: int | str, batch_size: int = 128, framework: bool = False) → ndarray

Return the output of the specified layer for input x. layer is specified by layer index (between 0 and nb_layers - 1) or by name. The number of layers can be determined by counting the results returned by calling layer_names.

Return type:

ndarray

Parameters:

• x (ndarray) – Input for computing the activations.

• layer – Layer for computing the activations.

• batch_size (int) – Size of batches.

• framework (bool) – If true, return the intermediate tensor representation of the activation.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_layer: int

The index of the layer considered as input for models with multiple input layers. For models with only one input layer the index is 0.

Returns:

The index of the layer considered as input for models with multiple input layers.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property layer_names: List[str] | None

Return the names of the hidden layers in the model, if applicable.

Returns:

The names of the hidden layers in the model, input and output layers are ignored.

Warning

layer_names tries to infer the internal structure of the model. This feature comes with no guarantees on the correctness of the result. The intended order of the layers tries to match their order in the model, but this is not guaranteed either.

loss_gradient(x: ndarray, y: ndarray, training_mode: bool = False, **kwargs) → ndarray

Compute the gradient of the loss function w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of gradients of the same shape as x.

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

property output_layer: int

The index of the layer considered as output for models with multiple output layers. For models with only one output layer the index is 0.

Returns:

The index of the layer considered as output for models with multiple output layers.

predict(*args, **kwargs)

Perform prediction for a batch of inputs.

Parameters:

• x – Input samples.

• batch_size – Size of batches.

• training_mode – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of predictions of shape (nb_inputs, nb_classes).

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework. For Keras, .h5 format is used.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

property use_logits: bool

A boolean representing whether the outputs of the model are logits.

Returns:

a boolean representing whether the outputs of the model are logits.

MXNet Classifier

class art.estimators.classification.MXClassifier(model: mx.gluon.Block, loss: mx.nd.loss | mx.gluon.loss, input_shape: Tuple[int, ...], nb_classes: int, optimizer: mx.gluon.Trainer | None = None, ctx: mx.context.Context | None = None, channels_first: bool = True, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0))

Class for importing MXNet Gluon models.

__init__(model: mx.gluon.Block, loss: mx.nd.loss | mx.gluon.loss, input_shape: Tuple[int, ...], nb_classes: int, optimizer: mx.gluon.Trainer | None = None, ctx: mx.context.Context | None = None, channels_first: bool = True, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0)) → None

Initialize an MXClassifier object. Assumes the model passed as parameter is a Gluon model.

Parameters:

• model – The Gluon model. The output of the model can be logits, probabilities or anything else. Logits output should be preferred where possible to ensure attack efficiency.

• loss – The loss function for which to compute gradients for training.

• input_shape – The shape of one input instance.

• nb_classes (int) – The number of classes of the model.

• optimizer – The optimizer used to train the classifier. This parameter is only required if fitting will be done with method fit.

• ctx – The device on which the model runs (CPU or GPU). If not provided, CPU is assumed.

• channels_first (bool) – Set channels first or last.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

property channels_first: bool

Returns:

Boolean to indicate index of the color channels in the sample x.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, training_mode: bool = False, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• label – Index of a specific per-class derivative. If an integer is provided, the gradient of that class output is computed for all samples. If multiple values as provided, the first dimension should match the batch size of x, and each value will be used as target for its corresponding sample in x. If None, then gradients for all classes will be computed for each sample.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → MXClassifier

Clone classifier for refitting.

compute_loss(x: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the neural network for samples x.

Parameters:

• x (ndarray) – Samples of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

property ctx: mx.context.Context

Return the device on which the model runs.

Returns:

The device on which the model runs (CPU or GPU).

fit(*args, **kwargs)

Fit the classifier on the training set (inputs, outputs).

Parameters:

• x – Training data.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or index labels of shape (nb_samples,).

• batch_size – Size of batches.

• nb_epochs – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. This parameter is not currently supported for MXNet and providing it takes no effect.

fit_generator(generator: DataGenerator, nb_epochs: int = 20, **kwargs) → None

Fit the classifier using the generator that yields batches as specified.

Parameters:

• generator – Batch generator providing (x, y) for each epoch.

• nb_epochs (int) – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. This parameter is not currently supported for MXNet and providing it takes no effect.

get_activations(x: ndarray, layer: int | str, batch_size: int = 128, framework: bool = False) → ndarray

Return the output of the specified layer for input x. layer is specified by layer index (between 0 and nb_layers - 1) or by name. The number of layers can be determined by counting the results returned by calling layer_names.

Return type:

ndarray

Parameters:

• x (ndarray) – Input for computing the activations.

• layer – Layer for computing the activations

• batch_size (int) – Size of batches.

• framework (bool) – If true, return the intermediate tensor representation of the activation.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property layer_names: List[str]

Return the hidden layers in the model, if applicable.

Returns:

The hidden layers in the model, input and output layers excluded.

Warning

layer_names tries to infer the internal structure of the model. This feature comes with no guarantees on the correctness of the result. The intended order of the layers tries to match their order in the model, but this is not guaranteed either.

property loss: mx.nd.loss | mx.gluon.loss

Return the loss function.

Returns:

The loss function.

loss_gradient(x: ndarray, y: ndarray, training_mode: bool = False, **kwargs) → ndarray

Compute the gradient of the loss function w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of gradients of the same shape as x.

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

property optimizer: mx.gluon.Trainer

Return the optimizer used to train the classifier.

Returns:

The optimizer used to train the classifier.

predict(*args, **kwargs)

Perform prediction for a batch of inputs.

Parameters:

• x – Input samples.

• batch_size – Size of batches.

• training_mode – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of predictions of shape (nb_inputs, nb_classes).

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework. For Gluon, only parameters are saved in file with name <filename>.params at the specified path. To load the saved model, the original model code needs to be run before calling load_parameters on the generated Gluon model.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

PyTorch Classifier

class art.estimators.classification.PyTorchClassifier(model: torch.nn.Module, loss: torch.nn.modules.loss._Loss, input_shape: Tuple[int, ...], nb_classes: int, optimizer: torch.optim.Optimizer | None = None, use_amp: bool = False, opt_level: str = 'O1', loss_scale: float | str | None = 'dynamic', channels_first: bool = True, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), device_type: str = 'gpu')

This class implements a classifier with the PyTorch framework.

__init__(model: torch.nn.Module, loss: torch.nn.modules.loss._Loss, input_shape: Tuple[int, ...], nb_classes: int, optimizer: torch.optim.Optimizer | None = None, use_amp: bool = False, opt_level: str = 'O1', loss_scale: float | str | None = 'dynamic', channels_first: bool = True, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), device_type: str = 'gpu') → None

Initialization specifically for the PyTorch-based implementation.

Parameters:

• model – PyTorch model. The output of the model can be logits, probabilities or anything else. Logits output should be preferred where possible to ensure attack efficiency.

• loss – The loss function for which to compute gradients for training. The target label must be raw categorical, i.e. not converted to one-hot encoding.

• input_shape – The shape of one input instance.

• optimizer – The optimizer used to train the classifier.

• use_amp (bool) – Whether to use the automatic mixed precision tool to enable mixed precision training or gradient computation, e.g. with loss gradient computation. When set to True, this option is only triggered if there are GPUs available.

• opt_level (str) – Specify a pure or mixed precision optimization level. Used when use_amp is True. Accepted values are O0, O1, O2, and O3.

• loss_scale – Loss scaling. Used when use_amp is True. If passed as a string, must be a string representing a number, e.g., “1.0”, or the string “dynamic”.

• nb_classes (int) – The number of classes of the model.

• optimizer – The optimizer used to train the classifier.

• channels_first (bool) – Set channels first or last.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

• device_type (str) – Type of device on which the classifier is run, either gpu or cpu.

property channels_first: bool

Returns:

Boolean to indicate index of the color channels in the sample x.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, training_mode: bool = False, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• label – Index of a specific per-class derivative. If an integer is provided, the gradient of that class output is computed for all samples. If multiple values as provided, the first dimension should match the batch size of x, and each value will be used as target for its corresponding sample in x. If None, then gradients for all classes will be computed for each sample.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode. Note on RNN-like models: Backpropagation through RNN modules in eval mode raises RuntimeError due to cudnn issues and require training mode, i.e. RuntimeError: cudnn RNN backward can only be called in training mode. Therefore, if the model is an RNN type we always use training mode but freeze batch-norm and dropout layers if training_mode=False.

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → PyTorchClassifier

Create a copy of the classifier that can be refit from scratch. Will inherit same architecture, same type of optimizer and initialization as the original classifier, but without weights.

Returns:

new estimator

compute_loss(x: ndarray | torch.Tensor, y: ndarray | torch.Tensor, reduction: str = 'none', **kwargs) → ndarray | torch.Tensor

Compute the loss.

Parameters:

• x – Sample input with shape as expected by the model.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• reduction (str) – Specifies the reduction to apply to the output: ‘none’ | ‘mean’ | ‘sum’. ‘none’: no reduction will be applied ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed.

Returns:

Array of losses of the same shape as x.

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

compute_losses(x: ndarray | torch.Tensor, y: ndarray | torch.Tensor, reduction: str = 'none') → Dict[str, ndarray | torch.Tensor]

Compute all loss components.

Parameters:

• x – Sample input with shape as expected by the model.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• reduction (str) – Specifies the reduction to apply to the output: ‘none’ | ‘mean’ | ‘sum’. ‘none’: no reduction will be applied ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed.

Returns:

Dictionary of loss components.

custom_loss_gradient(loss_fn, x: ndarray | torch.Tensor, y: ndarray | torch.Tensor, layer_name, training_mode: bool = False) → ndarray | torch.Tensor

Compute the gradient of the loss function w.r.t. x.

Loss_fn:

Loss function w.r.t to which gradient needs to be calculated.

Parameters:

• x – Sample input with shape as expected by the model(base image).

• y – Sample input with shape as expected by the model(target image).

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.`

• layer_name – Name of the layer from which activation needs to be extracted/activation layer.

Returns:

Array of gradients of the same shape as x.

property device: torch.device

Get current used device.

Returns:

Current used device.

property device_type: str

Return the type of device on which the estimator is run.

Returns:

Type of device on which the estimator is run, either gpu or cpu.

fit(*args, **kwargs)

Fit the classifier on the training set (x, y).

Parameters:

• x – Training data.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or index labels of shape (nb_samples,).

• batch_size – Size of batches.

• nb_epochs – Number of epochs to use for training.

• training_mode – True for model set to training mode and ‘False for model set to evaluation mode.

• drop_last – Set to True to drop the last incomplete batch, if the dataset size is not divisible by the batch size. If False and the size of dataset is not divisible by the batch size, then the last batch will be smaller. (default: False)

• scheduler – Learning rate scheduler to run at the start of every epoch.

• kwargs – Dictionary of framework-specific arguments. This parameter is not currently supported for PyTorch and providing it takes no effect.

fit_generator(generator: DataGenerator, nb_epochs: int = 20, **kwargs) → None

Fit the classifier using the generator that yields batches as specified.

Parameters:

• generator – Batch generator providing (x, y) for each epoch.

• nb_epochs (int) – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. This parameter is not currently supported for PyTorch and providing it takes no effect.

get_activations(x: ndarray | torch.Tensor, layer: int | str | None = None, batch_size: int = 128, framework: bool = False) → ndarray | torch.Tensor

Return the output of the specified layer for input x. layer is specified by layer index (between 0 and nb_layers - 1) or by name. The number of layers can be determined by counting the results returned by calling layer_names.

Parameters:

• x – Input for computing the activations.

• layer – Layer for computing the activations

• batch_size (int) – Size of batches.

• framework (bool) – If true, return the intermediate tensor representation of the activation.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property layer_names: List[str] | None

Return the names of the hidden layers in the model, if applicable.

Returns:

The names of the hidden layers in the model, input and output layers are ignored.

Warning

layer_names tries to infer the internal structure of the model. This feature comes with no guarantees on the correctness of the result. The intended order of the layers tries to match their order in the model, but this is not guaranteed either.

property loss: torch.nn.modules.loss._Loss

Return the loss function.

Returns:

The loss function.

loss_gradient(x: ndarray | torch.Tensor, y: ndarray | torch.Tensor, training_mode: bool = False, **kwargs) → ndarray | torch.Tensor

Compute the gradient of the loss function w.r.t. x.

Parameters:

• x – Sample input with shape as expected by the model.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode. Note on RNN-like models: Backpropagation through RNN modules in eval mode raises RuntimeError due to cudnn issues and require training mode, i.e. RuntimeError: cudnn RNN backward can only be called in training mode. Therefore, if the model is an RNN type we always use training mode but freeze batch-norm and dropout layers if training_mode=False.

Returns:

Array of gradients of the same shape as x.

property loss_scale: float | str

Return the loss scaling value.

Returns:

Loss scaling. Possible values for string: a string representing a number, e.g., “1.0”, or the string “dynamic”.

property model: torch.nn.Module

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

property opt_level: str

Return a string specifying a pure or mixed precision optimization level.

Returns:

A string specifying a pure or mixed precision optimization level. Possible values are O0, O1, O2, and O3.

property optimizer: torch.optim.Optimizer

Return the optimizer.

Returns:

The optimizer.

predict(*args, **kwargs)

Perform prediction for a batch of inputs.

Parameters:

• x – Input samples.

• batch_size – Size of batches.

• training_mode – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of predictions of shape (nb_inputs, nb_classes).

reduce_labels(y: ndarray | torch.Tensor) → ndarray | torch.Tensor

Reduce labels from one-hot encoded to index labels.

reset() → None

Resets the weights of the classifier so that it can be refit from scratch.

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

set_batchnorm(train: bool) → None

Set all batch normalization layers into train or eval mode.

Parameters:

train (bool) – False for evaluation mode.

set_dropout(train: bool) → None

Set all dropout layers into train or eval mode.

Parameters:

train (bool) – False for evaluation mode.

set_multihead_attention(train: bool) → None

Set all multi-head attention layers into train or eval mode.

Parameters:

train (bool) – False for evaluation mode.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

property use_amp: bool

Return a boolean indicating whether to use the automatic mixed precision tool.

Returns:

Whether to use the automatic mixed precision tool.

Query-Efficient Black-box Gradient Estimation Classifier

class art.estimators.classification.QueryEfficientGradientEstimationClassifier(classifier: CLASSIFIER_CLASS_LOSS_GRADIENTS_TYPE, num_basis: int, sigma: float, round_samples: float = 0.0)

Implementation of Query-Efficient Black-box Adversarial Examples. The attack approximates the gradient by maximizing the loss function over samples drawn from random Gaussian noise around the input.

Paper link: https://arxiv.org/abs/1712.07113

__init__(classifier: CLASSIFIER_CLASS_LOSS_GRADIENTS_TYPE, num_basis: int, sigma: float, round_samples: float = 0.0) → None

Parameters:

• classifier – An instance of a classification estimator whose loss_gradient is being approximated.

• num_basis (int) – The number of samples to draw to approximate the gradient.

• sigma (float) – Scaling on the Gaussian noise N(0,1).

• round_samples (float) – The resolution of the input domain to round the data to, e.g., 1.0, or 1/255. Set to 0 to disable.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Input with shape as expected by the classifier’s model.

• label – Index of a specific per-class derivative. If an integer is provided, the gradient of that class output is computed for all samples. If multiple values as provided, the first dimension should match the batch size of x, and each value will be used as target for its corresponding sample in x. If None, then gradients for all classes will be computed for each sample.

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → ESTIMATOR_TYPE

Clone estimator for refitting.

compute_loss(x: ndarray, y: Any, **kwargs) → ndarray

Compute the loss of the estimator for samples x.

Parameters:

• x (ndarray) – Input samples.

• y – Target values.

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

fit(*args, **kwargs)

Fit the classifier using the training data (x, y).

Parameters:

• x – Features in array of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• y – Target values (class labels in classification) in array of shape (nb_samples, nb_classes) in one-hot encoding format.

• kwargs – Dictionary of framework-specific arguments.

get_activations(x: ndarray, layer: int | str, batch_size: int) → ndarray

Return the output of the specified layer for input x. layer is specified by layer index (between 0 and nb_layers - 1) or by name. The number of layers can be determined by counting the results returned by calling layer_names.

Return type:

ndarray

Parameters:

• x (ndarray) – Input for computing the activations.

• layer – Layer for computing the activations.

• batch_size (int) – Size of batches.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

loss_gradient(x: ndarray, y: ndarray, **kwargs) → ndarray

Compute the gradient of the loss function w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• y (ndarray) – Correct labels, one-vs-rest encoding.

Returns:

Array of gradients of the same shape as x.

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

predict(*args, **kwargs)

Perform prediction of the classifier for input x. Rounds results first.

Parameters:

• x – Features in array of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• batch_size – Size of batches.

Returns:

Array of predictions of shape (nb_inputs, nb_classes).

save(filename: str, path: str | None = None) → None

Save a model to file specific to the backend framework.

Parameters:

• filename (str) – Name of the file where to save the model.

• path – Path of the directory where to save the model. If no path is specified, the model will be stored in the default data location of ART at ART_DATA_PATH.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

TensorFlow Classifier

class art.estimators.classification.TensorFlowClassifier(input_ph: tf.Placeholder, output: tf.Tensor, labels_ph: tf.Placeholder | None = None, train: tf.Tensor | None = None, loss: tf.Tensor | None = None, learning: tf.Placeholder | None = None, sess: tf.Session | None = None, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), feed_dict: Dict[Any, Any] | None = None)

This class implements a classifier with the TensorFlow framework.

__init__(input_ph: tf.Placeholder, output: tf.Tensor, labels_ph: tf.Placeholder | None = None, train: tf.Tensor | None = None, loss: tf.Tensor | None = None, learning: tf.Placeholder | None = None, sess: tf.Session | None = None, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), feed_dict: Dict[Any, Any] | None = None) → None

Initialization specific to TensorFlow models implementation.

Parameters:

• input_ph – The input placeholder.

• output – The output layer of the model. This can be logits, probabilities or anything else. Logits output should be preferred where possible to ensure attack efficiency.

• labels_ph – The labels placeholder of the model. This parameter is necessary when training the model and when computing gradients w.r.t. the loss function.

• train – The train tensor for fitting, including an optimizer. Use this parameter only when training the model.

• loss – The loss function for which to compute gradients. This parameter is necessary when training the model and when computing gradients w.r.t. the loss function.

• learning – The placeholder to indicate if the model is training.

• sess – Computation session.

• channels_first (bool) – Set channels first or last.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

• feed_dict – A feed dictionary for the session run evaluating the classifier. This dictionary includes all additionally required placeholders except the placeholders defined in this class.

property channels_first: bool

Returns:

Boolean to indicate index of the color channels in the sample x.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, training_mode: bool = False, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• label – Index of a specific per-class derivative. If an integer is provided, the gradient of that class output is computed for all samples. If multiple values as provided, the first dimension should match the batch size of x, and each value will be used as target for its corresponding sample in x. If None, then gradients for all classes will be computed for each sample.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → TensorFlowClassifier

Clone classifier for refitting.

compute_loss(x: ndarray, y: ndarray, reduction: str = 'none', **kwargs) → ndarray

Compute the loss of the neural network for samples x.

Parameters:

• x (ndarray) – Samples of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• reduction (str) – Specifies the reduction to apply to the output: ‘none’ | ‘mean’ | ‘sum’. ‘none’: no reduction will be applied ‘mean’: Not supported ‘sum’: Not supported

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

property feed_dict: Dict[Any, Any]

Return the feed dictionary for the session run evaluating the classifier.

Returns:

The feed dictionary for the session run evaluating the classifier.

fit(*args, **kwargs)

Fit the classifier on the training set (x, y).

Parameters:

• x – Training data.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or index labels of shape (nb_samples,).

• batch_size – Size of batches.

• nb_epochs – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. This parameter is not currently supported for TensorFlow and providing it takes no effect.

fit_generator(generator: DataGenerator, nb_epochs: int = 20, **kwargs) → None

Fit the classifier using the generator that yields batches as specified.

Parameters:

• generator – Batch generator providing (x, y) for each epoch. If the generator can be used for native training in TensorFlow, it will.

• nb_epochs (int) – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. This parameter is not currently supported for TensorFlow and providing it takes no effect.

get_activations(x: ndarray, layer: int | str, batch_size: int = 128, framework: bool = False) → ndarray

Return the output of the specified layer for input x. layer is specified by layer index (between 0 and nb_layers - 1) or by name. The number of layers can be determined by counting the results returned by calling layer_names.

Return type:

ndarray

Parameters:

• x (ndarray) – Input for computing the activations.

• layer – Layer for computing the activations.

• batch_size (int) – Size of batches.

• framework (bool) – If true, return the intermediate tensor representation of the activation.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_ph: tf.Placeholder

Return the input placeholder.

Returns:

The input placeholder.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property labels_ph: tf.Placeholder

Return the labels placeholder of the model.

Returns:

The labels placeholder of the model.

property layer_names: List[str] | None

Return the names of the hidden layers in the model, if applicable.

Returns:

The names of the hidden layers in the model, input and output layers are ignored.

Warning

layer_names tries to infer the internal structure of the model. This feature comes with no guarantees on the correctness of the result. The intended order of the layers tries to match their order in the model, but this is not guaranteed either.

property learning: tf.Placeholder

Return the placeholder to indicate if the model is training.

Returns:

The placeholder to indicate if the model is training.

property loss: tf.Tensor

Return the loss function.

Returns:

The loss function.

loss_gradient(x: ndarray, y: ndarray, training_mode: bool = False, **kwargs) → ndarray

Compute the gradient of the loss function w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of gradients of the same shape as x.

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

property output: tf.Tensor

Return the output layer of the model.

Returns:

The output layer of the model.

predict(*args, **kwargs)

Perform prediction for a batch of inputs.

Parameters:

• x – Input samples.

• batch_size – Size of batches.

• training_mode – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of predictions of shape (num_inputs, nb_classes).

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework. For TensorFlow, .ckpt is used.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

property sess: tf.python.client.session.Session

Get current TensorFlow session.

Returns:

The current TensorFlow session.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

property train: tf.Tensor

Return the train tensor for fitting.

Returns:

The train tensor for fitting.

TensorFlow v2 Classifier

class art.estimators.classification.TensorFlowV2Classifier(model: Callable, nb_classes: int, input_shape: Tuple[int, ...], loss_object: tf.keras.losses.Loss | None = None, optimizer: tf.keras.optimizers.Optimizer | None = None, train_step: Callable | None = None, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0))

This class implements a classifier with the TensorFlow v2 framework.

__init__(model: Callable, nb_classes: int, input_shape: Tuple[int, ...], loss_object: tf.keras.losses.Loss | None = None, optimizer: tf.keras.optimizers.Optimizer | None = None, train_step: Callable | None = None, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0)) → None

Initialization specific to TensorFlow v2 models.

Parameters:

• model – a python functions or callable class defining the model and providing it prediction as output.

• nb_classes (int) – the number of classes in the classification task.

• input_shape – shape of one input for the classifier, e.g. for MNIST input_shape=(28, 28, 1).

• loss_object – The loss function for which to compute gradients. This parameter is applied for training the model and computing gradients of the loss w.r.t. the input.

• optimizer – The optimizer used to train the classifier.

• train_step – A function that applies a gradient update to the trainable variables with signature train_step(model, images, labels). This will override the default training loop that uses the provided loss_object and optimizer parameters. It is recommended to use the @tf.function decorator, if possible, for efficient training.

• channels_first (bool) – Set channels first or last.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

property channels_first: bool

Returns:

Boolean to indicate index of the color channels in the sample x.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, training_mode: bool = False, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• label – Index of a specific per-class derivative. If an integer is provided, the gradient of that class output is computed for all samples. If multiple values as provided, the first dimension should match the batch size of x, and each value will be used as target for its corresponding sample in x. If None, then gradients for all classes will be computed for each sample.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → TensorFlowV2Classifier

Create a copy of the classifier that can be refit from scratch. Will inherit same architecture, optimizer and initialization as cloned model, but without weights.

Returns:

new estimator

compute_loss(x: ndarray | tf.Tensor, y: ndarray | tf.Tensor, reduction: str = 'none', training_mode: bool = False, **kwargs) → ndarray

Compute the loss.

Return type:

ndarray

Parameters:

• x – Sample input with shape as expected by the model.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• reduction (str) – Specifies the reduction to apply to the output: ‘none’ | ‘mean’ | ‘sum’. ‘none’: no reduction will be applied ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of losses of the same shape as x.

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

compute_losses(x: ndarray | tf.Tensor, y: ndarray | tf.Tensor, reduction: str = 'none') → Dict[str, ndarray | tf.Tensor]

Compute all loss components.

Parameters:

• x – Sample input with shape as expected by the model.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• reduction (str) – Specifies the reduction to apply to the output: ‘none’ | ‘mean’ | ‘sum’. ‘none’: no reduction will be applied ‘mean’: the sum of the output will be divided by the number of elements in the output, ‘sum’: the output will be summed.

Returns:

Dictionary of loss components.

fit(*args, **kwargs)

Fit the classifier on the training set (x, y).

Parameters:

• x – Training data.

• y – Labels, one-hot-encoded of shape (nb_samples, nb_classes) or index labels of shape (nb_samples,).

• batch_size – Size of batches.

• nb_epochs – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. This parameter currently only supports “scheduler” which is an optional function that will be called at the end of every epoch to adjust the learning rate.

fit_generator(generator: DataGenerator, nb_epochs: int = 20, **kwargs) → None

Fit the classifier using the generator that yields batches as specified.

Parameters:

• generator – Batch generator providing (x, y) for each epoch. If the generator can be used for native training in TensorFlow, it will.

• nb_epochs (int) – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments. This parameter currently only supports “scheduler” which is an optional function that will be called at the end of every epoch to adjust the learning rate.

get_activations(x: ndarray, layer: int | str, batch_size: int = 128, framework: bool = False) → ndarray | None

Return the output of the specified layer for input x. layer is specified by layer index (between 0 and nb_layers - 1) or by name. The number of layers can be determined by counting the results returned by calling layer_names.

Parameters:

• x (ndarray) – Input for computing the activations.

• layer – Layer for computing the activations.

• batch_size (int) – Batch size.

• framework (bool) – Return activation as tensor.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property layer_names: List[str] | None

Return the hidden layers in the model, if applicable.

Returns:

The hidden layers in the model, input and output layers excluded.

Warning

layer_names tries to infer the internal structure of the model. This feature comes with no guarantees on the correctness of the result. The intended order of the layers tries to match their order in the model, but this is not guaranteed either.

loss_gradient(x: ndarray | tf.Tensor, y: ndarray | tf.Tensor, training_mode: bool = False, **kwargs) → ndarray | tf.Tensor

Compute the gradient of the loss function w.r.t. x.

Parameters:

• x – Sample input with shape as expected by the model.

• y – Correct labels, one-vs-rest encoding.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of gradients of the same shape as x.

property loss_object: tf.keras.losses.Loss

Return the loss function.

Returns:

The loss function.

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

property optimizer: tf.keras.optimizers.Optimizer

Return the optimizer.

Returns:

The optimizer.

predict(*args, **kwargs)

Perform prediction for a batch of inputs.

Parameters:

• x – Input samples.

• batch_size – Size of batches.

• training_mode – True for model set to training mode and ‘False for model set to evaluation mode.

Returns:

Array of predictions of shape (nb_inputs, nb_classes).

reset() → None

Resets the weights of the classifier so that it can be refit from scratch.

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework. For TensorFlow, .ckpt is used.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

property train_step: Callable

Return the function that applies a gradient update to the trainable variables.

Returns:

The function that applies a gradient update to the trainable variables.

Ensemble Classifier

class art.estimators.classification.EnsembleClassifier(classifiers: List[CLASSIFIER_NEURALNETWORK_TYPE], classifier_weights: list | ndarray | None = None, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0))

Class allowing to aggregate multiple classifiers as an ensemble. The individual classifiers are expected to be trained when the ensemble is created and no training procedures are provided through this class.

__init__(classifiers: List[CLASSIFIER_NEURALNETWORK_TYPE], classifier_weights: list | ndarray | None = None, channels_first: bool = False, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0)) → None

Initialize a EnsembleClassifier object. The data range values and colour channel index have to be consistent for all the classifiers in the ensemble.

Parameters:

• classifiers – List of Classifier instances to be ensembled together.

• classifier_weights – List of weights, one scalar per classifier, to assign to their prediction when aggregating results. If None, all classifiers are assigned the same weight.

• channels_first (bool) – Set channels first or last.

• clip_values – Tuple of the form (min, max) of floats or np.ndarray representing the minimum and maximum values allowed for features. If floats are provided, these will be used as the range of all features. If arrays are provided, each value will be considered the bound for a feature, thus the shape of clip values needs to match the total number of features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier. Not applicable in this classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one. Not applicable in this classifier.

property channels_first: bool

Returns:

Boolean to indicate index of the color channels in the sample x.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, training_mode: bool = False, raw: bool = False, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• label – Index of a specific per-class derivative. If None, then gradients for all classes will be computed.

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

• raw (bool) – Return the individual classifier raw outputs (not aggregated).

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified. If raw=True, an additional dimension is added at the beginning of the array, indexing the different classifiers.

property classifier_weights: ndarray

Return the list of classifier weights to assign to their prediction when aggregating results.

Returns:

The list of classifier weights to assign to their prediction when aggregating results.

property classifiers: List[ClassifierNeuralNetwork]

Return the Classifier instances that are ensembled together.

Returns:

Classifier instances that are ensembled together.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → EnsembleClassifier

Clone classifier for refitting.

compute_loss(x: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the neural network for samples x.

Parameters:

• x (ndarray) – Samples of shape (nb_samples, nb_features) or (nb_samples, nb_pixels_1, nb_pixels_2, nb_channels) or (nb_samples, nb_channels, nb_pixels_1, nb_pixels_2).

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

fit(*args, **kwargs)

Fit the classifier on the training set (x, y). This function is not supported for ensembles.

Parameters:

• x – Training data.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes).

• batch_size – Size of batches.

• nb_epochs – Number of epochs to use for training.

• kwargs – Dictionary of framework-specific arguments.

Raises:

NotImplementedException – This method is not supported for ensembles.

fit_generator(generator: DataGenerator, nb_epochs: int = 20, **kwargs) → None

Fit the classifier using the generator that yields batches as specified. This function is not supported for ensembles.

Parameters:

• generator – Batch generator providing (x, y) for each epoch. If the generator can be used for native training in Keras, it will.

• nb_epochs (int) – Number of epochs to use for trainings.

• kwargs – Dictionary of framework-specific argument.

Raises:

NotImplementedException – This method is not supported for ensembles.

get_activations(x: ndarray, layer: int | str, batch_size: int = 128, framework: bool = False) → ndarray

Return the output of the specified layer for input x. layer is specified by layer index (between 0 and nb_layers - 1) or by name. The number of layers can be determined by counting the results returned by calling layer_names. This function is not supported for ensembles.

Return type:

ndarray

Parameters:

• x (ndarray) – Input for computing the activations.

• layer – Layer for computing the activations.

• batch_size (int) – Size of batches.

• framework (bool) – If true, return the intermediate tensor representation of the activation.

Returns:

The output of layer, where the first dimension is the batch size corresponding to x.

Raises:

NotImplementedException – This method is not supported for ensembles.

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

property layer_names: List[str]

Return the hidden layers in the model, if applicable. This function is not supported for ensembles.

Returns:

The hidden layers in the model, input and output layers excluded.

Raises:

NotImplementedException – This method is not supported for ensembles.

loss_gradient(x: ndarray, y: ndarray, training_mode: bool = False, raw: bool = False, **kwargs) → ndarray

Compute the gradient of the loss function w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

• training_mode (bool) – True for model set to training mode and ‘False for model set to evaluation mode.

• raw (bool) – Return the individual classifier raw outputs (not aggregated).

Returns:

Array of gradients of the same shape as x. If raw=True, shape becomes [nb_classifiers, x.shape].

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

predict(*args, **kwargs)

Perform prediction for a batch of inputs. Predictions from classifiers should only be aggregated if they all have the same type of output (e.g., probabilities). Otherwise, use raw=True to get predictions from all models without aggregation. The same option should be used for logits output, as logits are not comparable between models and should not be aggregated.

Parameters:

• x – Input samples.

• batch_size – Size of batches.

• raw – Return the individual classifier raw outputs (not aggregated).

Returns:

Array of predictions of shape (nb_inputs, nb_classes), or of shape (nb_classifiers, nb_inputs, nb_classes) if raw=True.

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework. This function is not supported for ensembles.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

Raises:

NotImplementedException – This method is not supported for ensembles.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.

Scikit-learn Classifier Classifier

art.estimators.classification.SklearnClassifier(model: sklearn.base.BaseEstimator, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0), use_logits: bool = False) → ScikitlearnClassifier

Create a Classifier instance from a scikit-learn Classifier model. This is a convenience function that instantiates the correct class for the given scikit-learn model.

Parameters:

• model – scikit-learn Classifier model.

• clip_values – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

GPy Gaussian Process Classifier

class art.estimators.classification.GPyGaussianProcessClassifier(model: GPClassification | None = None, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0))

Wrapper class for GPy Gaussian Process classification models.

__init__(model: GPClassification | None = None, clip_values: CLIP_VALUES_TYPE | None = None, preprocessing_defences: Preprocessor | List[Preprocessor] | None = None, postprocessing_defences: Postprocessor | List[Postprocessor] | None = None, preprocessing: PREPROCESSING_TYPE = (0.0, 1.0)) → None

Create a Classifier instance GPY Gaussian Process classification models.

Parameters:

• model – GPY Gaussian Process Classification model.

• clip_values – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

• preprocessing_defences – Preprocessing defence(s) to be applied by the classifier.

• postprocessing_defences – Postprocessing defence(s) to be applied by the classifier.

• preprocessing – Tuple of the form (subtrahend, divisor) of floats or np.ndarray of values to be used for data preprocessing. The first value will be subtracted from the input. The input will then be divided by the second one.

class_gradient(x: ndarray, label: int | List[int] | ndarray | None = None, eps: float = 0.0001, **kwargs) → ndarray

Compute per-class derivatives w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• label – Index of a specific per-class derivative. If an integer is provided, the gradient of that class output is computed for all samples. If multiple values as provided, the first dimension should match the batch size of x, and each value will be used as target for its corresponding sample in x. If None, then gradients for all classes will be computed for each sample.

• eps (float) – Fraction added to the diagonal elements of the input x.

Returns:

Array of gradients of input features w.r.t. each class in the form (batch_size, nb_classes, input_shape) when computing for all classes, otherwise shape becomes (batch_size, 1, input_shape) when label parameter is specified.

property clip_values: CLIP_VALUES_TYPE | None

Return the clip values of the input samples.

Returns:

Clip values (min, max).

clone_for_refitting() → ESTIMATOR_TYPE

Clone estimator for refitting.

compute_loss(x: ndarray, y: Any, **kwargs) → ndarray

Compute the loss of the estimator for samples x.

Parameters:

• x (ndarray) – Input samples.

• y – Target values.

Returns:

Loss values.

Return type:

Format as expected by the model

compute_loss_from_predictions(pred: ndarray, y: ndarray, **kwargs) → ndarray

Compute the loss of the estimator for predictions pred.

Return type:

ndarray

Parameters:

• pred (ndarray) – Model predictions.

• y (ndarray) – Target values.

Returns:

Loss values.

fit(*args, **kwargs)

Fit the classifier on the training set (x, y).

Parameters:

• x – Training data. Not used, as given to model in initialized earlier.

• y – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes).

get_params() → Dict[str, Any]

Get all parameters and their values of this estimator.

Returns:

A dictionary of string parameter names to their value.

property input_shape: Tuple[int, ...]

Return the shape of one input sample.

Returns:

Shape of one input sample.

loss_gradient(x: ndarray, y: ndarray, **kwargs) → ndarray

Compute the gradient of the loss function w.r.t. x.

Return type:

ndarray

Parameters:

• x (ndarray) – Sample input with shape as expected by the model.

• y (ndarray) – Target values (class labels) one-hot-encoded of shape (nb_samples, nb_classes) or indices of shape (nb_samples,).

Returns:

Array of gradients of the same shape as x.

property model

Return the model.

Returns:

The model.

property nb_classes: int

Return the number of output classes.

Returns:

Number of classes in the data.

predict(*args, **kwargs)

Perform prediction for a batch of inputs.

Parameters:

• x – Input samples.

• logits – True if the prediction should be done without squashing function.

Returns:

Array of predictions of shape (nb_inputs, nb_classes).

predict_uncertainty(x: ndarray) → ndarray

Perform uncertainty prediction for a batch of inputs.

Return type:

ndarray

Parameters:

x (ndarray) – Input samples.

Returns:

Array of uncertainty predictions of shape (nb_inputs).

save(filename: str, path: str | None = None) → None

Save a model to file in the format specific to the backend framework.

Parameters:

• filename (str) – Name of the file where to store the model.

• path – Path of the folder where to store the model. If no path is specified, the model will be stored in the default data location of the library ART_DATA_PATH.

set_params(**kwargs) → None

Take a dictionary of parameters and apply checks before setting them as attributes.

Parameters:

kwargs – A dictionary of attributes.