art.defences.preprocessor

Module implementing preprocessing defences against adversarial attacks.

Base Class Preprocessor

class art.defences.preprocessor.Preprocessor

Abstract base class for preprocessing defences.

abstract __call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Perform data preprocessing and return preprocessed data as tuple.

Return type

Tuple

Parameters
  • x (ndarray) – Dataset to be preprocessed.

  • y – Labels to be preprocessed.

Returns

Preprocessed data.

__init__() → None

Create a preprocessing object.

abstract property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

abstract property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

abstract estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

abstract fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

Fit the parameters of the data preprocessor if it has any.

Parameters
  • x (ndarray) – Training set to fit the preprocessor.

  • y – Labels for the training set.

  • kwargs – Other parameters.

property is_fitted

Return the state of the preprocessing object.

Returns

True if the preprocessing model has been fitted (if this applies).

set_params(**kwargs) → None

Take in a dictionary of parameters and apply checks before saving them as attributes.

Feature Squeezing

class art.defences.preprocessor.FeatureSqueezing(clip_values: Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]], bit_depth: int = 8, apply_fit: bool = False, apply_predict: bool = True)

Reduces the sensibility of the features of a sample.

Please keep in mind the limitations of defences. For more information on the limitations of this defence, see https://arxiv.org/abs/1803.09868 . For details on how to evaluate classifier security in general, see https://arxiv.org/abs/1902.06705
__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply feature squeezing to sample x.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to squeeze. x values are expected to be in the data range provided by clip_values.

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Squeezed sample.

__init__(clip_values: Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]], bit_depth: int = 8, apply_fit: bool = False, apply_predict: bool = True) → None

Create an instance of feature squeezing.

Parameters
  • clip_values (Tuple) – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

  • bit_depth (int) – The number of bits per channel for encoding the data.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

Gaussian Data Augmentation

class art.defences.preprocessor.GaussianAugmentation(sigma: float = 1.0, augmentation: bool = True, ratio: float = 1.0, clip_values: Optional[Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]]] = None, apply_fit: bool = True, apply_predict: bool = False)

Add Gaussian noise to a dataset in one of two ways: either add noise to each sample (keeping the size of the original dataset) or perform augmentation by keeping all original samples and adding noisy counterparts. When used as part of a Classifier instance, the defense will be applied automatically only when training if augmentation is true, and only when performing prediction otherwise.

__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Augment the sample (x, y) with Gaussian noise. The result is either an extended dataset containing the original sample, as well as the newly created noisy samples (augmentation=True) or just the noisy counterparts to the original samples.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to augment with shape (batch_size, width, height, depth).

  • y – Labels for the sample. If this argument is provided, it will be augmented with the corresponded original labels of each sample point.

Returns

The augmented dataset and (if provided) corresponding labels.

__init__(sigma: float = 1.0, augmentation: bool = True, ratio: float = 1.0, clip_values: Optional[Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]]] = None, apply_fit: bool = True, apply_predict: bool = False)

Initialize a Gaussian augmentation object.

Parameters
  • sigma (float) – Standard deviation of Gaussian noise to be added.

  • augmentation (bool) – If true, perform dataset augmentation using ratio, otherwise replace samples with noisy counterparts.

  • ratio (float) – Percentage of data augmentation. E.g. for a rate of 1, the size of the dataset will double. If augmentation is false, ratio value is ignored.

  • clip_values – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

InverseGAN

class art.defences.preprocessor.InverseGAN(sess: tf.compat.v1.Session, gan: TensorFlowGenerator, inverse_gan: TensorFlowEncoder, apply_fit: bool = False, apply_predict: bool = False)

Given a latent variable generating a given adversarial sample, either inferred by an inverse GAN or randomly generated, the InverseGAN optimizes that latent variable to project a sample as close as possible to the adversarial sample without the adversarial noise.

__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Applies the InverseGAN defence upon the sample input.

Return type

Tuple

Parameters
  • x (ndarray) – Sample input.

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Defended input.

__init__(sess: tf.compat.v1.Session, gan: TensorFlowGenerator, inverse_gan: TensorFlowEncoder, apply_fit: bool = False, apply_predict: bool = False)

Create an instance of an InverseGAN.

Parameters
  • sess – TF session for computations.

  • gan – GAN model.

  • inverse_gan – Inverse GAN model.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(z_encoding: numpy.ndarray, y: numpy.ndarray) → numpy.ndarray

Compute the gradient of the loss function w.r.t. a z_encoding input within a GAN against a corresponding adversarial sample.

Return type

ndarray

Parameters
  • z_encoding (ndarray) –

  • y (ndarray) – Target values of shape (nb_samples, nb_classes).

Returns

Array of gradients of the same shape as z_encoding.

fit(x, y=None, **kwargs)

No parameters to learn for this method; do nothing.

loss(z: numpy.ndarray, image_adv: numpy.ndarray) → numpy.ndarray

Given a encoding z, computes the loss between the projected sample and the original sample.

Return type

ndarray

Parameters
  • z (ndarray) – encoding z

  • image_adv (ndarray) –

Returns

The loss value

DefenseGAN

class art.defences.preprocessor.DefenseGAN(sess, gan)
__init__(sess, gan)

Create an instance of DefenseGAN.

JPEG Compression

class art.defences.preprocessor.JpegCompression(**kwargs)

Implement the JPEG compression defence approach.

Please keep in mind the limitations of defences. For more information on the limitations of this defence, see https://arxiv.org/abs/1802.00420 . For details on how to evaluate classifier security in general, see https://arxiv.org/abs/1902.06705
__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply JPEG compression to sample x.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to compress with shape of NCHW, NHWC, NCFHW or NFHWC. x values are expected to be in the data range [0, 1] or [0, 255].

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

compressed sample.

__init__(clip_values: Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]], quality: int = 50, channel_index=<art.utils._Deprecated object>, channels_first: bool = False, apply_fit: bool = True, apply_predict: bool = True)

Create an instance of JPEG compression.

Parameters
  • clip_values (Tuple) – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

  • quality (int) – The image quality, on a scale from 1 (worst) to 95 (best). Values above 95 should be avoided.

  • channel_index (int) – Index of the axis in data containing the color channels or features.

  • channels_first (bool) – Set channels first or last.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

Label Smoothing

class art.defences.preprocessor.LabelSmoothing(max_value: float = 0.9, apply_fit: bool = True, apply_predict: bool = False)

Computes a vector of smooth labels from a vector of hard ones. The hard labels have to contain ones for the correct classes and zeros for all the others. The remaining probability mass between max_value and 1 is distributed uniformly between the incorrect classes for each instance.

Please keep in mind the limitations of defences. For details on how to evaluate classifier security in general, see https://arxiv.org/abs/1902.06705 .
__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply label smoothing.

Return type

Tuple

Parameters
  • x (ndarray) – Input data, will not be modified by this method.

  • y – Original vector of label probabilities (one-vs-rest).

Returns

Unmodified input data and the vector of smooth probabilities as correct labels.

Raises

ValueError – If no labels are provided.

__init__(max_value: float = 0.9, apply_fit: bool = True, apply_predict: bool = False) → None

Create an instance of label smoothing.

Parameters
  • max_value (float) – Value to affect to correct label

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

Mp3 Compression

class art.defences.preprocessor.Mp3Compression(**kwargs)

Implement the MP3 compression defense approach.

__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply MP3 compression to sample x.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to compress with shape (batch_size, length, channel). x values are recommended to be of type np.int16.

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Compressed sample.

__init__(sample_rate: int, channel_index=<art.utils._Deprecated object>, channels_first: bool = False, apply_fit: bool = False, apply_predict: bool = True) → None

Create an instance of MP3 compression.

Parameters
  • sample_rate (int) – Specifies the sampling rate of sample.

  • channel_index (int) – Index of the axis containing the audio channels.

  • channels_first (bool) – Set channels first or last.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

PixelDefend

class art.defences.preprocessor.PixelDefend(clip_values: Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]] = 0.0, 1.0, eps: int = 16, pixel_cnn: Optional[ClassifierNeuralNetwork] = None, batch_size: int = 128, apply_fit: bool = False, apply_predict: bool = True)

Implement the pixel defence approach. Defense based on PixelCNN that projects samples back to the data manifold.

Please keep in mind the limitations of defences. For more information on the limitations of this defence, see https://arxiv.org/abs/1802.00420 . For details on how to evaluate classifier security in general, see https://arxiv.org/abs/1902.06705
__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply pixel defence to sample x.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to defense with shape (batch_size, width, height, depth). x values are expected to be in the data range [0, 1].

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Purified sample.

__init__(clip_values: Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]] = 0.0, 1.0, eps: int = 16, pixel_cnn: Optional[ClassifierNeuralNetwork] = None, batch_size: int = 128, apply_fit: bool = False, apply_predict: bool = True) → None

Create an instance of pixel defence.

Parameters
  • clip_values (Tuple) – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

  • eps (int) – Defense parameter 0-255.

  • pixel_cnn – Pre-trained PixelCNN model.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

Resample

class art.defences.preprocessor.Resample(**kwargs)

Implement the resampling defense approach.

Resampling implicitly consists of a step that applies a low-pass filter. The underlying filter in this implementation is a Windowed Sinc Interpolation function.

__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Resample x to a new sampling rate.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to resample of shape (batch_size, length, channel) or (batch_size, channel, length).

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Resampled audio sample.

__init__(sr_original: int, sr_new: int, channel_index=<art.utils._Deprecated object>, channels_first: bool = False, apply_fit: bool = False, apply_predict: bool = True)

Create an instance of the resample preprocessor.

Parameters
  • sr_original (int) – Original sampling rate of sample.

  • sr_new (int) – New sampling rate of sample.

  • channel_index (int) – Index of the axis containing the audio channels.

  • channels_first (bool) – Set channels first or last.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x, grad)

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Parameters
  • x – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

Spatial Smoothing

class art.defences.preprocessor.SpatialSmoothing(**kwargs)

Implement the local spatial smoothing defence approach.

Please keep in mind the limitations of defences. For more information on the limitations of this defence, see https://arxiv.org/abs/1803.09868 . For details on how to evaluate classifier security in general, see https://arxiv.org/abs/1902.06705
__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply local spatial smoothing to sample x.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to smooth with shape (batch_size, width, height, depth).

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Smoothed sample.

__init__(window_size: int = 3, channel_index=<art.utils._Deprecated object>, channels_first: bool = False, clip_values: Optional[Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]]] = None, apply_fit: bool = False, apply_predict: bool = True) → None

Create an instance of local spatial smoothing.

Parameters
  • channel_index (int) – Index of the axis in data containing the color channels or features.

  • channels_first (bool) – Set channels first or last.

  • window_size (int) – The size of the sliding window.

  • clip_values – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

Thermometer Encoding

class art.defences.preprocessor.ThermometerEncoding(**kwargs)

Implement the thermometer encoding defence approach.

Please keep in mind the limitations of defences. For more information on the limitations of this defence, see https://arxiv.org/abs/1802.00420 . For details on how to evaluate classifier security in general, see https://arxiv.org/abs/1902.06705
__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply thermometer encoding to sample x. The new axis with the encoding is added as last dimension.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to encode with shape (batch_size, width, height, depth).

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Encoded sample with shape (batch_size, width, height, depth x num_space).

__init__(clip_values: Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]], num_space: int = 10, channel_index=<art.utils._Deprecated object>, channels_first: bool = False, apply_fit: bool = True, apply_predict: bool = True) → None

Create an instance of thermometer encoding.

Parameters
  • clip_values (Tuple) – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

  • num_space (int) – Number of evenly spaced levels within the interval of minimum and maximum clip values.

  • channel_index (int) – Index of the axis in data containing the color channels or features.

  • channels_first (bool) – Set channels first or last.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. For thermometer encoding, the gradient estimate is the one used in https://arxiv.org/abs/1802.00420, where the thermometer encoding is replaced with a differentiable approximation: g(x_{i,j,c})_k = min(max(x_{i,j,c} - k / self.num_space, 0), 1).

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

Total Variance Minimization

class art.defences.preprocessor.TotalVarMin(prob: float = 0.3, norm: int = 2, lamb: float = 0.5, solver: str = 'L-BFGS-B', max_iter: int = 10, clip_values: Optional[Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]]] = None, apply_fit: bool = False, apply_predict: bool = True)

Implement the total variance minimization defence approach.

Please keep in mind the limitations of defences. For more information on the limitations of this defence, see https://arxiv.org/abs/1802.00420 . For details on how to evaluate classifier security in general, see https://arxiv.org/abs/1902.06705
__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply total variance minimization to sample x.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to compress with shape (batch_size, width, height, depth).

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Similar samples.

__init__(prob: float = 0.3, norm: int = 2, lamb: float = 0.5, solver: str = 'L-BFGS-B', max_iter: int = 10, clip_values: Optional[Tuple[Union[int, float, numpy.ndarray], Union[int, float, numpy.ndarray]]] = None, apply_fit: bool = False, apply_predict: bool = True)

Create an instance of total variance minimization.

Parameters
  • prob (float) – Probability of the Bernoulli distribution.

  • norm (int) – The norm (positive integer).

  • lamb (float) – The lambda parameter in the objective function.

  • solver (str) – Current support: L-BFGS-B, CG, Newton-CG.

  • max_iter (int) – Maximum number of iterations when performing optimization.

  • clip_values – Tuple of the form (min, max) representing the minimum and maximum values allowed for features.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.

Video Compression

class art.defences.preprocessor.VideoCompression(*, video_format: str, constant_rate_factor: int = 28, channels_first: bool = False, apply_fit: bool = False, apply_predict: bool = True)

Implement FFmpeg wrapper for video compression defence based on H.264/MPEG-4 AVC.

Video compression uses H.264 video encoding. The video quality is controlled with the constant rate factor parameter. More information on the constant rate factor: https://trac.ffmpeg.org/wiki/Encode/H.264.

__call__(x: numpy.ndarray, y: Optional[numpy.ndarray] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

Apply video compression to sample x.

Return type

Tuple

Parameters
  • x (ndarray) – Sample to compress of shape NCFHW or NFHWC. x values are expected to be in the data range [0, 255].

  • y – Labels of the sample x. This function does not affect them in any way.

Returns

Compressed sample.

__init__(*, video_format: str, constant_rate_factor: int = 28, channels_first: bool = False, apply_fit: bool = False, apply_predict: bool = True)

Create an instance of VideoCompression.

Parameters
  • video_format (str) – Specify one of supported video file extensions, e.g. avi, mp4 or mkv.

  • constant_rate_factor (int) – Specifiy constant rate factor (range 0 to 51, where 0 is lossless).

  • channels_first (bool) – Set channels first or last.

  • apply_fit (bool) – True if applied during fitting/training.

  • apply_predict (bool) – True if applied during predicting.

property apply_fit

Property of the defence indicating if it should be applied at training time.

Returns

True if the defence should be applied when fitting a model, False otherwise.

property apply_predict

Property of the defence indicating if it should be applied at test time.

Returns

True if the defence should be applied at prediction time, False otherwise.

estimate_gradient(x: numpy.ndarray, grad: numpy.ndarray) → numpy.ndarray

Provide an estimate of the gradients of the defence for the backward pass. If the defence is not differentiable, this is an estimate of the gradient, most often replacing the computation performed by the defence with the identity function.

Return type

ndarray

Parameters
  • x (ndarray) – Input data for which the gradient is estimated. First dimension is the batch size.

  • grad (ndarray) – Gradient value so far.

Returns

The gradient (estimate) of the defence.

fit(x: numpy.ndarray, y: Optional[numpy.ndarray] = None, **kwargs) → None

No parameters to learn for this method; do nothing.