Welcome to the Adversarial Robustness Toolbox

Adversarial Robustness Toolbox (ART) is a Python library for Machine Learning Security. ART provides tools that enable developers and researchers to evaluate, defend, certify and verify Machine Learning models and applications against the adversarial threats of Evasion, Poisoning, Extraction, and Inference. ART supports all popular machine learning frameworks (TensorFlow, Keras, PyTorch, MXNet, scikit-learn, XGBoost, LightGBM, CatBoost, GPy, etc.), all data types (images, tables, audio, video, etc.) and machine learning tasks (classification, object detection, generation, certification, etc.).

The code of ART is on GitHub and the Wiki contains overviews of implemented attacks, defences and metrics.

The library is under continuous development. Feedback, bug reports and contributions are very welcome!

Supported Machine Learning Libraries

• TensorFlow (v1 and v2) (https://www.tensorflow.org)

• Keras (https://www.keras.io)

• PyTorch (https://www.pytorch.org)

• MXNet (https://mxnet.apache.org)

• Scikit-learn (https://www.scikit-learn.org)

• XGBoost (https://www.xgboost.ai)

• LightGBM (https://lightgbm.readthedocs.io)

• CatBoost (https://www.catboost.ai)

• GPy (https://sheffieldml.github.io/GPy/)

User guide

• Setup
• Examples
• Notebooks

Modules

• art.attacks
  • Base Class Attacks
  • Base Class Evasion Attacks
  • Base Class Poisoning Attacks
  • Base Class Extraction Attacks
  • Base Class Inference Attacks
  • Base Class Reconstruction Attacks
• art.attacks.evasion
  • Adversarial Patch
  • Adversarial Patch - Numpy
  • Adversarial Patch - PyTorch
  • Adversarial Patch - TensorFlowV2
  • Adversarial Texture - PyTorch
  • Auto Attack
  • Auto Projected Gradient Descent (Auto-PGD)
  • Auto Conjugate Gradient (Auto-CG)
  • Boundary Attack / Decision-Based Attack
  • Brendel and Bethge Attack
  • Carlini and Wagner L_0 Attack
  • Carlini and Wagner L_2 Attack
  • Carlini and Wagner L_inf Attack
  • Carlini and Wagner ASR Attack
  • Composite Adversarial Attack - PyTorch
  • Decision Tree Attack
  • DeepFool
  • DPatch
  • RobustDPatch
  • Elastic Net Attack
  • Fast Gradient Method (FGM)
  • Feature Adversaries - Numpy
  • Feature Adversaries - PyTorch
  • Feature Adversaries - TensorFlow
  • Frame Saliency Attack
  • Geometric Decision Based Attack
  • GRAPHITE - Blackbox
  • GRAPHITE - Whitebox - PyTorch
  • High Confidence Low Uncertainty Attack
  • HopSkipJump Attack
  • Imperceptible ASR Attack
  • Imperceptible ASR Attack - PyTorch
  • Basic Iterative Method (BIM)
  • Projected Gradient Descent (PGD)
  • Projected Gradient Descent (PGD) - Numpy
  • Projected Gradient Descent (PGD) - PyTorch
  • Projected Gradient Descent (PGD) - TensorFlowV2
  • LaserAttack
  • LowProFool
  • NewtonFool
  • Malware Gradient Descent - TensorFlow
  • Over The Air Flickering Attack - PyTorch
  • PixelAttack
  • ThresholdAttack
  • Jacobian Saliency Map Attack (JSMA)
  • Shadow Attack
  • ShapeShifter Attack
  • Sign-OPT Attack
  • Simple Black-box Adversarial Attack
  • Spatial Transformations Attack
  • Square Attack
  • Targeted Universal Perturbation Attack
  • Universal Perturbation Attack
  • Virtual Adversarial Method
  • Wasserstein Attack
  • Zeroth-Order Optimization (ZOO) Attack
• art.attacks.extraction
  • Copycat CNN
  • Functionally Equivalent Extraction
  • Knockoff Nets
• art.attacks.inference.attribute_inference
  • Attribute Inference Baseline
  • Attribute Inference Black-Box
  • Attribute Inference Membership
  • Attribute Inference Base Line True Label
  • Attribute Inference White-Box Lifestyle Decision-Tree
  • Attribute Inference White-Box Decision-Tree
• art.attacks.inference.membership_inference
  • Membership Inference Black-Box
  • Membership Inference Black-Box Rule-Based
  • Membership Inference Label-Only - Decision Boundary
  • Membership Inference Label-Only - Gap Attack
  • Shadow Models
• art.attacks.inference.model_inversion
  • Model Inversion MIFace
• art.attacks.inference.reconstruction
  • Database Reconstruction
• art.attacks.poisoning
  • Backdoor Attack DGM ReD
  • Backdoor Attack DGM Trail
  • Adversarial Embedding Attack
  • Backdoor Poisoning Attack
  • Hidden Trigger Backdoor Attack
  • Bullseye Polytope Attack
  • Clean Label Backdoor Attack
  • Feature Collision Attack
  • Gradient Matching Attack
  • Poisoning SVM Attack
  • Sleeper Agent Attack
• art.defences
• art.defences.detector.evasion
  • Base Class
  • Binary Input Detector
  • Binary Activation Detector
  • Subset Scanning Detector
• art.defences.detector.poison
  • Base Class
  • Activation Defence
  • Data Provenance Defense
  • Reject on Negative Impact (RONI) Defense
  • Spectral Signature Defense
• art.defences.postprocessor
  • Base Class Postprocessor
  • Class Labels
  • Gaussian Noise
  • High Confidence
  • Reverse Sigmoid
  • Rounded
• art.defences.preprocessor
  • Base Class Preprocessor
  • CutMix
  • CutMix - PyTorch
  • CutMix - TensorFlowV2
  • Cutout
  • Cutout - PyTorch
  • Cutout - TensorFlowV2
  • Feature Squeezing
  • Gaussian Data Augmentation
  • InverseGAN
  • DefenseGAN
  • JPEG Compression
  • Label Smoothing
  • Mixup
  • Mixup - PyTorch
  • Mixup - TensorFlowV2
  • Mp3 Compression
  • PixelDefend
  • Resample
  • Spatial Smoothing
  • Spatial Smoothing - PyTorch
  • Spatial Smoothing - TensorFlow v2
  • Thermometer Encoding
  • Total Variance Minimization
  • Video Compression
• art.defences.trainer
  • Base Class Trainer
  • Adversarial Training
  • Adversarial Training Madry PGD
  • Adversarial Training Adversarial Weight Perturbation (AWP) - PyTorch
  • Adversarial Training Oracle Aligned Adversarial Training (OAAT) - PyTorch
  • Adversarial Training TRADES - PyTorch
  • Base Class Adversarial Training Fast is Better than Free
  • Adversarial Training Fast is Better than Free - PyTorch
  • Adversarial Training Certified - PyTorch
  • Adversarial Training Certified Interval Bound Propagation - PyTorch
  • DP - InstaHide Training
• art.defences.transformer.evasion
  • Defensive Distillation
• art.defences.transformer.poisoning
  • Neural Cleanse
  • STRIP
• art.estimators
  • Base Class Estimator
  • Mixin Base Class Loss Gradients
  • Mixin Base Class Neural Networks
  • Mixin Base Class Decision Trees
  • Base Class KerasEstimator
  • Base Class MXEstimator
  • Base Class PyTorchEstimator
  • Base Class ScikitlearnEstimator
  • Base Class TensorFlowEstimator
  • Base Class TensorFlowV2Estimator
• art.estimators.certification
• art.estimators.certification.deep_z
  • PytorchDeepZ
  • Zono Dense Layer
  • Zono Bounds
  • Zono Convolution Layer
  • Zono ReLU
• art.estimators.certification.interval
  • PyTorchIBPClassifier
  • Interval Dense Layer
  • Interval Convolution Layer
  • Interval Flatten Layer
  • Interval ReLU
  • Interval Bounds
• art.estimators.certification.randomized_smoothing
  • Mixin Base Class Randomized Smoothing
  • PyTorch Randomized Smoothing Classifier
  • TensorFlow V2 Randomized Smoothing Classifier
• art.estimators.classification
  • Mixin Base Class Classifier
  • Mixin Base Class Class Gradients
  • BlackBox Classifier
  • BlackBox Classifier NeuralNetwork
  • Deep Partition Aggregation Classifier
  • Keras Classifier
  • MXNet Classifier
  • PyTorch Classifier
  • Query-Efficient Black-box Gradient Estimation Classifier
  • TensorFlow Classifier
  • TensorFlow v2 Classifier
  • Ensemble Classifier
  • Scikit-learn Classifier Classifier
  • GPy Gaussian Process Classifier
• art.estimators.classification.scikitlearn
  • Base Class Scikit-learn
  • Scikit-learn DecisionTreeClassifier Classifier
  • Scikit-learn ExtraTreeClassifier Classifier
  • Scikit-learn AdaBoostClassifier Classifier
  • Scikit-learn BaggingClassifier Classifier
  • Scikit-learn ExtraTreesClassifier Classifier
  • Scikit-learn GradientBoostingClassifier Classifier
  • Scikit-learn RandomForestClassifier Classifier
  • Scikit-learn LogisticRegression Classifier
  • Scikit-learn SVC Classifier
• art.estimators.encoding
  • Mixin Base Class Encoder
  • TensorFlow Encoder
• art.estimators.gan
  • TensorFlowV2 GAN
• art.estimators.generation
  • Mixin Base Class Generator
  • TensorFlow Generator
  • TensorFlow 2 Generator
• art.estimators.object_detection
  • Mixin Base Class Object Detector
  • Object Detector PyTorch
  • Object Detector PyTorch Faster-RCNN
  • Object Detector PyTorch YOLO
  • Object Detector TensorFlow Faster-RCNN
• art.estimators.object_tracking
  • Mixin Base Class Object Tracker
  • Object Tracker PyTorch GOTURN
• art.estimators.poison_mitigation
  • Keras Neural Cleanse Classifier
  • Mixin Base Class STRIP
• art.estimators.regression
  • Mixin Base Class Regressor
• art.estimators.regression.scikitlearn
  • Base Class Scikit-learn
  • Scikit-learn Decision Tree Regressor
• art.estimators.speech_recognition
  • Mixin Base Class Speech Recognizer
  • Speech Recognizer Deep Speech - PyTorch
  • Speech Recognizer Espresso - PyTorch
  • Speech Recognizer Lingvo ASR - TensorFlow
• art.experimental.estimators
  • Base Class JaxEstimator
• art.experimental.estimators.classification
  • JAX Classifier
• art.evaluations
• art.metrics
  • Clique Method Robustness Verification
  • Loss Sensitivity
  • Empirical Robustness
  • CLEVER
  • Wasserstein Distance
  • Pointwise Differential Training Privacy
  • SHAPr Membership Privacy Risk
• art.preprocessing
• art.preprocessing.audio
  • L-Filter
  • LFilter - PyTorch
• art.preprocessing.expectation_over_transformation
  • EOT Image Center Crop - PyTorch
  • EOT Image Rotation - TensorFlow V2
  • EOT Image Rotation - PyTorch
  • EOT Brightness - PyTorch
  • EOT Brightness - TensorFlow V2
  • EOT Contrast - PyTorch
  • EOT Contrast - TensorFlow V2
  • EOT Gaussian Noise - PyTorch
  • EOT Gaussian Noise - TensorFlow V2
  • EOT Shot Noise - PyTorch
  • EOT Shot Noise - TensorFlow V2
  • EOT Zoom Blur - PyTorch
  • EOT Zoom Blur - TensorFlow V2
• art.preprocessing.standardisation_mean_std
  • Standardisation Mean and Std
  • Standardisation Mean and Std - PyTorch
  • Standardisation Mean and Std - TensorFlow V2
• art.data_generators
  • Base Class
  • Framework-Specific Data Generators
• art.exceptions
  • EstimatorError
• art.summary_writer
  • Base Class SummaryWriter
  • Summary Writer Default
• art.utils
  • Deprecation Operations
  • Math Operations
  • Label Operations
  • Dataset Operations
• tests.utils
  • Test Base Classes
  • Trained Models for Unittests, MNIST
  • Trained Models for Unittests, Iris
  • Random Number Generators

Indices and Tables

• Index

• Module Index

• Search Page