Randomized Smoothing Variations Implementation

[f4str]

Description

Implement the following variations of randomized smoothing:

• MACER for PyTorch
• MACER for TensorFlow v2
• SmoothAdv for PyTorch
• SmoothAdv for TensorFlow 2
• SmoothMix for PyTorch

Additionally implement the following changes

• Rewrite the randomized smoothing test cases from unittest to pytest
• Add support to schedulers for all TensorFlow v2 classifiers (following the implementation from TensorFlowV2DeRandomizedSmoothing)
• Add the verbose parameter to all randomized smoothing classifiers to hide/show the progress bar

This PR is continued and consolidated from

• PR Addition of SmoothMix Training in the Randomized Smoothing Module #1668
• PR SmoothMix Addition #1705
• PR Addition of SmoothAdversarial and MACER Implementation in the Randomized Smoothing Module #1968
• PR SmoothMix PyTorch Implementation #1988

Links to Paper and Authors' code repository:

• MACER: https://arxiv.org/abs/2001.02378 (https://github.com/RuntianZ/macer)
• SmoothAdv: https://arxiv.org/abs/1906.04584 (https://github.com/Hadisalman/smoothing-adversarial)
• SmoothMix: https://arxiv.org/abs/2111.09277 (https://github.com/jh-jeong/smoothmix)

Type of change

Please check all relevant options.

• Improvement (non-breaking)
• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

• Updated unit tests for Randomized Smoothing
• New unit tests for MACER
• New unit tests for SmoothAdv
• New unit tests for SmoothMix

Test Configuration:

• OS
• Python version
• ART version or commit number
• TensorFlow / Keras / PyTorch / MXNet version

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[codecov-commenter]

Codecov Report

Merging #2218 (63612d2) into dev_1.16.0 (82f8fa2) will decrease coverage by 4.25%.
The diff coverage is 95.02%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@              Coverage Diff               @@
##           dev_1.16.0    #2218      +/-   ##
==============================================
- Coverage       85.61%   81.36%   -4.25%     
==============================================
  Files             308      313       +5     
  Lines           27448    27782     +334     
  Branches         5044     5082      +38     
==============================================
- Hits            23499    22606     -893     
- Misses           2669     3932    +1263     
+ Partials         1280     1244      -36     

Files Changed	Coverage Δ
art/estimators/classification/tensorflow.py	82.72% <50.00%> (-0.61%)	⬇️
...s/certification/randomized_smoothing/tensorflow.py	87.09% <60.00%> (-2.74%)	⬇️
...ification/randomized_smoothing/macer/tensorflow.py	91.89% <91.89%> (ø)
...tion/randomized_smoothing/smooth_adv/tensorflow.py	94.11% <94.11%> (ø)
...ication/randomized_smoothing/smooth_mix/pytorch.py	97.22% <97.22%> (ø)
...ication/randomized_smoothing/smooth_adv/pytorch.py	98.36% <98.36%> (ø)
...ertification/randomized_smoothing/macer/pytorch.py	98.38% <98.38%> (ø)
...ors/certification/randomized_smoothing/__init__.py	100.00% <100.00%> (ø)
...tors/certification/randomized_smoothing/pytorch.py	85.54% <100.00%> (ø)
...ation/randomized_smoothing/randomized_smoothing.py	97.36% <100.00%> (+0.03%)	⬆️

... and 21 files with indirect coverage changes

[beat-buesser]

Hi @f4str Thank you very much for your pull request! The code looks great! I have a dded a few suggestions for minor improvements, what do you think?

[beat-buesser]

Is this type ignore required? The None should be covered by the Optional type.

[f4str]

The type: ignore is not required. The original randomized_smoothing/pytorch.py had this so I assumed it was needed. It can be removed from all variations including the original.

[beat-buesser]

Suggested change

	optimizer: Optional["torch.optim.Optimizer"] = None, # type: ignore
	optimizer: Optional["torch.optim.Optimizer"] = None,

[beat-buesser]

Suggested change

	optimizer: Optional["torch.optim.Optimizer"] = None, # type: ignore
	optimizer: Optional["torch.optim.Optimizer"] = None,

[beat-buesser]

Let's include their MIT License text (https://github.com/jh-jeong/smoothmix/blob/main/LICENSE) at the top of our new file after ART's MIT license text.

Suggested change

	Code modified from https://github.com/jh-jeong/smoothmix/code/train.py
	Code modified from https://github.com/jh-jeong/smoothmix/blob/main/code/train.py

[f4str]

Hi @beat-buesser thank you for the review. I've addressed your comments and made the corresponding changes. Feel free to let me know if there's any other issues.

[beat-buesser]

Hi @f4str Thank you very much for extending the randomised smoothing capabilities in ART! The changes look good to me.