Add PRVAccountant #493

tcbegley · 2022-09-01T13:30:19Z

This PR implements a new accountant PRVAccountant based on the paper Numerical Composition of Differential Privacy.

Code inspired heavily by the code that accompanied the paper: https://github.com/microsoft/prv_accountant

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Docs change / refactoring / dependency upgrade

Motivation and Context / Related issue

How Has This Been Tested (if it applies)

I have tested these changes with the following scripts, but would welcome suggestions on how to test further or write unit tests to cover these changes:

validate_gaussian.py recreates this notebook, which checks that we can recover upper and lower bounds on the privacy curve of a Gaussian mechanism correctly.
prv_accountant_cifar10.py runs this tutorial from the Opacus docs with the PRVAccountant instead of RDPAccountant.

Checklist

I have not yet written docstrings or tests for these changes both as it was slightly unclear to me how best to proceed, but also because I would like to validate the approach taken in this initial implementation before polishing.

The documentation is up-to-date with the changes I made.
I have read the CONTRIBUTING document and completed the CLA (see CONTRIBUTING).
All tests passed, and additional code has been covered with new tests.

facebook-github-bot · 2022-09-01T13:34:20Z

@tcbegley has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-09-01T13:54:17Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-09-01T13:54:28Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

romovpa · 2022-09-13T14:04:39Z

@tcbegley Thank you for contributing to Opacus! Great job!

You asked me a few questions in the chat, here is the top of my mind:

Documentation
- Mandatory: docstring for PRVAccountant with the reference to the paper, short description of benefits/limitations of this accountant, anything that the user should know before using this accountant
- Optional: put a concise engineer-faced description of the accounting method that allows to quickly understand how it works without having much context and going into deep details (which can be clarified by reading the paper); I think you may put some of your notes in the docstring
- Ideal: use this accountant in one of the tutorials (or create a new one); this is another big task though.
Testing
- We have accountant_test.py that checks for regressions, please add PRVAccountant there.
- However, the unit tests does not validate the correctness of the theory and the implementation. One known way to do the validation is to run membership inference attacks. We have privacy_lint for this, the new API that allows to easily do that is on the way. If anyone interested, validating accountants and mechanisms with the MI attacks is another good tasks / mini-project. cc @alexandresablayrolles

facebook-github-bot · 2022-09-14T08:05:38Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-09-14T08:05:46Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-09-14T08:06:35Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-09-14T08:06:43Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

tcbegley · 2022-09-14T08:49:29Z

opacus/accountants/analysis/prv/domain.py

+    total_compositions = sum(max_self_compositions)
+
+    rdp_accountant = RDPAccountant()
+    for prv, max_self_composition in zip(prvs, max_self_compositions):
+        rdp_accountant.history.append(
+            (prv.noise_multiplier, prv.sample_rate, max_self_composition)
+        )
+
+    L_max = rdp_accountant.get_epsilon(delta_error / 4)
+
+    for prv, max_self_composition in zip(prvs, max_self_compositions):
+        rdp_accountant = RDPAccountant()
+        rdp_accountant.history = [(prv.noise_multiplier, prv.sample_rate, 1)]
+        L_max = max(
+            L_max,
+            rdp_accountant.get_epsilon(delta=delta_error / (8 * total_compositions)),
+        )
+
+    # FIXME: this implementation is adapted from the code accompanying the paper, but
+    # disagrees subtly with the theory from remark 5.6. It's not immediately clear this
+    # gives the right guarantees in all cases, though it's fine for eps_error < 1 and
+    # hence generic cases.
+    # cf. https://github.com/microsoft/prv_accountant/discussions/34
+    return max(L_max, eps_error) + 3


I'm in two minds about whether to keep this as it is which follows the implementation from microsoft/prv_accountant or to follow the paper more precisely. It's not immediately clear to me that this implementation always satisfies the conditions required in the paper without making an assumption like eps_error < 1.

The changes required are minor and would look something like the following

- L_max = rdp_accountant.get_epsilon(delta_error / 4) + L_max = rdp_accountant.get_epsilon(delta_error / 4) + eps_error

- return max(L_max, eps_error) + 3 + return L_max + 2

facebook-github-bot · 2022-09-14T09:13:34Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-09-14T09:13:42Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-09-14T09:31:19Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-09-14T09:31:26Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-11-03T13:52:10Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-03T13:52:15Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Code inspired heavily by https://github.com/microsoft/prv_accountant

facebook-github-bot · 2022-11-03T14:36:06Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-03T14:36:11Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-11-03T16:18:34Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-03T16:18:40Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-11-04T16:51:03Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-04T16:51:10Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…ountant default in engine

facebook-github-bot · 2022-11-07T10:39:43Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-07T10:39:51Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-11-07T13:20:50Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-07T13:20:59Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…ng examples

facebook-github-bot · 2022-11-07T14:10:15Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-07T14:10:23Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-11-07T15:27:30Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-07T15:27:37Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-11-07T18:06:01Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-07T18:06:10Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-11-07T18:51:15Z

@tcbegley has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-11-07T18:51:22Z

@facebook-github-bot has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 1, 2022

romovpa linked an issue Sep 13, 2022 that may be closed by this pull request

[New accountant] Numerical composition #378

Open

tcbegley marked this pull request as ready for review September 13, 2022 13:48

romovpa self-requested a review September 13, 2022 13:50

tcbegley commented Sep 14, 2022

View reviewed changes

tcbegley and others added 8 commits November 3, 2022 15:35

Initial implementation of PRV Accountant

a53c353

Code inspired heavily by https://github.com/microsoft/prv_accountant

Fix linting issues

0dd2788

Add docstrings and warning

a8ee88a

Add PRVAcountant regression tests

2663fb0

Remove spurious import added by VSCode

7a1f42d

Sort imports

9456bb8

Add PRVAccountant test

334b5a8

Fix values now that accountant's defualts have changed

bb2f182

alexandresablayrolles force-pushed the prv-accountant branch from 4d502a4 to bb2f182 Compare November 3, 2022 14:36

Remove warning

35e01af

Fix lint and assert_allclose

d011f40

Disable coveralls. Make prv accountant test not nightly. Make prv acc…

e92938d

…ountant default in engine

Couple of fixes

d569416

Remove long tests for PRV (keeping the ones that run on time). Updati…

9c67382

…ng examples

Change accountant

b7a4b93

Undo deterministic for test

4632c73

Removing deterministic for dp_layers/dp_rnn_test.py

4fbc963

facebook-github-bot closed this in c89a3bf Nov 8, 2022

tcbegley deleted the prv-accountant branch November 8, 2022 14:37

Add PRVAccountant #493

Add PRVAccountant #493

Uh oh!

Conversation

tcbegley commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Types of changes

Motivation and Context / Related issue

How Has This Been Tested (if it applies)

Checklist

Uh oh!

facebook-github-bot commented Sep 1, 2022

Uh oh!

facebook-github-bot commented Sep 1, 2022

Uh oh!

facebook-github-bot commented Sep 1, 2022

Uh oh!

romovpa commented Sep 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Sep 14, 2022

Uh oh!

facebook-github-bot commented Sep 14, 2022

Uh oh!

facebook-github-bot commented Sep 14, 2022

Uh oh!

facebook-github-bot commented Sep 14, 2022

Uh oh!

tcbegley Sep 14, 2022

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Sep 14, 2022

Uh oh!

facebook-github-bot commented Sep 14, 2022

Uh oh!

facebook-github-bot commented Sep 14, 2022

Uh oh!

facebook-github-bot commented Sep 14, 2022

Uh oh!

facebook-github-bot commented Nov 3, 2022

Uh oh!

facebook-github-bot commented Nov 3, 2022

Uh oh!

facebook-github-bot commented Nov 3, 2022

Uh oh!

facebook-github-bot commented Nov 3, 2022

Uh oh!

facebook-github-bot commented Nov 3, 2022

Uh oh!

facebook-github-bot commented Nov 3, 2022

Uh oh!

facebook-github-bot commented Nov 4, 2022

Uh oh!

facebook-github-bot commented Nov 4, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

facebook-github-bot commented Nov 7, 2022

Uh oh!

Reviewers

tcbegley commented Sep 1, 2022 •

edited

Loading

romovpa commented Sep 13, 2022 •

edited

Loading