Make signatures deterministic by default?

[jku]

#128 seems to be an example of the flaw in current RepositorySimulator signing key & signature management:

• The design is optimized for minimal test code: this leads to signing metadata (apart from root) at request time
• The keys are created (in multiple places) with CryptoSigner.generate_ecdsa() because ecdsa keys are fast to generate: this way each test simply generates whatever keys it needs

Unfortunately ecdsa signatures are not deterministic so the combination of these two features can easily lead to issues that are difficult to debug.

Obvious options to fix this include:

1. start explicitly signing metadata -- I would still like to avoid this if possible
2. switch to a deterministic signature by default

I think I lean to option 2, we should at least try this with another keytype:

• ed25519 is maybe not a great option, I'm not sure if every client has support for that. RSA is likely the right default
• the issue with RSA is the key generation time: in the long term we should not generate keys in every test
  • as the initial implementation we can try generating keys in tests like now, if it's not horribly slow
  • we could also include a list of ~20 hard coded keys in sources and have RepositorySimulator take keys from the list (keys are also generated in test_updater_key_rotations.py and test_keys.py: these should likely use the same source for key material)

[jku]

1. start explicitly signing metadata

the upside of this is that we could more easily test e.g. #134

[AdamKorcz]

I would still like to avoid this if possible

Is this because of UX? - that it is simpler to write tests?

[jku]

Yeah: I was expecting most tests to modify targets metadata (which is currently very easy), but I'm beginning to think the benefits may not be greater than the disadvantages... The fact that modifying root is different from other roles is already annoying (considering that a lot of tests want to modify root).

I'd be happy to see a refactoring that does explicit signing in a clean way, but I have a feeling that's a large change -- at the very least it will touch every single test, and the repository http API will change slightly (as currently we return 404 on "old" version requests but if we have the old versions stored we would logically return them when asked). Would be nice to get the hashes/length length in metafiles issue solved first (as that's currently actually broken and means our test results are partly incorrect).

We could do both options: first the second option (hard code RSA keys), and then look at a explicit signing refactoring .

Opinions @AdamKorcz ?

[jku]

hard code RSA keys

alternative that has less hard coded key material would be to generate 20 signers once in conftest.py and pass a copy of the list to all RepositorySimulators when they are created

[jku]

Just writing down some notes on the "make RepositorySimulator sign explicitly" idea

• keep mds dictionary for storing the "current metadata working set"
• remove signed_roots, add signed_mds: dict[str, list[bytes]] where key is rolename
• remove publish_root() and bump_root_by_one(), add something like publish(role: str, bump_version: bool = True) that bumps version if requested, makes a copy of roles current metadata from mds, signs it, adds bytes to signed_mds
• fetch_metadata() now looks up data only from signed_mds (and can return old versions)
• There's plenty of potential test changes, every test needs to be reviewed:
  • any test that modifies any targets role likely needs to publish(role) at least once
  • same for any tests that modify timestamp or snapshot without calling update_*() afterwards
  • old metadata versions are now available from repository (this is more realistic so sounds good)
  • anything else?

I agree it seems like a good move but seems like quite a bit of work and a source of new bugs: this is why I suggested changing the default keytype so that we would have less bugs in code to begin with

[jku]

Quick test with rsa keys shows some issue with go-tuf... see https://gist.github.com/jku/5262baefeb705663999a43a7cf412bfc

[jku]

Quick test with rsa keys shows some issue with go-tuf... see https://gist.github.com/jku/5262baefeb705663999a43a7cf412bfc

It seems go-tuf has an issue with RSA-PSS: luckily we actually wanted RSA-PKCS anyway (since we're looking for reproducible signatures)

This seems to work:

-        signer = CryptoSigner.generate_ecdsa()
+        signer = CryptoSigner.generate_rsa(scheme="rsa-pkcs1v15-sha256")

[jku]

I'll take this to try the "quick fix" (RSA keys by default): let's see about the refactor afterwards