Add UniFFI client with Python bindings

[guzman-raphael]

Depends on #70, #74

UniFFI Note:

• An Object is when something is Rust-owned and passed by reference to the client. Methods are supported but indexing is only supported via methods e.g. getters.
• A Record is when you transfer ownership to the client and passed by value. Methods are not supported but the client provides native indexing support.
• An Enum is most similar to a Record above but for a Rust enum.

In this PR, we:

1. Add UniFFI dependency. It can generate the C FFI lib + the client-specific bindings e.g. Python. Included the uniffi-bindgen binary directly in our package since it simplifies the orcapod installation steps (users don't need to install it separately).
   1. Add async-trait dependency so async fn work with dyn traits.
   2. Add getset dependency which allows auto deriving getters for structs. Sourcing from my fork since needed to add a feature to propagate specified attributes (Add impl_attrs feature jbaublitz/getset#114).
   3. Add derive_more (display) to simplify how to modify Display implementations since there is good integration with UniFFI i.e. this is how you modify how a Rust-owned object is displayed on the client side.
2. Modify source.
   1. Expose the following in C lib using UniFFI.
      1. Object: Pod, PodJob, OrcaError, LocalDockerOrchestrator, LocalFileStore.
      2. Record: PodResult, Annotation, GPURequirement, StreamInfo, OrcaPath, Blob, RunInfo, PodRun, ModelInfo.
      3. Enum: Model, GPUModel, Input, BlobKind, ImageKind, Status, ModelID.
   2. Change any inheritance-like relationships to using Arc since UniFFI requires this for any Object usage e.g. PodJob { pod: Pod, .. } -> PodJob { pod: Arc<Pod>, .. }.
   3. Convert delete_annotation to non-generic since this is not allowed across CFFI boundary. Used a variation based on enum.
   4. Add parse_debug_name: a similar method to get_type_name but sourcing from the debug display. This is useful since it can be called with a struct or an enum.
   5. Change our opaque error from OrcaError(Kind) to OrcaError { kind: Kind} since UniFFI did not allow modifying display on tuple structs.
   6. Move MODEL_NAMESPACE constant out of trait since this is not supported in UniFFI.
3. Add Python utilities to IDE to help in building and testing the Python client.
4. Add pyproject.toml so others can easily install from source e.g. pip install git+https://github.com/walkerlab/orcapod.
5. Reorg non-Rust test data into a extra/ subdirectory.
6. Add tests/extra/python/smoke_test.py to verify some basic features in Python client. Supports using debugger.
7. Add docs/examples and move diagram to docs/images/.
8. Clean up .clippy.toml.
   1. UniFFI generates some idents we don't have control over so set min-ident-chars-threshold=1 (default).
   2. Update allow-duplicate-crates to current offenders

[Synicix]

Some discussion points for next meeting

[Synicix]

What was the reason to change

pub fn make_path<T>(&self, hash: &str, relpath: impl AsRef<Path>) -> PathBuf

to include model: &T, when users of this function can still use

::<T>make_path()

[Synicix]

What was the reason behind adding this function when it does similar to what get_type_name does?

[Synicix]

Perhaps go with ModelType or Type?

[Synicix]

What was the reason to add extra?

[Synicix]

Perhaps later we should match against the kind to see if it returns the correct error type?

[Synicix]

Stale comment code?

[codecov]

Codecov Report

Attention: Patch coverage is 86.98630% with 19 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/uniffi/model.rs	16.66%	10 Missing ⚠️
src/uniffi/orchestrator/docker.rs	40.00%	3 Missing ⚠️
src/uniffi/error.rs	60.00%	2 Missing ⚠️
src/uniffi/store/filestore.rs	87.50%	2 Missing ⚠️
src/uniffi/orchestrator/mod.rs	0.00%	1 Missing ⚠️
src/uniffi/store/mod.rs	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[guzman-raphael]

Patch coverage here is less than 100% but I think for this particular case it is reasonable so leaving it as-is. It is because of UniFFI.

Reasons:

1. UniFFI has not added automatically_derived to all generated impl cases which means that the generated code isn't omitted from coverage. Fixing this would require us to enhance UniFFI and use our own forked version of UniFFI until they accept the improvement.
2. Omitting coverage from generated functions (for us these are from UniFFI) is only possible by moving them to a specific file that can be omitted on Rust stable which would be too disruptive simply just for coverage. On Rust nightly, this feature seems relevant but even then, it would require us to enhance UniFFI and use our own forked version of UniFFI until they accept the improvement.

The benefit doesn't seem worth it right now so I recommend we defer.

[Synicix]

Reviewed with zoom meeting