Lance Namespace Specification is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines should integrate with Lance tables.
Directory | Description |
---|---|
spec | Lance Namespace Specification |
rust/lance-namespace-reqwest-client | Generated Rust reqwest client for Lance REST Namespace |
python/lance_namespace_urllib3_client | Generated Python urllib3 client for Lance REST Namespace |
java/lance-namespace-apache-client | Generated Java Apache HTTP client for Lance REST Namespace |
java/lance-namespace-springboot-server | Generated Java SpringBoot server for Lance REST Namespace |
We use OpenAPI Generator to generate various clients and servers for the catalog specification. We recommend installing the tool through pip for consistent experience across platforms. First time setup of virtual environment and installation:
python3 -m venv .env
source .env/bin/activate
pip install -r requirements.txt
If you want to build the entire project, or develop python specifically, you need to install poetry. To quickly install it in your virtual environment:
pip install -r python/requirements.txt
There are 3 commands that is available at top level as well as inside each language folder:
make clean
: remove all codegen modulesmake gen
: codegen and lint all modules (depends onclean
)make build
: build all modules (depends ongen
)
You can also run make <command>-<language>
to only run the command in the specific language, for example:
make gen-python
: codegen and lint all Python modulesmake build-rust
: build all Rust modules
You can also run make <command>-<language>-<module>
inside a language folder to run the command against a specific module, for example:
make gen-rust-reqwest-client
: codegen and lint the Rust reqwest client modulemake build-java-springboot-server
: build the Java Spring Boot server module