Skip to content

Adds documentation for data catalog #21422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Apr 9, 2025
Merged

Adds documentation for data catalog #21422

merged 9 commits into from
Apr 9, 2025

Conversation

jonesphillip
Copy link
Contributor

Adds documentation for R2 Data Catalog

@hyperlint-ai Hyperlint AI
Copy link
Contributor

hyperlint-ai bot commented Apr 4, 2025

Howdy and thanks for contributing to our repo. The Cloudflare team reviews new, external PRs within two (2) weeks. If it's been two weeks or longer without any movement, please tag the PR Assignees in a comment.

We review internal PRs within 1 week. If it's something urgent or has been sitting without a comment, start a thread in the Developer Docs space internally.

PR Change Summary

Added comprehensive documentation for the R2 Data Catalog, including setup instructions, configuration examples, and usage guides for various query engines.

  • Introduced a 'Get Started' guide for R2 Data Catalog, detailing bucket creation and API token setup.
  • Added configuration examples for connecting Apache Spark, Snowflake, and PyIceberg to R2 Data Catalog.
  • Updated permissions in the API tokens documentation to include data catalog access.

Modified Files

  • src/content/docs/r2/api/tokens.mdx
  • src/content/docs/r2/data-migration/index.mdx
  • src/content/docs/r2/demos.mdx
  • src/content/docs/r2/examples/index.mdx
  • src/content/docs/r2/platform/index.mdx
  • src/content/docs/r2/pricing.mdx
  • src/content/docs/r2/reference/index.mdx
  • src/content/docs/r2/tutorials/index.mdx
  • src/content/partials/workers/wrangler-commands/r2.mdx

Added Files

  • src/content/docs/r2/data-catalog/config-examples/index.mdx
  • src/content/docs/r2/data-catalog/config-examples/pyiceberg.mdx
  • src/content/docs/r2/data-catalog/config-examples/snowflake.mdx
  • src/content/docs/r2/data-catalog/config-examples/spark.mdx
  • src/content/docs/r2/data-catalog/get-started.mdx
  • src/content/docs/r2/data-catalog/index.mdx
How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

@github-actions github-actions bot added size/l product:r2 R2 object storage: https://developers.cloudflare.com/r2 labels Apr 4, 2025
@jonesphillip jonesphillip changed the title Adds documentation for R2 Data Catalog Adds documentation for data catalog Apr 4, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 4, 2025 •
edited
Loading

Preview URL: https://7fa96e3b.preview.developers.cloudflare.com
Preview Branch URL: https://pjones-r2-data-catalog.preview.developers.cloudflare.com

Files with changes (up to 15)

Original Link Updated Link
https://developers.cloudflare.com/r2/data-catalog/get-started/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-catalog/get-started/
https://developers.cloudflare.com/r2/data-catalog/config-examples/spark-scala/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-catalog/config-examples/spark-scala/
https://developers.cloudflare.com/r2/data-catalog/manage-catalogs/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-catalog/manage-catalogs/
https://developers.cloudflare.com/r2/data-migration/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-migration/
https://developers.cloudflare.com/r2/data-catalog/config-examples/spark-python/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-catalog/config-examples/spark-python/
https://developers.cloudflare.com/r2/data-catalog/config-examples/snowflake/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-catalog/config-examples/snowflake/
https://developers.cloudflare.com/r2/data-catalog/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-catalog/
https://developers.cloudflare.com/r2/api/tokens/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/api/tokens/
https://developers.cloudflare.com/r2/data-catalog/config-examples/pyiceberg/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-catalog/config-examples/pyiceberg/
https://developers.cloudflare.com/r2/pricing/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/pricing/
https://developers.cloudflare.com/changelog/2025-04-10-r2-data-catalog-beta/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/changelog/2025-04-10-r2-data-catalog-beta/
https://developers.cloudflare.com/r2/demos/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/demos/
https://developers.cloudflare.com/r2/data-catalog/config-examples/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/data-catalog/config-examples/
https://developers.cloudflare.com/r2/reference/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/reference/
https://developers.cloudflare.com/r2/examples/ https://pjones-r2-data-catalog.preview.developers.cloudflare.com/r2/examples/

@jonesphillip jonesphillip force-pushed the pjones/r2-data-catalog branch from 6b4a82d to ffb61c4 Compare April 4, 2025 21:15
@jonesphillip jonesphillip requested a review from a team as a code owner April 6, 2025 15:11
@github-actions github-actions bot removed the size/l label Apr 6, 2025
@afaftrad1
Copy link

afaftrad1 commented Apr 6, 2025 via email

@kodster28 kodster28 linked an issue Apr 7, 2025 that may be closed by this pull request
Add docs for the R2 Data Catalog #21030
Closed
@garvit-gupta
Copy link
Contributor

Fixes #21030

@jonesphillip jonesphillip merged commit f541c04 into production Apr 9, 2025
11 checks passed
@jonesphillip jonesphillip deleted the pjones/r2-data-catalog branch April 9, 2025 22:46
RebeccaTamachiro pushed a commit that referenced this pull request Apr 21, 2025
@jonesphillip @Oxyjun @RebeccaTamachiro
Adds documentation for data catalog (#21422)
f7faddd 
* Adds documentation for R2 Data Catalog

* Added managing catalogs documentation and R2 Data Catalog as a product.

* Add changelog entry

* PCX review

* Fix PR comments/typos.

* Added PySpark example configuration.

* Update src/content/docs/r2/data-catalog/config-examples/spark-scala.mdx

* Added more context for data catalog auth

* Add access policy example for r2 data catalog API tokens

---------

Co-authored-by: Jun Lee <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@garvit-gupta garvit-gupta garvit-gupta left review comments

@Oxyjun Oxyjun Oxyjun approved these changes

@musamo37 musamo37 musamo37 left review comments

@irvinebroque irvinebroque irvinebroque approved these changes

@GregBrimble GregBrimble Awaiting requested review from GregBrimble GregBrimble is a code owner

@mikenomitch mikenomitch Awaiting requested review from mikenomitch mikenomitch is a code owner

@WalshyDev WalshyDev Awaiting requested review from WalshyDev WalshyDev is a code owner

@ToriLindsay ToriLindsay Awaiting requested review from ToriLindsay ToriLindsay is a code owner

@elithrar elithrar Awaiting requested review from elithrar elithrar is a code owner

@harshil1712 harshil1712 Awaiting requested review from harshil1712 harshil1712 is a code owner

Assignees

@elithrar elithrar

@irvinebroque irvinebroque

@mikenomitch mikenomitch

@GregBrimble GregBrimble

@WalshyDev WalshyDev

@harshil1712 harshil1712

@Oxyjun Oxyjun

@ToriLindsay ToriLindsay

Labels
product:r2 R2 object storage: https://developers.cloudflare.com/r2 size/xl
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add docs for the R2 Data Catalog