docs: add OpenAPI routes documentation and add autogenerated API reference with mkdocstrings #701
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fix: annoying wrapper script docs: fix task doc fix: add ci skip settings-doc-check
fix: cleanups
fix: well that didn't work fix: update tasks fix: format settings doc files
…API ref page looks like
ryuwd
changed the title
ryuwd
changed the title
Add comprehensive REST API route documentation automatically generated from the OpenAPI specification using the neoteroi-mkdocs plugin. Changes: - Add neoteroi CSS styling (v1.1.3) for OpenAPI documentation rendering - Create generate_openapi_spec.py script to generate docs/openapi.json - Add docs/dev/reference/routes/index.md with route documentation page - Integrate neoteroi.mkdocsoad plugin in mkdocs.yml configuration - Add generate-openapi-spec pixi task for OpenAPI spec generation - Make mkdocs-build depend on generate-openapi-spec task - Add generate-entrypoints-docs pre-commit hook - Exclude docs/dev/reference/api/ from mdformat to preserve indentation - Ignore generated docs/openapi.json and Copilot instructions in git - Update routers index page to clarify REST API vs infrastructure docs - Fix mkdocstrings directive indentation across API reference docs - Enhance router documentation with better descriptions and examples The OpenAPI spec generation uses test fixtures similar to the test suite, creating a minimal DiracX app with test auth settings, config repository, and database URLs. The spec is generated before each documentation build. The neoteroi plugin renders the OpenAPI spec into human-readable route documentation with request/response examples, while mdformat exclusion preserves the required indentation for mkdocstrings YAML options.
ryuwd
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR extends #698 (settings-doc) by adding comprehensive documentation infrastructure for DiracX's REST API routes and Python API reference using mkdocstrings, Griffe, and Griffe-Pydantic.
Changes Summary
OpenAPI Routes Documentation
scripts/generate_openapi_spec.py- Generates OpenAPI specification from FastAPI app (27 paths, 35 schemas)docs/dev/reference/routes/index.md- REST API routes documentation using neoteroi pluginPython API Reference Documentation
Created 93+ API reference pages covering all major DiracX modules:
docs/dev/reference/api/writing-api-docs.md- Comprehensive guide for writing API documentation with Google-style docstringsdocs/dev/reference/api/CONTRIBUTING.md- Guidelines for maintaining API documentationdocs/dev/reference/api/check_coverage.py- Pre-commit hook ensuring complete API coverageExtension Entry Points Documentation
generate-entrypoints-docs) ensures entry point documentation stays synchronized with codescripts/generate_entrypoints_docs.py- Automatically generates documentation for DiracX extension entry pointsdocs/dev/reference/entrypoints.md- Auto-generated reference cataloging all available entry points for creating DiracX extensionsDocumentation Tooling
mkdocstrings + griffe + griffe-pydantic for automatic Python API documentation
neoteroi-mkdocs for OpenAPI specification rendering
Configuration Updates
Updated
mkdocs.ymlwith:show_if_no_docstring: true(see Open Questions)Updated
pixi.tomlwith:generate-openapi-spectask for OpenAPI generationmkdocs-buildtask now depends on OpenAPI generationUpdated
.pre-commit-config.yaml:settings-doc-checkhook to validate settings documentationapi-doc-checkhook to ensure API documentation coveragedocs/dev/reference/api/from mdformat (preserves mkdocstrings directive indentation)Example Documentation
JobDBclass indiracx-db/src/diracx/db/sql/job/db.pywith comprehensive docstrings as a reference example showing:Open Questions & Discussion Points
1. Which docstring format to use?
This PR: Google-style docstrings (couldn't really find a consensus yet in the codebase)
Others are supported:
https://mkdocstrings.github.io/griffe/reference/docstrings/
2. API Documentation Coverage
Current approach: Pre-commit hook that checks all Python modules have corresponding
.mdfiles indocs/dev/reference/api/.Is there a nicer way?
3.
show_if_no_docstring: trueCurrent setting: Enabled globally in mkdocs.yml
Rationale:
Trade-off:
Should undocumented members be visible in API docs, or should we require docstrings before showing them?
4. Documentation Organization
Current structure: Organized by package (core, routers, logic, db, cli)
It's a bit difficult to read the pages and navigate. Open to suggestions!
5. Example Documentation Density
Current approach: Added comprehensive examples to
JobDBclass as a reference6. Pydantic Model Documentation
Griffe-Pydantic automatically extracts:
Field(description=...)Do we need anything else?
7. Adding docstrings where missing
self explanatory
Dependencies
This PR builds on the settings documentation work from #698 and uses similar infrastructure (auto-discovery, validation).