A Python API extractor written in Rust. I got frustrated with the standard tools used for documentation generation used in Python, namely that all of the ones I've worked with have at least one of the following to characteristics:
- slow
- poor developer experience (e.g. Sphinx lacks a
serve
functionality)
Number 1 is particularly a concern for packages that are slow to import, or have a convoluted structure (which is the case for some packages I work with)
There are many excellent Static Site Generators like Hugo or Zola that are fast and offer a much better experience when developing. Typically, documentation are static pages so they should be a perfect fit, except for one issue: automatic API reference generation. This was a big showstopper... until now!
SnakeDown is an experiment to get to a place where using Hugo or Zola for python documentation generation is a viable option.
(Currently mostly for shits and giggles, but who knows.)
While the project might not cover all of these goals yet, these are some of the things I try to keep in mind when working on SnakeDown (in loose order of precedence):
- Sensible defaults over infinite configurability. Infinite configuration comes with a lot of complexity both for user and developer, and in my opinion it is not worth it, especially at the stage SnakeDown is at now. I'd rather do 10 things right 80% of the time, then 2 things right 99.999% of the time.
- Correctness over speed While speed is an explicit goal of this project, correctness should take precedent. Doing the wrong thing extremely fast is useless.
- Be fast We only parse the source code to extract the docstrings from it, but we don't actually have to do any execution. While this might mean we can't always fully expand everything, type signatures are constants or names in the vast majority of cases, so this is a decent alternative. This by itself should make us Fast Enough ™️
- Be fun This is a personal project for the time being, and I like to have fun with those.
Currently, SnakeDown is solidly in the MVP state. While I am quite happy with it so far, you use it at your own risk. That said, I always welcome bug reports and feature requests if you do decide to try it! Below is a loose planning of the features I want to work on
- Walk file tree to find python objects
- Parse them to extract documentation
- Dump documentation in similar file structure to original package
- Fill out the CLI
- Test output in SSGs (zola, hugo for now, please submit a feature request if you want others included)
- Logging at appropriate levels
- Parse/render docstring formats like numpy and google so we can render them better
- Configuration file
- Support multiple formats? (md, rst)
- Do reference linking inside the docs
- Do reference linking to external docs
- Benchmarking & optimisation
No. It is a personal project for the time being so while there may be persiods of inactivity, I intend to keep working on it until it serves my personal needs. (Poe's law dictates that I mention this is mostly a joke. It's a common question for FOSS projects that don't have the high velocity of larger projects and this is a tongue in cheek way of beating this question to the punch)
To quote one of the giants in our field BurtnSushi:
ripgrep is a project whose contributors are volunteers. A release schedule adds undue stress to said volunteers. Therefore, releases are made on a best effort basis and no dates will ever be given.
An exception to this can be high impact bugs. If a ripgrep release contains a significant regression, then there will generally be a strong push to get a patch release out with a fix. However, no promises are made.
(source: ripgrep)
The same applies to SnakeDown.
Three main reasons:
- I was first going to call it "SnakeOil" for extraing things from snakes of dubious value but that name was already taken
- it's about shaking down your python code for docs
- I find it funny
Not yet, but I suspect they will.
To develop SnakeDown you'll want to have these tools installed:
just
A command runner to run (and document) workflows we run, including installing dev and publish dependenciestypos-cli
Fixing typos... not that we make any... but you know, just in case.taplo-cli
Keeping our.toml
files nice and cleanbacon
A runner that will watch your files and run checks, tests, linting etc. when they change. Very useful while developing
If you have to publish, or otherwise fiddle with dependencies of SnakeDown you'll want these installed as well:
cargo-semver
A cargo plugin to check that we haven't accidentally broken our API when we didn't mean to.cargo-edit
A cargo plugin for managing dependencies, incl updating them.git-cliff
A neat tool to generate our changelog
This repo was initially setup using cargo-generate
and this template