Convert our documentation to MyST Markdown

[choldgraf]

Context

This documentation is written in reStructuredText, the default for Sphinx documentation sites. However, the myst parser sphinx package allows you to write Sphinx documentation in Markdown with all of the same functionality.

Markdown is much more commonly-used and (arguably) more user-friendly than reStructuredText, and using it may make our documentation easier to maintain and for other non-rST users to contribute to it.

Moreover, there is an rst-to-myst package that makes it easy to do a one-time conversion of our documentation into MyST Markdown, so it should be quick to do this.

Proposal

I propose that we run rst-to-myst on all of our documentation except for the "kitchen sink" (since we copy that directly from other docs). From now on, we use MyST Markdown for our documentation rather than rST.

Note
To clarify: this is not a dependency of the theme itself, just our documentation. So it wouldn't affect any downstream users.

[12rambau]

presentation

Let me introduce myself, I’m the devil’s advocate 😈

I assume that you want to change everything that lives in the docs folder from .rst to .md. The main reason for that would be that peoples tends to have difficulties to contribute to our doc right ?

questions

Is there a precedent of people refusing to contribute to pydata-sphinx-theme documentation because of the .rst format? My guess is that people using the pydata-sphinx-theme are here because they need to document massive lib/software. For simpler stuff, with less pages, other theme are more appropriate. What I mean is that these users need lots of custom stuff and already know sphinx (i.e. .rst).

Do we want this librairy to depend on myst ? there were already issues with just the Sphinx version, is using myst as well to transform .md files to .rst to then call Sphinx to create html kind of an overkill?

I’m always a bit confused with people complaining about .rst. I’ve search on the web and of course fell on your article and I agree with the 6 points you are mentioning. Even though, .md is kinda limited. To write an admonition in myst I need to write:

<div class="admonition note">
    <p>Paragraph 1</p>
    <p>Paragraph 2</p>
</div>

Which is, according to me, writing down pure html and not markdown anymore. On the other hand I have that:

.. note::

    Paragraph 1
    Paragraph 2

Is it that complex ?

I also agree that

``foo`` which is equivalent to :code:`foo`

is kinda disturbing but once you get your head around it, roles are extremely powerful tools. I’m curious how they are included in .md files.

a priori

In short, I see why some people would prefer .md (if you don’t need fancy roles or directive that will be just fine) and thus use myst to gain access to Sphinx directives through classes in inline html. But I don’t see why the documentation of this theme should be moved to this format; even less if we keep the kitchen sink meaning that we’ll have a mixbag of different formats.

conclusion

that was my 2 cents, devil's advocate out 😈

[choldgraf]

Just some quick clarifications:

• the conversion from rst to Myst is a one-time thing. From then on, Myst is parsed directly into sphinx.
• the syntax you shared for Myst directives isn't right. There is a myst syntax for directives and roles. Here is a good reference page to compare to rst: https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#syntax-directives

I'll admit to having a personal preference for Myst markdown. I find that it is easier to remember and work with than rst, and easier to teach to others who are familiar with markdown. For me it's something that makes my docs contributions more efficient and enjoyable, but I recognize that's a personal preference!

[12rambau]

thanks for the clarification. looking at the doc it's looking exactly like .rst to me ;-) (and that's a great thing because you can then use any custom directive). Which also mean that it's really a matter of taste 🫢

One remaining question though how does it behave with the following list:

#. a first item 
#. a second item

wich will be replaced with a numbered list in .rst but not in normal .md. Is it also interpreted by myst ?

[choldgraf]

I believe the way to do this in markdown is just to use 1 over and over. E.g.

1. One
1. Two
1. Three

Becomes

1. One
2. Two
3. Three

Here is a nifty demo page to see more Myst syntax and try it out live: https://executablebooks.github.io/mystjs/features/commonmark.html

[damianavila]

I like the general proposal written by @choldgraf at the top message.
I also see some good points from @12rambau on his devil's advocate post 😉.

Regardless of the subjective/personal preferences, I think it is important to ponder:

1. what our users are doing in terms of rst usage
2. be proactive in showing our users there might be other alternatives to rst

The question is what is the strategic approach we want to take? Do we support people showing the usual/most common things or do we show them cutting edge stuff and try to push them forward (modulo you understand myst as a step forward, which I believe it is... but others could not agree).

[jarrodmillman]

Myst-parser aggressively upper pins sphinx. So I worry that adding it as a dependency make it more difficult for the community to support new releases of Sphinx. I think it is more important to support new versions of Sphinx than it is to change the format of the documentation for this project for the possible benefit of making it easier for people to contribute to the documentation.

More concretely, I volunteered to release a version of pydata-sphinx-theme so I can test numpydoc on Sphinx 5 easily. It would be unfortunate if we get in a situation where numpydoc needs to first get myst-parser to make a new release, so pydata-sphinx-theme can make a new release, so I can test whether I need to make a new release of numpydoc every year when sphinx makes a new release.

[choldgraf]

I don't think this issue would affect any downstream projects. This would only affect the documentation builds of this theme's docs alone. It wouldn't be a dependency of the theme itself

[12rambau]

I think this conversation needs to be settled as there is now a mix of both .md and .rst files in our documentation. I recently created a PR #894 where I was forced to go to myst documentation to understand how I should add options to a directive.

As the main purpose of this suggestion was to help people dive in our documentation; note that it actually slowed me down as I'm now fluent in .rst and don't feel the need to move to myst.

To give my own impression, even though I agree with

Markdown is much more commonly-used and (arguably) more user-friendly than reStructuredText

I'm pretty sure it does not apply to the people that use Sphinx and build documentations. My feeling is that it's too opinionated.

[choldgraf]

Personally I am fine keeping a mix of both and generally following a strategy of:

• whoever is willing to write the documentation should use rST vs. MyST depending on their preference and comfort level
• generally, don't totally change the markup language of a pre-existing page unless there's discussion ahead of time and agreement that it's a good idea

Personally, I much prefer MyST given that it has the exact same functionality of rST, but shares most of its syntax with another language that I use every day (commonmark markdown).

[12rambau]

I like this "come as you are" approach, should we close this issue then?