Inconsistent output for optional parameters/attributes

[doerwalter]

Describe the bug

The type of an optional parameter in

:param Optional[str] bar: The bar parameter

will be output differently than that of an optional attribute:

.. attribute :: bar
   :type: Optional[str]

or when there's a type annotation on the paramter itself:

def foo(bar: Optional[str]):
   ...

How to Reproduce

The following Python sourceode:

class Test:
    """
    Test class.
    """
    def foo(self, bar: Optional[str]):
        """
        .. attribute :: bar
           :type: Optional[str]

            The bar attribute

        :param Optional[str] bar: The bar parameter.
        """
        pass

when incorporated into a Sphinx project via .. automodule:: etc. gives the following HTML output:

The type annotation Optional[str] should be displayed consistently as str | None in all three spots, instead of as Optional[str] for :param:, and str | None for .. attribute and type annotations in the function signature.

Environment Information

`sphinx-build --bug-report` gives:


Platform:              darwin; (macOS-10.16-x86_64-i386-64bit)
Python version:        3.9.13 (v3.9.13:6de2ca5339, May 17 2022, 11:23:25)
[Clang 6.0 (clang-600.0.57)])
Python implementation: CPython
Sphinx version:        7.1.0
Docutils version:      0.18.1
Jinja2 version:        3.1.2
Pygments version:      2.15.1

Sphinx extensions

No response

Additional context

No response

[picnixz]

It's funny because this is exactly what I've been doing yesterday. Actually, the reason why the docstring still contains Optional[str] is because it is never updated. More precisely, it is first transformed into

:param bar: ...
:type bar: Optional[str]

and left untouched. The :type: field for parameters is not handled the same as the :type: option for the .. attribute:: directive. For the latter, the :type: option value is formatted using sphinx.domains.python._parse_annotation (see PyAttribute.handle_signature) but the original docstring value is left untouched (only the resulting docutils node is affected). For the former, nothing is done and the field list uses the value of :type bar: directly, namely Optional[str]. If you want to convert this value as well, we have two possibilities:

• Directly do it at the level of the Napoleon / Google docstring parser. That way, the docstring is transformed at an early stage. This would require a variant of sphinx.domains.python._parse_annotation but since I've already worked quite a lot with AST, it shouldn't be too hard for me to come up with a good implementation. Also, for efficiency reasons, we could disable this behaviour by default. In addition, I would like to support PyCharm callable and tuple syntax:

  :type func: (U, V) -> R
  :type pair: (int, int)

The output would then be Callable[[U, V], R] and tuple[int, int]. This would be very convenient for every projects with docstrings at the level of the source code (again this would be a feature to enable).

• Alternatively, we could do the transformation only for the docutils nodes. In other words, the RST auto-generated content would still contain Optional[str] but when creating the corresponding nodes, the text would be changed to str | None. This would be handled by PyTypedField.make_xrefs.

The reason why Optional[str] is converted to a type annotation is because it is resolvable as a (real) annotation at runtime (and is not parsed from the docstring). The involved formatter is sphinx.util.inspect.stringify_signature.

[electric-coder]

@picnixz two relevant side notes:

1. To have Optional render as a hyperlink in the docstring it would have to be written as typing.Optional.
2. Using the explicit str | None notation the right-side None currently never becomes a hyperlink see py domain: None is no longer hyperlinked after 4.4.0 #10899