Add a `--index-urls` CLI flag for pip-install magics

[agriyakhetarpal]

Problem

piplite supports passing a string or a list of strings of index URLs

pyodide-kernel/packages/pyodide-kernel/py/piplite/piplite/piplite.py

Line 134 in 36aedb9

index_urls: list[str] | str | None = None,

but it does not yet provide a CLI flag for them, so it is currently not possible to write commands %pip install numpy --index-urls https://anaconda.org/scientific-python-nightly-wheels/simple, as extra indices are/will be ignored and NumPy 2.0.2 from the Pyodide distribution is installed instead.

Proposed Solution

Extend https://github.com/jupyterlite/pyodide-kernel/blob/36aedb97d38cf283df8c3b0d65d7d6b8e575f0ab/packages/pyodide-kernel/py/piplite/piplite/cli.py to have a simple --index-urls option that is passed to piplite.py, at first. It could be extended to --index-url or --extra-index-url options later if and when such a change is brought upstream in micropip.

Additional context

xref: pyodide/pyodide#4898 (comment), as I've been meaning to test out the rollout of CORS headers for the Anaconda.org PyPI-like index through piplite, but I found I was unable to do so, while micropip works.

[agriyakhetarpal]

I'm happy to assign this to myself, and volunteer to work on it next week :D

[jtpio]

I'm happy to assign this to myself, and volunteer to work on it next week :D

Thanks for volunteering!

[bollwyvl]

Sure, that's a fine one to support. In general, when adding features to piplite, I would suggest:

• try to match as closely as possible as what a "real" %pip install would do in IPython
  • so -i and --index-url
• support parsing of said feature as a line of -r requirements.txt
• add it as a site-level configurable in the kernel section of jupyter-lite.json
  • this has been discussed elsewhere (Allow to install extra packages from extra indexes from jupyter-lite.json #158, pre-install packages concurrently #164), and likely makes the most sense as something like:

{
  "jupyter-lite-schema-version": 0,
  "jupyter-config-data": {
    "appName": "JupyterLite Pyodide Kernel",
    "litePluginSettings": {
      "@jupyterlite/pyodide-kernel-extension:kernel": {
        "pipliteInstallDefaultOptions": {
          "indexUrls": ["http://..."]
        }
      }
    }
  }
}

Of note, until micropip 0.9.0+ lands in a pyodide release, there will be some fairly surprising behavior with under-specified installs, or %pip calls that occur in the same cell as imports due to the auto-import behavior. So:

%pip install thing-i-want-but-also-in-pyodide -i http://wherever
import some_thing_provided_by_that_package

vs

%pip install thing-i-want-but-also-in-pyodide -i http://wherever

and then

import some_thing_provided_by_that_package

Custom indexes are not included in this, as no upstream list of wheels provide importable names, to my knowledge.