Add nanobind to mlir-nvidia Dockerfile

[makslevental]

To support llvm/llvm-project#117922

[makslevental]

@joker-eph does this container get rebuilt each time the bot runs? I would assume so but just double checking there's not another step.

[hawkinsp]

I'd guess that you may be able to set install_pip_requirements=True on the failing builders in builders.py.

[makslevental]

I'd guess that you may be able to set install_pip_requirements=True on the failing builders in builders.py.

ah good catch. There's a comment in the Dockerfile here that the reqs should be pinned so I'll leave this (@joker-eph will comment on how he wants to handle it) but I'll do that for the AMD bot in a separate PR.

[joker-eph]

The comment about pinned revision was about reproducibility of the docker image. That said the same comment could apply to the content of the requirements.txt in the monorepo... If we don't pin revision bisections and other reproduction can become difficult I think.

The install_pip_requirements didn't exist initially, I added it earlier this year to fix similar issue on pre-merge config and enable testing the Python bindings on these bots replicating the pre-merge config.

We could cleanup the docker image for the mlir-nvidia bot and switch to install_pip_requirements, that seems like a net improvement I think.

[makslevental]

The comment about pinned revision was about reproducibility of the docker image. That said the same comment could apply to the content of the requirements.txt in the monorepo... If we don't pin revision bisections and other reproduction can become difficult I think.

Yea I don't know why those deps are ranged rather than fixed. I can make a discourse post about it.

We could cleanup the docker image for the mlir-nvidia bot and switch to install_pip_requirements, that seems like a net improvement I think.

The AMD bot does this

ADD "https://raw.githubusercontent.com/llvm/llvm-project/main/mlir/python/requirements.txt" requirements.txt
RUN python3 -m pip install --upgrade pip && python3 -m pip install -r requirements.txt

which can actually be shortened here to just

pip install -r https://raw.githubusercontent.com/llvm/llvm-project/main/mlir/python/requirements.txt

Is this what you had in mind for clean up?

[joker-eph]

What I meant by cleanup is to not install these at all in the image, since the bot will pip install them later.

[makslevental]

What I meant by cleanup is to not install these at all in the image, since the bot will pip install them later.

Done

[makslevental]

Bump

[joker-eph]

FYI, post merge this will only take effect when the master buildbot restarts.
@gkistanova is handling this is believe.

[makslevental]

FYI, post merge this will only take effect when the master buildbot restarts. @gkistanova is handling this is believe.

Good to know - are the bots restarted on a periodic basis or just when someone triggers it?

[joker-eph]

I suspect it is manual, but I am not sure. I usually email/ping @gkistanova :)

[joker-eph]

I think you forgot to update the PR title/description before merging?

[makslevental]

Sorry yes that was a mistake on my part.

[hawkinsp]

@makslevental Do I need to resend you the original nanobind PR or can you reland that without me doing anything?

[makslevental]

@makslevental Do I need to resend you the original nanobind PR or can you reland that without me doing anything?

I can reland by reverting the revert (I'll fix up whatever has drifted) but I'll email Galina first to make sure the bots are rebooted.