Skip to content

Commit 0953704

Browse files
vsochvsoch
authored
Merge pull request #38 from flux-framework/2024-radiuss-aws-july
refactor: content and organization for RADIUSS 2024
2 parents ff66ebf + 95798a0 commit 0953704

33 files changed

+273
-2030
lines changed

‎.github/workflows/docker-builds.yaml

+4-1
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,13 @@ jobs:
1010
strategy:
1111
fail-fast: false
1212
matrix:
13-
# Tutorial is over - these builds are disabled
13+
# Tutorials are over - these builds are disabled
1414
# test: [["2023-RADIUSS-AWS/JupyterNotebook", "docker/Dockerfile.hub", "ghcr.io/flux-framework/flux-jupyter-hub:2023"],
1515
# ["2023-RADIUSS-AWS/JupyterNotebook", "docker/Dockerfile.init", "ghcr.io/flux-framework/flux-jupyter-init:2023"],
1616
# ["2023-RADIUSS-AWS/JupyterNotebook", "docker/Dockerfile.spawn", "ghcr.io/flux-framework/flux-jupyter-spawn:2023"]]
17+
# ["2024-RIKEN-AWS/JupyterNotebook", "docker/Dockerfile.hub", "ghcr.io/flux-framework/flux-jupyter-hub:riken-2024"],
18+
# ["2024-RIKEN-AWS/JupyterNotebook", "docker/Dockerfile.init", "ghcr.io/flux-framework/flux-jupyter-init:riken-2024"],
19+
# ["2024-RIKEN-AWS/JupyterNotebook", "docker/Dockerfile.spawn", "ghcr.io/flux-framework/flux-jupyter-spawn:riken-2024"]]
1720
test: [["2024-RADIUSS-AWS/JupyterNotebook", "docker/Dockerfile.hub", "ghcr.io/flux-framework/flux-jupyter-hub:radiuss-2024"],
1821
["2024-RADIUSS-AWS/JupyterNotebook", "docker/Dockerfile.init", "ghcr.io/flux-framework/flux-jupyter-init:radiuss-2024"],
1922
["2024-RADIUSS-AWS/JupyterNotebook", "docker/Dockerfile.spawn", "ghcr.io/flux-framework/flux-jupyter-spawn:radiuss-2024"],

‎2024-RADIUSS-AWS/JupyterNotebook/README.md

+5-5
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
This set of tutorials provides:
44

55
- [Building Base Images](#build-images)
6-
- [Deploy A Cluster to AWS or Google Cloud Using](#deploy-to-kubernetes) using Google Cloud or AWS
76
- [Local Development or Usage](#local-usage)
7+
- [Deploy A Cluster to AWS or Google Cloud Using](#deploy-to-kubernetes) using Google Cloud or AWS
88

99
Pre-requisites:
1010

@@ -19,6 +19,7 @@ For AWS Tutorial Day users:
1919
## Build Images
2020

2121
Let's build a set of images - one spawner and one hub, and an init. You can customize the tag to your liking.
22+
Remember that if you just want to test locally, you can jump to the [local usage](#local-usage) section.
2223

2324
```bash
2425
docker build -t ghcr.io/flux-framework/flux-jupyter-hub:radiuss-2024 -f docker/Dockerfile.hub .
@@ -30,12 +31,11 @@ Note that these are available under the flux-framework organization GitHub packa
3031
to build them unless you are developing or changing them.
3132

3233
If you do build (and use a different name) be sure to push your images to a public registry (or load them locally to your development cluster).
33-
Remember that if you just want to test locally, you can jump to the [local usage](#local-usage) section.
3434

35-
## Local Deploy
3635

37-
While the tutorial here is intended for deployment on AWS or Google Cloud, you can also give it a try on your local machine with a single container! You will need to [install Docker](https://docs.docker.com/engine/install/).
38-
When you have Docker available, you can build and run the tutorial with:
36+
## Local Usage
37+
38+
While the tutorial here is intended for deployment on AWS or Google Cloud, you can also give it a try on your local machine with a single container! You will need to [install Docker](https://docs.docker.com/engine/install/). When you have Docker available, you can build and run the tutorial with:
3939

4040
```bash
4141
docker build -t flux-tutorial -f docker/Dockerfile.spawn .

‎2024-RADIUSS-AWS/JupyterNotebook/docker/Dockerfile.spawn

+16-9
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM fluxrm/flux-sched:focal
1+
FROM fluxrm/flux-sched:jammy
22

33
# Based off of https://github.com/jupyterhub/zero-to-jupyterhub-k8s/tree/main/images/singleuser-sample
44
# Local usage
@@ -26,8 +26,8 @@ RUN apt-get update \
2626
ca-certificates \
2727
dnsutils \
2828
iputils-ping \
29-
python3.9 \
30-
python3.9-dev \
29+
python3 \
30+
python3-dev \
3131
python3-pip \
3232
python3-venv \
3333
openmpi-bin \
@@ -47,8 +47,6 @@ RUN python3 -m pip install -r requirements.txt && \
4747
python3 -m pip install ipython==7.34.0 && \
4848
python3 -m IPython kernel install
4949

50-
COPY ./tutorial /home/jovyan/flux-tutorial-2024
51-
5250
# This is code to install DYAD
5351
# This was added to the RADIUSS 2023 tutorials on AWS
5452
RUN git clone https://github.com/openucx/ucx.git \
@@ -76,22 +74,31 @@ RUN git clone https://github.com/flux-framework/dyad.git \
7674
&& cd ../.. \
7775
&& rm -rf dyad
7876

79-
8077
# This adds the flux-tree command, which is provided in flux-sched source
8178
# but not installed alongside production flux-core
8279
COPY ./flux-tree/* /usr/libexec/flux/cmd/
8380
RUN chmod +x /usr/libexec/flux/cmd/flux-tree*
8481

82+
RUN apt-get update && apt-get install -y nodejs && apt-get clean && rm -rf /var/lib/apt/lists/*
83+
84+
RUN wget https://nodejs.org/dist/v20.15.0/node-v20.15.0-linux-x64.tar.xz && \
85+
apt-get update && apt-get install -y xz-utils && rm -rf /var/lib/apt/lists/* && \
86+
xz -d -v node-v20.15.0-linux-x64.tar.xz && \
87+
tar -C /usr/local --strip-components=1 -xvf node-v20.15.0-linux-x64.tar
88+
8589
# This customizes the launcher UI
8690
# https://jupyter-app-launcher.readthedocs.io/en/latest/usage.html
8791
RUN python3 -m pip install jupyter_app_launcher && \
8892
python3 -m pip install --upgrade jupyter-server && \
93+
python3 -m pip install jupyter-launcher-shortcuts && \
8994
mkdir -p /usr/local/share/jupyter/lab/jupyter_app_launcher
90-
COPY ./docker/jupyter-launcher.yaml /usr/local/share/jupyter/lab/jupyter_app_launcher/config.yaml
91-
ENV JUPYTER_APP_LAUNCHER_PATH /usr/local/share/jupyter/lab/jupyter_app_launcher
95+
96+
COPY ./tutorial /home/jovyan/
97+
COPY ./docker/jupyter-launcher.yaml /usr/local/share/jupyter/lab/jupyter_app_launcher/jp_app_launcher.yaml
98+
ENV JUPYTER_APP_LAUNCHER_PATH=/usr/local/share/jupyter/lab/jupyter_app_launcher/
9299

93100
# Give jovyan user permissions to tutorial materials
94-
RUN chmod -R 777 ~/flux-tutorial-2024
101+
RUN chmod -R 777 ~/ /home/jovyan
95102

96103
WORKDIR $HOME
97104
COPY ./docker/flux-icon.png $HOME/flux-icon.png

‎2024-RADIUSS-AWS/JupyterNotebook/docker/jupyter-launcher.yaml

+22-15
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,26 @@
1-
# These don't work, but we can try again next year.
2-
#- title: Flux Tutorial Notebook
3-
# description: This is the main Flux Framework Tutorial
4-
# source: /home/jovyan/flux-tutorial/notebook/flux.ipynb
5-
# cwd: /home/jovyan/flux-tutorial/notebook/
6-
# type: notebook
7-
# catalog: Notebook
8-
# icon: /home/jovyan/flux-icon.png
1+
- title: Flux Tutorial Notebook
2+
description: This is the main Flux Framework Tutorial
3+
type: jupyterlab-commands
4+
icon: ./flux-icon.png
5+
source:
6+
- label: Flux Tutorial
7+
id: 'filebrowser:open-path'
8+
args:
9+
path: 01_flux_tutorial.ipynb
10+
icon: ./flux-icon.png
11+
catalog: Notebook
912

10-
# - title: Dyad Notebook Tutorial
11-
# description: This is a tutorial for using Dyad
12-
# source: /home/jovyan/flux-tutorial/notebook/dyad.ipynb
13-
# cwd: /home/jovyan/flux-tutorial/notebook/
14-
# type: notebook
15-
# catalog: Notebook
16-
# icon: /home/jovyan/flux-icon.png
13+
- title: Dyad Notebook Tutorial
14+
description: This is a tutorial for using Dyad
15+
type: jupyterlab-commands
16+
icon: flux-icon.png
17+
source:
18+
- label: Dyad Tutorial
19+
id: 'filebrowser:open-path'
20+
args:
21+
path: dyad_dlio.ipynb
22+
icon: ./flux-icon.png
23+
catalog: Notebook
1724

1825
- title: Flux Framework Portal
1926
description: Flux Framework portal for projects, releases, and publication.

‎2024-RADIUSS-AWS/JupyterNotebook/flux-tree/flux-tree

+1-1
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,7 @@ jsonify() {
212212

213213
if [[ "${avail}" = "yes" ]]
214214
then
215-
avg=$(flux ion-resource stat | grep "Avg" | awk '{print $4}')
215+
avg=$(flux ion-resource stats | grep "Avg" | awk '{print $4}')
216216
el_match=$(awk "BEGIN {print ${avg}*${njobs}*1000000.0}")
217217
fi
218218

0 commit comments

Comments
 (0)