Dask is an open-source Python library for parallel/distributed computing.

• This tutorial explains two fundamental problem paralleization concepts: domain decomposition and functional decomposition.
• It exemplifies how these two concepts can be achieved using Dask.
• It exemplifies how Dask enable parallel computing/distributed, out-of-core computations and scalability (from a local machine to cloud computing systems)

This tutorial start at 0.Introduction.

Run this tutorial using your own machine

• Create a virtual enviroment using conda or pip.
• Install the requirements.txt
• Run the jupyterlab enviroment.

Run this tutorial using Vagrant

Vagrant automates the deployment of virtual machines. The Vagrantfile defines a ready to use jupyterlab virtual machine for this tutorial. Use the following commands to deploy the machine.

Install vagrant

sudo apt update
sudo apt install virtualbox
sudo apt install vagrant

Provision the virtual machine

vagrant up

Get into the virtual machine

vagrant ssh

Stop the virtual machine

vagrant ssh

Destroy the virtual machine

vagrant ssh

Use the following url to access the jupyterlab enviroment.

http://192.168.33.10:8000

This tutorial was presented in

1. HPCSS2023: Colombian HPC Summer School 2023, June 20 - 22, 2023.
2. CARLA2023: Latin American High Performance Computing Conference, September 18 - 22

• This project is partially supported by CyberColombia
• We thank Chameleon Cloud for its support in compute time.
• We thank Coiled for their technical support and compute time provided on their ready-to-use Dask Cloud Computing Platform.
• We thank Naty Clementi from Coiled for her support and suggestions to improve this tutorial.
• We thank Alfonso Ladino for extending this tutorial to include an example of processing Zaar analysis-ready radar data by streaming it from the cloud and processing it on an HPC system.