Why?
- you are using singularity containers
- you need jupyter notebooks
What we already have?
For single jupyter notebooks on Mistral, we have the start-jupyter script. This script makes it possible starting jupyter notebooks by providing some parameters and without strugling with ssh tunneling issues.
% ./start-jupyter --help
Usage: start-jupyter [OPTION]
Available values for OPTION include:
-A acctcode start a job with acctcode
-c command invoke jupyter with command
-d check for presence of jupyter
-i file source file prior to running jupyter
-n ntasks request ntasks tasks for job
-p partition run job on partition
-t time job runtime
-u username use username for login
What is missing?
The script does not support singularity, it just looks for the jupyter
binary in the Mistral host and use it to start the notebook.
In order to use another path (e.g. from a container), this must be specified. So, the idea is simple: instead of using the remote host jupyter
,
I will use jupyter
from the singularity container. This page explains how to use Singularity containers on Mistral.
The solution
I modified the original script to support singularity containers. Now, if you specify a singularity image with the -s
flag, the script will use it to start the notebook.
% ./start-jupyter --help
Usage: start-jupyter [OPTION]
Available values for OPTION include:
-A acctcode start a job with acctcode
...
-u username use username for login
-s image singularity image
The script needs the Singularity module
loaded, so I just use the jupyter_preload
file for this purpose.
module load python3/unstable
module load singularity
All I need is a working image with jupyter
installed, so I did create it and push it to the hub.
The image must be available in the HOME
directory.
This is how I call the script:
./start-jupyter -u xxxxxxx -i jupyter_preload -s jupyter_mistral_latest.sif
The new script
The modified script can be found here. These modifications can be merged with the original script later.
Next steps
- download/pull the image if not available
- create further customized images
Note
Depending on your workflow and the required softwares/packages, additional steps might be necessary, thus the jupyter_preload
needs to be adapted accordingly.