BRC Cluster Workshop

These notes apply to JupyterLab 1.0 run in a Singularity container.

JupyterLab is a web-based interactive development environment for Jupyter notebooks, code, and data. JupyterLab is flexible: configure and arrange the user interface to support a wide range of workflows in data science, scientific computing, and machine learning.

JupyterLab web page: https://jupyter.org/

Once you have created a Singularity container with JupyterLab installed the following code can be run on the head node of the Cluster. You will need to tunnel into your JupyterLab instance therefore I recommend that Windows users installing MobaXterm (https://mobaxterm.mobatek.net/). Follow the tunneling instructions for either MacOS / linux or Windows.

Note that your Singularity container will run on node91 but this can be changed to any of the Cluster nodes but you will need to change -p (i.e. standard) and -w (i.e. node10) arguments.

Once finished shutdown JupyterLab and exit your job to free up used resources.

#Running jupyter in a Singularity container on captainkirk

srun -p gpu -w node91 -J jupGAN -c 1 --pty --mem=4000 bash

# get tunneling info
XDG_RUNTIME_DIR=""
port=$(shuf -i8000-9999 -n1)
# GPU node
node=192.168.1.191
user=$(whoami)
cluster=captainkirk.statgen.ncsu.edu

# print tunneling instructions jupyter-log
echo -e "

MacOS or linux terminal command to create your ssh tunnel
ssh -N -L ${port}:${node}:${port} ${user}@${cluster}

Windows MobaXterm info for Tools->ModaSSHTunnel
Forwarded port: ${port}
Remote server: ${node}
Remote port: ${port}
SSH server: ${cluster}
SSH login: ${user}
SSH port: 22

Use a Browser on your local machine to go to:
http://localhost:${port}  (prefix w/ https:// if using password)

DON'T USE ADDRESS BELOW.
DO USE TOKEN BELOW

"

# Load singularity container with tensorflow2, GPU support, Python3, Jupyter lab
singularity exec --nv ~/images/tensorflow2-0-gpu-py3-jupyterNL.sif jupyter lab --no-browser --port=${port} --ip=${node}