Overview

Accessing JupyterHub

CSaW’s JupyterHub instances are reachable from on campus or via the VPN when at home.

The CSE JupyterHub instance can be reached by visiting:

The CSCI JupyterHub instance can be reached by visiting:

Implementation

CSaW’s JupyterHub is setup using Apptainer containers that are scheduled to run on a node in the cluster via HTCondor. The web interface makes it easier for researchers to request and access resources from their browser instead of the traditional command line, though still provides access to the command when needed.

Due to the nature of running in a container, there will be differences when compared to submitting a traditional HTCondor job and running directly on a node.

Limitations

Currently there are limits placed on all JupyterLab instances that are started in the cluster. These include:

  • Maximum run-time of ~24 hours

  • Maximum idle time of one (1) hour

  • Maximum of 8 cores

  • Maximum of 32GB Memory

  • Maximum of 4 GPUs

If you need more resources or run-time than this, your notebook should be converted to a traditional program that can then be scheduled to run in the cluster.

If you have exceeded the run-time or idle limit and return to your JupyterLab page and see the error that it’s not running, you can click the button to restart it.

Requesting resources

  • Just like normal cluster usage, ask for only what you need and intend to use

  • Asking for more cpus does not make your code magically run faster. You must write code to utilize the additional cpus.

  • Asking for a gpu does not mean your code will use it. You must write code to utilize the gpu.

    • The CSE cluster does not have GPUs available to all by default. Those who are in research groups that have purchased GPU nodes can schedule on them, but those who are not and wish to use a GPU will need to accept preemption.

  • Estimate Memory usage for you job based on the amount of data you plan to load at any given time.

    • Memory usage outside of the JupyterLab environment is not tracked in the monitor at the bottom of the page, but is tracked by HTCondor

      • This includes applications launched in the terminal tab(s) and remote desktop

It may not be possible for HTCondor to schedule your request due to the resources being unavailable.

Basic JupyterLab usage

Please see the JupyterLab documentation for basic usage of JupyterLab. Not all functionality described there may work due to the nature of how JupyterLab is run in the cluster environment.

Stopping the JupyterLab server

File ‣ Hub Control Panel once the new tab loads click the Stop My Server button to stop the server and free the resources that HTCondor had allocated to you so the next person can use them. Logging out does not stop your server from running, which means you are still consuming cluster resources in the background. Please be sure to stop your server when you’ve finished your tasks.