This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
update_2021_overview [2021/06/29 14:39] root |
update_2021_overview [2021/12/14 17:18] (current) root |
||
---|---|---|---|
Line 1: | Line 1: | ||
===== Update 2021 Overview ===== | ===== Update 2021 Overview ===== | ||
- | The cluster is being updated to Ubuntu 20.04. Along with the OS update comes several changes to the way the cluster works. This page summarizes those changes. Workshops | + | The cluster is being updated to Ubuntu 20.04. Along with the OS update comes several changes to the way the cluster works. This page summarizes those changes. Workshops |
Rather than shutting the cluster down and doing the upgrade in one swell foop, we are providing a new cluster (with a new login node) already upgraded to Ubuntu 20.04 "Focal Fossa" | Rather than shutting the cluster down and doing the upgrade in one swell foop, we are providing a new cluster (with a new login node) already upgraded to Ubuntu 20.04 "Focal Fossa" | ||
- | The new cluster | + | ** From September 6th no new jobs will be allowed on the old cluster. Jobs which are already running on that date will be allowed to continue running, but the old cluster now has a 30-day time limit on all queues. ** |
+ | |||
+ | Most of the old cluster nodes have now been moved to the new cluster. The new cluster includes | ||
Line 34: | Line 36: | ||
So, you should be more careful in specifying how many cores your job needs. See the " | So, you should be more careful in specifying how many cores your job needs. See the " | ||
+ | |||
+ | See [[Enforcing Core Counts]] for more details. | ||
=== Enforcing Memory Allocation === | === Enforcing Memory Allocation === | ||
Line 44: | Line 48: | ||
You can change the amount of memory your job is allocated using either the " | You can change the amount of memory your job is allocated using either the " | ||
- | If you have previously run jobs on the old cluster, you can find out how much memory they used using the **sacct** command. You can either find the job id of a completed job on the old cluster (perhaps from the slurm-NNNNN.out file name) and then run a command like this: | + | See [[Enforcing Memory Allocation]] |
- | + | ||
- | < | + | |
- | sacct -j NNNNN -o elapsed, | + | |
- | </ | + | |
- | + | ||
- | Or you can use the job name (which defaults to the name of the script you submitted) and get a list of all jobs with the same name: | + | |
- | + | ||
- | < | + | |
- | sacct --user=chris -- -S 2020-01-01 --name=myscript.bash --format=JobID, | + | |
- | </ | + | |
- | + | ||
- | The output of this command could be parsed to get the maximum amount of memory used by this particular type of job). (If you use the same script name for multiple different jobs, this will mix them up.) | + | |
==== Environment Modules ==== | ==== Environment Modules ==== | ||
- | Environment modules allow you to control which software (and which version of that software) is available in your environment. For instance the new cluster has 4 different | + | Environment modules allow you to control which software (and which version of that software) is available in your environment. For instance the new cluster has 4 different |
, 3.6.3, 4.0.5, 4.1.0. When you first log in and try to run R the OS will respond with " | , 3.6.3, 4.0.5, 4.1.0. When you first log in and try to run R the OS will respond with " | ||
Line 67: | Line 59: | ||
</ | </ | ||
- | That would then give you access to the most recent version of R available (4.1.0 in this case). | + | See [[Environment Modules]] |
- | + | ||
- | To use a different version you would have typed something like: | + | |
- | + | ||
- | < | + | |
- | module add R/3.6.3 | + | |
- | </ | + | |
- | + | ||
- | To get a list of all available software you can type: | + | |
- | + | ||
- | < | + | |
- | module avail | + | |
- | </ | + | |
- | + | ||
- | To get a full list of module commands: | + | |
- | + | ||
- | < | + | |
- | module --help | + | |
- | </ | + | |
- | + | ||
- | There' | + | |
- | + | ||
- | < | + | |
- | ml fastp | + | |
- | </ | + | |
- | + | ||
- | to, for instance, load the fastp program. | + | |
- | + | ||
- | You can also issue the other module commands using ml: | + | |
- | + | ||
- | < | + | |
- | ml avail | + | |
- | ml list | + | |
- | ml purge | + | |
- | ... | + | |
- | </ | + | |
- | + | ||
- | More information about environment modules can be found here: https:// | + | |
==== Connecting to the Updated Cluster ==== | ==== Connecting to the Updated Cluster ==== |