User Tools
enforcing_memory_allocation
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
enforcing_memory_allocation [2021/06/29 14:50] root |
enforcing_memory_allocation [2022/01/26 10:04] (current) root |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ===== Enforcing Memory Allocation ===== | ===== Enforcing Memory Allocation ===== | ||
| - | On the old cluster | + | On the cluster, |
| - | + | ||
| - | On the new cluster, memory is a SLURM consumable resource. That means that you should be careful to specify how much memory your job needs. By default your job will be allocated 8GB per core that it has requested. By default 1 core is allocated, in which case 8GB of memory would be allocated to your job. If your job attempts to use more than its memory allocation **it will be killed by the operating system**. | + | |
| You can change the amount of memory your job is allocated using either the “–mem” option, or the “–mem-per-cpu” option on sbatch. | You can change the amount of memory your job is allocated using either the “–mem” option, or the “–mem-per-cpu” option on sbatch. | ||
| Line 17: | Line 15: | ||
| </ | </ | ||
| - | If you have previously run jobs on the old cluster, you can find out how much memory they used by using the **sacct** command. You can either find the job id of a completed job on the old cluster (perhaps from the slurm-NNNNNN.out file name) and then run a command like this: | + | You can request all memory on a node (whatever amount that is) using: |
| + | |||
| + | < | ||
| + | sbatch --mem=0 my_script.bash | ||
| + | </ | ||
| + | |||
| + | If you have no idea how much memory your job will use, but are convinced that it will use more than the default amount (8GB per core allocated to the job), then you should run at least one test case using the **--exclusive** option and the " | ||
| + | |||
| + | If you have previously run jobs on the old cluster | ||
| < | < | ||
| Line 26: | Line 32: | ||
| < | < | ||
| - | sacct --user=chris | + | sacct --user=chris -S 2020-01-01 --name=myscript.bash --format=JobID, |
| </ | </ | ||
| The output of this command could be parsed to get the maximum amount of memory used by this particular type of job). (If you use the same script name for multiple different jobs, this will mix them up.) | The output of this command could be parsed to get the maximum amount of memory used by this particular type of job). (If you use the same script name for multiple different jobs, this will mix them up.) | ||
| - | If you have no idea how much memory your job will use, but are convinced that it will use more than the default amount (8GB per core allocated to the job), then you should run at least one test case using the **--exclusive** option to **sbatch** which allocates you a whole node. When you job completes you can then use the **sacct** command to find out how much memory it used. | ||
enforcing_memory_allocation.1624992638.txt.gz · Last modified: 2021/06/29 14:50 by root
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Public Domain
