Differences

This shows you the differences between two versions of the page.

--- connecting [2021/10/19 10:31]
root
+++ connecting [2022/01/26 09:48] (current)
root
@@ Line 9: / Line 9: @@
   * brccluster2021.statgen.ncsu.edu
-** From October 18th 2021 OIT are blocking the default SSH port (port 22) at the campus gateway. You should  connect to the campus VPN and then use SSH to get to the cluster. SFTP and SCP are also affected by this change. **
+These are just different names for the same machine. This machine is known as **sisko** within the cluster.
-** Alternatively, you can SSH to port 222 on the cluster. This requires you setting your SSH client to use port 222 (the "-p" option on command-line ssh). For scp the option is "-P" (uppercase). **
+** From October 18th 2021 OIT blocked the default SSH port (port 22) at the campus gateway. You should  connect to the campus VPN and then use SSH to get to the cluster. SFTP, SCP, and SSHFS are also affected by this change. **
-These are just different names for the same machine. This machine is known as **sisko** within the new cluster. Please read the notes on the new cluster here: [[Update 2021 Overview]] before really using it. And perhaps attend (or watch a recording of) a class on the differences between the old and new clusters.
+** Alternatively, you can connect to port 222 on the cluster. This requires you setting your SSH client to use port 222 (the "-p" option on command-line ssh). For command-line scp and sftp the option is "-P" (uppercase). **
 Under Windows you can use [[http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html|putty]] or some other ssh client. e.g. ssh comes with the MinGW shell, openssh for windows, MobaXterm, or Windows Subsystem for Linux.
@@ Line 27: / Line 27: @@
 You will be prompted for your password. When you type your password, no characters will be displayed as you type (not even asterisks or dots).
+==== Keeping Jobs Running ====
+If you have started a long-running job on the head node from your command line, or are running a job using **srun**, then a network glitch, or forgetting and closing your laptop lid, will drop your connection and kill your job.
+It is best to run jobs using **sbatch** which detaches the job from the shell, but you can also use a **terminal multiplexer**. The cluster has 3 such programs installed: **screen**, **byobu**, and **tmux**. These programs start up shells that are immune to your network connection to the cluster being dropped (as well as letting you run multiple shells from the one connection). If your network connection drops, you can log in again and re-attach to the shells running under the terminal multiplexer.
+  * Screen: https://www.gnu.org/software/screen/
+  * Byobu: https://www.byobu.org/
+  * Tmux: man tmux
+Finally, you can use the bash built-in **disown** command to detach a job from your terminal.
+<code>
+./mycoolprogram > coolprog.out 2>&1 &
+disown
+</code>
+These commands start your program with output redirected to a file, put it in the background, and then tell bash to detach the program from your terminal.
 ==== Copying Files ====
@@ Line 60: / Line 79: @@
 **sshfs**
   * Allows you to mount a cluster directory as a subdirectory on your local machine.
-  * https://osxfuse.github.io/ (for Mac)
+  * https://osxfuse.github.io/ (for Mac, see note below about permissions)
   * win-sshfs (for Windows)
+  * sshfs-win (for Windows)
   * [[https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh|Using sshfs]]
+If you are using sshfs on a Mac and find that you cannot access directories (folders) through sshfs that you can access on the cluster because of supplementary groups that you user belongs to, then you may want to try this option:
+<code>
+sshfs -o defer_permissions username@brccluster.cos.ncsu.edu:/directory mountpoint
+</code>
+**Globus**
+You can use Globus Connect Personal to create a Globus endpoint allowing you to copy files to or from the cluster using the Globus interface. The instructions for doing so are here: https://docs.globus.org/how-to/globus-connect-personal-linux. You should use the "Running with no GUI" instructions, or see below.
+Globus Connect Personal is available as a module on the cluster (with name "globuspersonal").
+The first time you run **globusconnectpersonal** you will be given a URL (to auth.globus.org) and a prompt for an auth code. Visit the URL on your local machine and log in to Globus to get the authorization code. Copy and paste the auth code into the cluster terminal window.
+You can then start globus with:
+<code>
+globusconnectpersonal -start &
+</code>
+You should then be able to find your cluster endpoint within your Globus account.
+Check status of your Globus connection with:
+<code>
+globusconnectpersonal -status
+</code>
+End your Globus session with:
+<code>
+globusconnectpersonal -stop
+</code>
+If you want to add directories outside your home directory to your Globus endpoint, take a look at the bottom of the page pointed to by the URL above where the format of the **config-paths** file is described.
 ==== Downloading Files ====

BRC Cluster Workshop

User Tools

Site Tools

Differences

Page Tools