Installing Programs

Two possibilities:

• Installed globally (by an administrator)
• Installed locally (in a user’s home directory)

A lot of widely-used programs have been installed globally (BLAST, tophat, bowtie, samtools, …)

• From time to time we upgrade.
• You may want to stick with a specific version.
• Sometimes we are a little behind. If you need the latest version you can ask me to upgrade. I usually check with people I know use the program to see if they are OK with upgrading.

Since we have moved to using Environment Modules on the cluster, the point about versions is less valid. Once a version of a piece of software is installed as a module it will not be removed, and will therefore remain available to you.

You are encouraged to install the software that you need into your own home directory (or possibly into a shared directory for the project you are working on). This gives you control of the versions of the software you are using, and when upgrades are made. This is good for reproducibility of results. You can use Conda to install specific versions of Python or R, and associated packages into your own home directory. Conda can also install many bioinformatics programs (e.g. STAR, bwa etc.) through the bioconda channel. You can also look at using a (Singularity) container to encapsulate a project or software pipeline - see Using Singularity.

You can install miniconda by downloading the installation script here: https://docs.conda.io/en/latest/miniconda.html.

apt-get won’t do it.

• dpkg apparently can (but doesn’t resolve dependencies).
• Exact commands depend on what you are installing.

Possibilities:

• Install binaries (if available). 64-bit preferred.
  • Specify the full path to the executable.
  • Update your path to include the binaries (in your .bashrc and .bash_profile files).
  • Maybe have a bin directory in your home directory.
• Install from source.
• Use dpkg and resolve dependencies yourself.

Since your home directories are available on all nodes, any program you install locally will automatically be available to you on the compute nodes. (This is not always true with apt-get installations, as they sometimes go into /usr/bin which is not shared across nodes.)

This can be quite simple, or a nightmare of dependencies.

• Read the README or INSTALL file.
• Often it's one of these two basic techniques.

cd source_dir
make

This first style will build the application in the current directory. You can then run it from where it is, or copy it to a directory from where you usually run locally installed programs (a location that is in your PATH).

./configure --prefix=$HOME
make install

This second style will install the program into $HOME/bin. You can also give a prefix that points to a subdirectory of your home directory, e.g. $HOME/local. The program would then be installed to $HOME/local/bin. You can add this directory to your PATH for ease of running the program.

If you do not specify the “–prefix” argument on the configure command the make install step will likely try to copy the program into /usr/local/bin. You do not have permission to write into /usr/local as a normal user.

apt-get source package-name
tar -zxf package-name.tar.gz
cd package-dir
./configure -prefix=$HOME
make install

• R packages
• Python modules
• Perl modules

• Sometimes R (or Python, or Perl) packages depend on software libraries that are not other R (or Python, or Perl) packages.
  • In this case you can send e-mail to the system administrators to get the dependency installed using the OS package manager, or possibly from a direct download and install.
  • If you have problems with an install you can ask send a help request to help@sciences.ncsu.edu). Send e-mail describing what you have tried and what happened.

You can install conda in your own home directory and then use it to install a specific version of python or R (for instance).

For instance, to install R 3.6.1 (currently):

conda create -n r-test r-base r-tidyverse r-rlang
...
conda activate r-test

To install R 4.0.2 inside a conda environment you can use the command:

conda install -c conda-forge r-base=4.0.2

Using a Singularity container allows you to have complete control over your environment and what is installed, and allows you to share that environment with others, but does require a little more effort than other installation techniques.

See the page on Using Singularity.