Difference between revisions of "Cluster-Software"

From RedwoodCenter
Jump to navigationJump to search
 
(2 intermediate revisions by the same user not shown)
Line 52: Line 52:
 
== CUDA ==
 
== CUDA ==
  
CUDA is a library to use the graphics processing units (GPU) on the graphics card for general-purpose computing. We have a separate wiki page to collect information on how to do general-purpose computing on the GPU: [[GPGPU]].
+
CUDA is a library to use the graphics processing units (GPU) on the graphics card for general-purpose computing.
The --constraint={cortex_k40, cortex_fermi} option must be used in order to schedule a node with a GPU.
+
The  
We have installed the CUDA 6.5 driver and toolkit.
+
 
 +
  #SBATCH--constraint={cortex_k40, cortex_fermi}  
 +
  #SBATCH --gres=gpu:1
 +
 
 +
options must be used in order to schedule a node with a GPU.
  
 
In order to use CUDA, you have to load the CUDA environment:
 
In order to use CUDA, you have to load the CUDA environment:
Line 65: Line 69:
 
   module unload intel
 
   module unload intel
  
Theano caches certain compiled libraries and these will sometimes cause errors when Theano gets updated. If you are experiencing problems with Theano, you can try clearing the cache with
+
Theano caches certain compiled functions and these will sometimes cause errors when Theano or CUDA gets updated. If you are experiencing problems with Theano, you can try clearing the cache with
 
   theano-cache clear
 
   theano-cache clear
 
and if you still have problems you can delete the .theano folder from your home directory.
 
and if you still have problems you can delete the .theano folder from your home directory.
Line 77: Line 81:
  
 
   [global]
 
   [global]
  root = /global/software/sl-6.x86_64/modules/langs/cuda/6.5/
 
 
   device = gpu
 
   device = gpu
 
   floatX = float32
 
   floatX = float32
Line 87: Line 90:
 
==== Using the CPU ====
 
==== Using the CPU ====
  
Theano can also run on the CPU. Any of the CPU nodes will work. You will want to have Theano build against the MKL BLAS library that comes with Anaconda and so your .theanorc might look like
+
Theano can also run on the CPU. Any of the CPU nodes will work. You will want to have Theano use the MKL BLAS library that comes with Anaconda and so your .theanorc might look like
  
 
   [global]
 
   [global]

Latest revision as of 17:52, 12 October 2016

Software

Matlab

In order to use matlab, you have to load the matlab environment:

 module load matlab/R2013a

Once the matlab environment is loaded, you can start a matlab session by running

 matlab -nodesktop

An example SLURM script for running matlab code is

 #!/bin/bash -l
 #SBATCH -p cortex
 #SBATCH --time=03:30:00
 #SBATCH --mem-per-cpu=2G
 module load matlab/R2013a
 matlab -nodesktop -r "scriptname. $variable1 $variable2"

The above script takes a matlab job with scriptname = scriptname and accepts two variables $variable1 and $variable2

If you would like to see who is using matlab licenses, enter

 lmstat

Python

Anaconda Python Distribution

The Anaconda Python 2.7 or 3.4 Distributions can be loaded through

 module load python/anaconda2

or

 module load python/anaconda3

respectively. This distribution has NumPy and SciPy built against the Intel MKL BLAS library (multicore BLAS). You will need to get an academic license from Continuum and copy it to the cluster.

On the cluster

 cd
 mkdir .continuum

On the machine where you downloaded the license file

 scp file_name username@hpc.brc.berkeley.edu:/global/home/users/username/.continuum/.

Local Install of Anaconda Python Distribution

If you want to manage your own python distribution the Anaconda Python is a very good distribution. To get it, go the the Continuum downloads page and select the linux distribution (penguin). Copy the download link address, and then in a terminal on the cluster run:

 wget paste_link_here

This should download a .sh file that can be run with

 bash Anaconda-version_info.sh

CUDA

CUDA is a library to use the graphics processing units (GPU) on the graphics card for general-purpose computing. The

 #SBATCH--constraint={cortex_k40, cortex_fermi} 
 #SBATCH --gres=gpu:1

options must be used in order to schedule a node with a GPU.

In order to use CUDA, you have to load the CUDA environment:

 module load cuda

Using Theano

By default, Theano expects the default compiler to be gcc, so you'll need to unload the intel compiler.

 module unload intel

Theano caches certain compiled functions and these will sometimes cause errors when Theano or CUDA gets updated. If you are experiencing problems with Theano, you can try clearing the cache with

 theano-cache clear

and if you still have problems you can delete the .theano folder from your home directory.

Using the GPU

You must request a GPU node. The Anaconda Python distribution comes with a version of Theano that should work. If you need new Theano features, the development version of Theano can be obtained from the github repository, installed locally, and added to your PYTHONPATH if you are using the preinstalled Python verions. If you have a local python install you can install theano with

 python setup.py develop

from the repository folder. Theano must be configured to use the GPU. General information can be found in the Theano documentation, but a working (June 2015) version is to create a .theanorc file in your HOME directory with the contents:

 [global]
 device = gpu
 floatX = float32
 force_device=True
 
 [nvcc]
 fastmath = True

Using the CPU

Theano can also run on the CPU. Any of the CPU nodes will work. You will want to have Theano use the MKL BLAS library that comes with Anaconda and so your .theanorc might look like

 [global]
 device = cpu
 floatX = float32
 ldflags = -lmkl_rt