ParallelNotes - Methods
location: ParallelNotes

Alex and Matthew playing with python parallel programming

Ipython1 - Docs therefore -

Then we copied the sshx script from the ipython1 distro into /imaging/local/linux/bin

Shell setup

The engines use the ~/.bashrc to initialize environment variables for the shell (this does not happen by default because sh is used, which does not read startup scripts). The problem being that Alex is set up with the usual default tcsh, which uses ~/.cshrc.

Accordingly, we added these lines to ~/bashrc:

# FSL stuff - if we are using FSL
export FSLDIR=/imaging/local/linux/bin/fsl-4.0.1
. ${FSLDIR}/etc/fslconf/
export PATH=${FSLDIR}/bin:$PATH

# If on linux, set stuff for libraries, and python
if [ "`/bin/uname`" = "Linux" ]; then
    export PATH=${cbu_img_sys}/bin:$PATH
    export LD_LIBRARY_PATH=${cbu_img_sys}/lib

Cluster setup

We made a tiny script to test with:

   1 controller = dict(host='l42',
   2                   engine_port=None, # default is 10105
   3                   control_port=None,
   4                   )
   6 # keys are hostnames, values are the number of engines on that host
   7 engines = dict(l31=2,
   8                l32=2,
   9                l33=2,
  10                l34=2,
  11                )

Then we ran ipcluster -f

and got this:

Starting controller:
Starting engines:   
Sarting 2 engines on l34
Sarting 2 engines on l32
Sarting 2 engines on l33
Sarting 2 engines on l31

Your cluster is up and running.

For interactive use, you can make a MultiEngineClient with:

from ipython1.kernel import client
mec = client.MultiEngineClient(('l42',10105))

You can then cleanly stop the cluster from IPython using:


None: ParallelNotes (last edited 2013-03-08 10:28:27 by localhost)