Useful scripts

The following are some scripts that may be of use in running multiple jobs on the cluster. Your kilometreage^TM may vary.

These scripts are at /cluster/scripts on this site.

sort-hosts

Script in /home/l/lha/bin/sort-hosts. Currently run every 5 minutes by a cron job (as lha).

-c option creates the file /scratch-emu/lha/sort-hosts-cache listing machines in the cluster. The machines are ordered with the first being the "best" host to run a job on (considering the machine's speed and current load).

spawn

Script in /home/l/lha/bin/spawn.

Spawn jobs on multiple machines
Usage:
   spawn init   ... 
   spawn nexthost
   spawn launch  
   spawn any 
   spawn done

Command  init   gets ready to run jobs on the listed hosts.  If one is
   already busy, the command fails
Command  nexthost  finds a free machine
Command  launch   runs  command  in the background on  host
Command  any   runs  command  in the background on a free machine
Command  done  waits until all jobs are finished, to allow postprocessing,
   and tidies up

Suggested use:
Write a program to generate one line of output, starting with the parameters.
Then run a script such as

    spawn init edison marconi morse nyquist bell
    for parameter in values; do
	host=`spawn nexthost`
	echo $parameter > config.$host
        spawn launch $host "simulator config.$host >> temp"
    done
    spawn done
    sort -n < temp > output
      

This script uses a directory $HOME/.spawn to keep track of which machines you are currently running jobs on. Files "busy.hostname" contain the command that is currently running on hostname, and "idle.hostname" contain the most recent command.