The following are some scripts that may be of use in running multiple jobs on the cluster. Your kilometreageTM may vary.
These scripts are at /cluster/scripts on this site.
Script in /home/l/lha/bin/sort-hosts. Currently run every 5 minutes by a cron job (as lha).
-c option creates the file /scratch-emu/lha/sort-hosts-cache listing machines in the cluster. The machines are ordered with the first being the "best" host to run a job on (considering the machine's speed and current load).
Script in /home/l/lha/bin/spawn.
Spawn jobs on multiple machines Usage: spawn init
... spawn nexthost spawn launch spawn any spawn done Command init gets ready to run jobs on the listed hosts. If one is already busy, the command fails Command nexthost finds a free machine Command launch runs command in the background on host Command any runs command in the background on a free machine Command done waits until all jobs are finished, to allow postprocessing, and tidies up Suggested use: Write a program to generate one line of output, starting with the parameters. Then run a script such as spawn init edison marconi morse nyquist bell for parameter in values; do host=`spawn nexthost` echo $parameter > config.$host spawn launch $host "simulator config.$host >> temp" done spawn done sort -n < temp > output
This script uses a directory $HOME/.spawn to keep track of which machines you are currently running jobs on. Files "busy.hostname" contain the command that is currently running on hostname, and "idle.hostname" contain the most recent command.