This tutorial should teach the relatively web and unix-friendly user how to set
up a wagon server.

Suppose I want to run a wagon server to distribute jobs in the illustrious
"produce the prime factorizations of all natural numbers from 1 to 9999"
project.

I begin by obtaining an account on a unix machine somewhere and building or
downloading the wagon binaries. this machine we will refer to as "www.foo.net",
and my account is called "eduardo", so the canonical URL for my home page is
"http://www.foo.net/~eduardo". if you do not understand what I'm talking about
so far, this tutorial is over your head, sorry. it won't get easier.

then I make a directory in which I am allowed to execute CGIs. if I can't do
this, I am screwed, so I phone my sysadmin and beg and plead until I get this
capability. this directory we will refer to as "~/public_html/cgi-bin". so the
canonical URL for my CGI directory is "http://www.foo.net/~eduardo/cgi-bin". 
again, if you are lost, go familliarize yourself with normal webserver setup
before proceeding any further.

I find out that my server runs programs as user "nobody" and I don't want to
bother the admin any more than I already have. so I opt for running the wagon
CGIs as setuid me, as described in the README file, and I install the following
programs into cgi-bin and set their modes accordingly:

$ install -m 4755 wagon-accept wagon-dispatch wagon-collect wagon-view \
  wagon-delete ~/public_html/cgi-bin

then I write the canonical URL of the CGI directory to the cfg/url file, so that
wagon can know where it's located when linking back to itself:

$ cd ~/public_html/cgi-bin
$ mkdir cfg
$ echo -n "http://www.foo.net/~eduardo/cgi-bin" >cfg/url

since this is going to be a somewhat public server, I'd like people to not flood
it with total crap and deny service. so the next stage of the configuration is
to teach wagon how to _throw out bogus junk_. I do this in 2 ways: with a byte
limit, and with verifiers. first, the byte limit: I choose a number of bytes
which is larger than any file I'd ever like wagon to even consider dealing
with. in this case, I choose the number 10000, because it is unrealistically
large for a sequence of bytes representing a number from 1 to 9999 or a
factorization thereof. I write this value to the configuration file
cfg/maxbytes:

$ echo -n "10000" >cfg/maxbytes

good, now people sending large files will be silently ignored. next up,
verifiers.  to verify data wagon will be dealing with, I write two small shell
scripts: one which verifies incoming jobs, and one which verifies uploaded
results. the first one verifies that an incoming job really contains a natural
number between 1 and 9999, and nothing else. to verify this fact, I will extract
the job from its container (a CDB file) and feed it into wc -l to confirm that
it consists of only 1 line, then feed it into grep to confirm that that line
contains only a sequence of between 1 and 4 digits:

#!/bin/sh
JOB=$1
(let "$(cdbget job <$JOB | wc -l)==1") && \
   (cdbget job <$JOB | egrep -q '^[0-9]{1,4}$') && exit 0;
exit 1;

the second program will confirm that the results of the computation match a
factorization of the natural number in question, as output by the famous unix
tool "factor". it will do this (perhaps more pedantically than you will choose
to) by checking that the results look like the output from factor, then by
opening the original job and extracting _its_ value, and checking that the
result is a sequence of numbers that multiply together to make the original
value. in your scripts, you might not care so much, but factoring is terribly
precise work:

#!/bin/sh
RESULTS=$1
JOB=$2
(let "$(cdbget job <$RESULTS | wc -l)==1") && \
   (egrep -q '^[0-9]{1,4}:( [0-9]{1,4})+$' $RESULTS) && \
   (let "$(cdbget job <$JOB)==$(cut -d: -f1 $RESULTS)") && \
   (let "$(cut -d: -f1 $RESULTS)==$(echo 1 1 `cut -d:\
	 -f2 $RESULTS | tr ' ' '*'` '*p'|dc)") && exit 0;
exit 1;


note: neither of these programs needs to be at all complex. they must simply
return 0 or 1 to indicate that you are satisfied or dissatisfied, respectively,
with the jobs or results being offered. if you have a loose security policy, or
trust your users, you can simply set both these programs to be a shell script
which always returns 0.

At any rate, I will deposit these two programs in jv/factor-job-verify and
rv/factor-result-verify, and make them executable. and this will complete the
server-side configuration of my wagon server. I should test it out now by
sending some numbers into it and seeing if they show up in the new/ directory
managed by wagon-accept:

$ ... edit the scripts in an editor ...
$ chmod 0755 jv/factor-job-verify rv/factor-result-verify
$ echo "3953" >number
$ wagon-push $(< cfg/url) $(wagon-build 'test job' 'factor' number)
$ ls new/
399352ba-www-foo-net-4564-1
$ 

hooray! my wagon server is working. how lovely. we are now in the third and
final phase of setting up my work: customizing the client to perform
factorizations. wagon does most of the work. all I must do is provide it with a
small shell script called factor-proxy which acts as a wrapper, or proxy, to the
factor program. note that it is called "factor-proxy" because it is a proxy for
jobs of type "factor". the fact that it calls "factor" inside itself is unknown
to wagon. wagon only understands that the job type is "factor" so it should run
"factor-proxy". the factor-proxy script will be called with 2 arguments when it
is run, on the client side: the name of the job input file, and the name of the
output file it should write results to. all setup and cleanup is done by wagon,
so the script is really simple:

#!/bin/sh
factor $(cdbget job <$1) >$2

that's it. we then make a little tarball for potential clients, containing a
timeout value (cfg/pause), our URL, this proxy, and the appropriate client
binaries (including factor, for those who don't have it)

$ mkdir prx
$ ... edit prx/factor-proxy ...
$ chmod 0755 prx/factor-proxy
$ mkdir todo scr fin
$ echo -n '150' >cfg/pause
$ tar -czf factoring-client.tgz prx todo scr fin cfg/url cfg/pause \
  $(which factor) wagon-client wagon-pull wagon-push 

now my clients can just untar that distribution and run ./wagon-client, and it
will fetch numbers from my server every 150 seconds, factor them, and submit the
results. that is all there is to it.