Hey everyone,
As many people have noticed, our world does not build with distcc. This
is no longer the case!
In discussing the issue with Simon and J顤g, it became apparent that
distcc was not equipt with the functionality that our build system
needs. Namely, when we bootstrap our GCC, we rely on the OBJFORMAT_PATH
variable in several places to tell objformat which CC to use.
The fix? Allow distcc to export environment variables.
The quirks? You're required to share /usr/obj via NFS when you want to
use distcc for building the world.
The result? I'm somewhat disappointed, actually. I remember doing a
buildworld on Leaf with no -j flags (it's not allowed) and it finished
in just under fifteen minutes. On DFCluster (my 7-blade P3 800 cluster)
it takes 1 hour and 10 minutes. There are a couple of possible
bottlenecks that I faced here:
1) I was running this under script buildworld.out, so that I could
determine where things were going wrong when / if they went wrong,
2) DFCluster uses 4200RPM hard drives (please, if you want to donate an
old SCSI fibre channel RAID array, I'd welcome it!)
3) I'm not utilizing any of the VFS improvements that have been made,
this was on a 1.0A system.
4) 2 blades are not functioning due to a botched upgrade on them.
5) I know nothing about NFS tuning, I started up portmap, mountd and
nfsd with no special options, no kernel hacks and a fairly standard
exports line:
usr -alldirs -mapall=root -network 192.168.1 -mask 255.255.255.0
which then had /usr/obj mounted on all of the blades.
Distcc was run as a handler for cc and c++. My buildworld line was:
make -j 20 CC=/usr/local/bin/distcc C++=/usr/local/bin/distcc buildworld
The world I built was yesterdays DragonFly_Stable tagged world.
So, overall, with 5 active blades (4000MHz of processor power and 4.5GB
RAM overall shared between them), I suppose I should be happy. I'll be
doing more testing in the future with it, now that it works. Namely, I'd
like to see how long it takes to build the world on _one_ blade, so that
we can get an idea what kind of improvement it provides.
Where do you get it? I've got a tarball for inclusion in dfports at:
http://www.the-bofh.org/projects/distcc-env/distcc.tar.gz
Have fun!
Devon H. O'Dell