Hack 88 Distribute Compiles with distccKNOPPIX

This Damn Small Linux-based distribution is aimed at making it easy to set up a cluster of machines running distccd, which enables the user to leverage multiple machines for compiling.

I was first introduced to distcc while compiling Debian unofficial KDE CVS packages. Distcc is a program that distributes builds of C, C++, Objective C or Objective C++ code across several machines on a network. I quickly installed distcc on my networked Linux systems and was happily using up unused cycles and speeding up my builds a great deal. However, the one Microsoft Windows system sitting idle as my compiles continued gave me an itch to investigate the possibility of using distcc on a Windows system. Enter distccKNOPPIX.

distccKNOPPIX is a remastered live CD based on Damn Small Linux (DSL). It runs a distcc daemon as its sole task and is as portable as any Knoppix CD. By taking advantage of some of the kernel options of Knoppix, such as toram and tohd [Hack #5], you can have an entire network handle the distributed compile and still be able to reboot the system to its regular settings when finished.

To run distccKNOPPIX, you just need to download the ISO from http://opendoorsoftware.com, burn it to CD, insert it in a bootable CD-ROM drive, and reboot. Once booted, confirm that your network is set up correctly as listed in [Hack #17], and note that your IP is using ifconfig. When you are ready to compile a program on the machine running distccd, add this IP and any other IPs for distcc machines on your network to the DISTCC_HOSTS environment variable:

knoppix@ttyp0[knoppix]$ export DISTCC_HOSTS="localhost 192.168.0.1 192.168.0.2'

Alternatively, you can run a program like nmap from your server to detect new IPs in your subnet. Look for open port number 3632.

root@ttyp0[root]# nmap -v -p 3632 192.168.0.0/16

This comes in handy if the distccKNOPPIX nodes have no display.

Once DISTCC_HOSTS has your list of machines, you can run make with the following command from your source tree:

knoppix@ttyp0[code]$ make -j5 CC=distcc

Replace 5 with the number of jobs you want make to create. A general rule of thumb is to have between one extra job than you have computers and twice the number of computers. This example has three computers, so you should pick between four and six jobs.

DistccKNOPPIX is released under the same licenses as Knoppix and is available from http://opendoorsoftware.com. At this writing, v0.8 is DisctccKNOPPIX's available version. Future features include a boot manager to manage some options (such as default-compiler versions), server notification upon boot, and better network detection.

8.10.1 See Also

• The distcc home page at http://distcc.samba.org.

—James Michael Greenhalgh