Setting up a DrQueue Render Farm for Blender
DrQueue is a powerful open source distributed render farm manager that I happened to set up
to manage a blender render farm while I was administering a training session for TOSMI.
A render farm consists of a master machine that controls what jobs are executed and a number of
slave machines that do the actual tasks. A detailed review can be found here
The following how-to applies to Ubuntu 8.04LTS but can easily be ported to any platform.
First we need to set up the master machine :
Install all needed software
apt-get install scons libgtk2.0-dev g++ python gcc samba
Download the latest version of DrQueue
cd /usr/src
wget http://drqueue.org/files/1-Sources_all_platforms/drqueue.0.64.3.tgz
tar xzvf drqueue.0.64.3.tgz
cd drqueue-0.64.3
DrQueue uses the scons build engine to handle compilation and installation.
scons PREFIX=/var install and drqueue will be installed in /var/drqueue directory.
We need to edit the configuration files
vi /var/drqueue/etc/master.conf
| logs=/var/drqueue/logs tmp=/var/drqueue/tmp db=/var/drqueue/db bin=/var/drqueue/bin etc=/var/drqueue/etc |
vi /var/drqueue/etc/slave.conf
| logs=/var/drqueue/logs tmp=/var/drqueue/tmp |
vi /var/drqueue/etc/drqman.conf
| logs=/var/drqueue/logs tmp=/var/drqueue/tmp db=/var/drqueue/db |
Create the start-up scripts
cd /etc/init.d
vi master-start.bash
| #!/bin/bash
export DRQUEUE_ROOT=/var/drqueue /var/drqueue/bin/master 1>> /var/drqueue/logs/drqueue-m.log & |
sudo chmod a+x master-start.bash
sudo update-rc.d master-start.bash start 98 2 3 4 5 . stop 98 0 1 6 .
Create a startup script for DrQueue manager
vi /var/drqueue/bin/drqman.sh
| #!/bin/bash
export DRQUEUE_ROOT=/var/drqueue /var/drqueue/bin/drqman |
sudo chmod a+x drqman.sh
ln -s /var/drqueue/bin/drqman.sh /usr/bin/drqman.sh
The DrQueue render farm needs to shared directory for the jobs as well
as the output files. We will use samba in case we want to add Windows
machines as slaves. The security is non-existant in this setup but it
is easily configurable to fit your needs.
vi /etc/samba/smb.conf
| [drqueue] path = /var/drqueue writable = yes public = yes comment = dr_queue available = yes browseable = yes guest ok = yes force user = root create mask = 0777 directory mask = 0777 |
Setting up the slaves :
In this setup since all the slaves were identical and indentical to the master
machine we set them up so that everything is shared even the binaries.
In a mixed environment the setup would need to be different.
Install needed software
apt-get install fping tcsh smbfs
mkdir /var/drqueue
Create the needed script :
vi /usr/bin/drqueue.sh
| #!/bin/bash
MASTER=192.168.0.224 # Renderfarm master server while [ 1 ] |
chmod a+x /usr/bin/drqueue.sh
Create a service init script
vi /etc/init.d/drqueue_slave.sh
| #!/bin/bash /usr/bin/drqueue.sh & |
chmod a+x /etc/init.d/drqueue_slave.sh
sudo update-rc.d drqueue_slave.sh start 98 2 3 4 5 . stop 98 0 1 6 .
