Setting up a DrQueue Render Farm for Blender

DrQueue is a powerful open source distributed render farm manager that I happened to set up
to manage a blender render farm while I was administering a training session for TOSMI.

A render farm consists of a master machine that controls what jobs are executed and a number of
slave machines that do the actual tasks. A detailed review can be found here

The following how-to applies to Ubuntu 8.04LTS but can easily be ported to any platform.

First we need to set up the master machine :

Install all needed software
apt-get install scons libgtk2.0-dev g++ python gcc samba

Download the latest version of DrQueue
cd /usr/src
wget http://drqueue.org/files/1-Sources_all_platforms/drqueue.0.64.3.tgz
tar xzvf drqueue.0.64.3.tgz
cd drqueue-0.64.3

DrQueue uses the scons build engine to handle compilation and installation.
scons PREFIX=/var install and drqueue will be installed in /var/drqueue directory.
We need to edit the configuration files
vi /var/drqueue/etc/master.conf

logs=/var/drqueue/logs
tmp=/var/drqueue/tmp
db=/var/drqueue/db
bin=/var/drqueue/bin
etc=/var/drqueue/etc

vi /var/drqueue/etc/slave.conf

logs=/var/drqueue/logs
tmp=/var/drqueue/tmp

vi /var/drqueue/etc/drqman.conf

logs=/var/drqueue/logs
tmp=/var/drqueue/tmp
db=/var/drqueue/db

Create the start-up scripts
cd /etc/init.d
vi master-start.bash

#!/bin/bash

export DRQUEUE_ROOT=/var/drqueue
export DRQUEUE_MASTER=<local ip address>
export DISPLAY=:0
export PATH=$PATH:/var/drqueue/bin

/var/drqueue/bin/master 1>> /var/drqueue/logs/drqueue-m.log &
sleep 2
/var/drqueue/bin/slave 1>> /var/drqueue/logs/drqueue-s.log &

sudo chmod a+x master-start.bash
sudo update-rc.d master-start.bash start 98 2 3 4 5 . stop 98 0 1 6 .

Create a startup script for DrQueue manager
vi /var/drqueue/bin/drqman.sh

#!/bin/bash

export DRQUEUE_ROOT=/var/drqueue
export DRQUEUE_MASTER=192.168.0.224
export DISPLAY=:0
export PATH=$PATH:/var/drqueue/bin

/var/drqueue/bin/drqman

sudo chmod a+x drqman.sh
ln -s /var/drqueue/bin/drqman.sh /usr/bin/drqman.sh

The DrQueue render farm needs to shared directory for the jobs as well
as the output files. We will use samba in case we want to add Windows
machines as slaves. The security is non-existant in this setup but it
is easily configurable to fit your needs.
vi /etc/samba/smb.conf

[drqueue]
path = /var/drqueue
writable = yes
public = yes
comment = dr_queue
available = yes
browseable = yes
guest ok = yes
force user = root
create mask = 0777
directory mask = 0777

Setting up the slaves :
In this setup since all the slaves were identical and indentical to the master
machine we set them up so that everything is shared even the binaries.
In a mixed environment the setup would need to be different.

Install needed software
apt-get install fping tcsh smbfs
mkdir /var/drqueue

Create the needed script :
vi /usr/bin/drqueue.sh

#!/bin/bash

MASTER=192.168.0.224 # Renderfarm master server
SHAREDDIR=/var/drqueue # Shared directory

while [ 1 ]
do
check=$(fping $MASTER |awk {'print $3'})
if [ "$check" = "alive" ]
then
mountchk=$( mount |grep drqueue )
if [ "$mountchk" ]
then
echo "mounted" >> /dev/null
else
mount -t smbfs -o username=tosmi,password=tosmi,dmode=777 //$MASTER/drqueue $SHAREDDIR
export DRQUEUE_MASTER=$MASTER
export DRQUEUE_ROOT=$SHAREDDIR
/var/drqueue/bin/slave 1>/dev/null 2>&1 &
fi
else
mountchk=$( mount |grep share )
if [ "$mountchk" ]
then
umount -fl $SHAREDDIR
pidis=$(ps ax |grep -i slave |grep -v grep)
kill -9 $(echo $pidis)
fi
fi
sleep 300
done

chmod a+x /usr/bin/drqueue.sh

Create a service init script
vi /etc/init.d/drqueue_slave.sh

#!/bin/bash
/usr/bin/drqueue.sh &

chmod a+x /etc/init.d/drqueue_slave.sh

sudo update-rc.d drqueue_slave.sh start 98 2 3 4 5 . stop 98 0 1 6 .

Bookmark/Search this post with: