Note: This article has been reworked to use Nginx as frontend to Apache2, instead of the fairly unusual configuration shown in this article. You probably want to view the reworked version instead.

When you are building something important to you, you want to build it on a solid foundation. There is nothing more frustrating than trying to build a great piece of software and then running it on a poorly configured server. Recently I decided to stop having that feeling and I rebuilt my server, here are my instructions.

The end product is an Ubuntu Feisty server using Apache2 and mod_python to serve Django, and lighttpd to serve static media. It uses memcached as its caching backend, and uses Postgres8.2 as its database. The machine built using these instructions is in fact running this blog, which is a Django application. I performed this installation on a SliceHost 256 meg slice, but they would apply equally well to any Ubuntu server (not so well to shared hosting).

On to the fun stuff.

Credit

This setup guide is a consolidation of a huge number of other resources written by a wide number of individuals. Here are the resources I used during this setup process, I will try to also link to them in the context that I used them.

Upgrading Ubuntu Dapper to Feisty

These instructions were taken from here.

Upgrading from Ubuntu Dapper (SliceHost's current Ubuntu OS) to Feisty is quick and painless. You do need to do the upgrades in order though, skipping straight to feisty is likely to make your installation unstable. The final 'lsb_release -a' is simply to confirm that the upgrade has occured. It should show that you now have Ubuntu Feisty installed. 

sed -e  's/dapper/ edgy/g'  -i /etc/apt/sources.list
   apt-get update
   apt-get dist-upgrade
   apt-get -f install
   dpkg --configure -a
   reboot



sed -e  's/edgy/ feisty/g'  -i /etc/apt/sources.list
   apt-get update
   apt-get dist-upgrade
   apt-get -f install
   dpkg --configure -a
   reboot



lsb_release -a

Adding Apache & Other Libraries

Mostly taken from here.

We install a number of libraries here. Some of them will have fun little installation screens to walk through. None of them were particularly difficult though. You may notice that psycopg2 is not being installed via apt-get, thats because the psycopg2 in the apt-get repository didn't work for me (kept causing segment faults). 

apt-get install emacs apache2 libapache2-mod-python subversion php5
apt-get install gcc libc6-dev build-essential python-dev python-setuptools
apt-get install python-egenix-mxdatetime memcached postfix
apt-get install postgresql-8.2 postgresql-server-dev-8.2 lighttpd
easy_install psycopg2
apache2ctl restart

This will be the easiest section of the installation. Still nice while it lasted.

PostgreSQL

Instructions taken from here and here.

Make sure to use your own password instead of just 'password' in the code below. 

su postgres -c psql template1
ALTER USER postgres WITH PASSWORD 'password';
\q

And then we edit the pg_hba.conf file. 

emacs /etc/postgresql/8.2/main/pg_hba.conf

Go to the end of the file and comment out all lines that start with host (unless you will be accessing your database remotely). Finally the local line should look like 

local  all         all                                             password

Finally we'll want to restart Postgres: 

/etc/init.d/postgresql-8.2 restart

Configuring memcached

Now we are going to setup memcached. This is pretty easy to do, and it is going to provide a great caching system for your Django system. The first step is: 

memcached -u www-data -p 11211 -m 32 -d

On Ubuntu the user www-data is the user that runs the webserver, making it very similar to the apache user on some other distributions. Running memcached with the -u www-data option means that we'll be running memcached with the same user as the webserver. Port 11211 is the default for memcached, and probably should not be changed unless you are running multiple memcached instances. I chose to start my memcached instance with 32 megs of memory because my slice only has 256 meg total, and my Django app simply doesn't have very much information to cache. You may want to devote more of your memory to memcached, especially if you have a larger slice.

Next we need to get python-memcached, which is a python memcached client. There is an alternate cmemcached library for Python that is twice as fast as python-memcached, but I had trouble getting it to compile (I believe because I installed memcached from a repository instead of from source). Python-memcached is easy to get: 

sudo wget ftp://ftp.tummy.com/pub/python-memcached/python-memcached-1.36.tar.gz
tar -zxvf python-memcached-1.36.tar
cd python-memcached-1.36
sudo python setup.py install

And that should be that.

At this exact moment (July 12, 2007) the current svn version of Django is broken with python-memcached. A patch has been submitted and is working its way through the submission system, so hopefully this issue will resolve itself soon. If anyone is affected by this situation, send me an email or leave a comment and I'll update this guide to include the required modifications.

Setting Up a Non-Root Account

Now for some security minded things. Not that I know very much about security, but every bit helps. 

useradd somename
passwd somename
emacs /etc/sudoers

Copy the line for root and replicate it for your new account.

Now you'll want to close ssh and ssh back in as the new user your just made.

Then you'll want to disable the root account. 

sudo passwd -l root

The line of thought behind disabling the root account is that it is bad to have a commonly known name that is available to SSH. Its also bad to have it accessible to a quick su. Admittedly your new account will still be accessible, but at least it will be your account and not an universally known one.

Setting up automatic SSH login with keys

This is more of a convience solution, but it is also a boost to your security once we disable password based access (although you might opt not to). I am assuming you have a set of rsa keys already on your local machine at ~/.ssh . If you don't have them (or you are using Windows, etc), you'd be better off looking for a more detailed guide. 

scp ~/.ssh/id_rsa.pub user@yourip:~/
ssh user@yourip
cd
mkdir .ssh
mv id_rsa.pub .ssh/authorized_keys
chmod go-w ~/.ssh/authorized_keys ~/.ssh/

Remote login should now be automatic. Now we will restrict SSH a bit more 

sudo groupadd sshers
sudo usermod -a -Gsshers yourusername
sudo emacs /etc/ssh/sshd_config

You will want to make these changes 

 #X11Forwarding yes
   X11Forwarding no


# these will have to be appended at end
UseDNS no
AllowGroups sshers

Now log out, ssh in. Everything worked, right? If so we are now going to disable passwords for ssh (have to have an enable key). Once again 

sudo emacs /etc/ssh/sshd_config

and append this line 

PasswordAuthentication no

and make this change 

#UsePAM yes
UserPAM no

(PAM helps prevent brute force SSH logins, which no longer applies since password based login is disabled.)

Now only users with valid keys can ssh in... which means only your one account can ssh in. If you want to allow others to log in you can have them send you their public rsa keys, or you could temporarily enable password based login while they set up ssh.

Configuring iptables

Yep... I don't know how to do this yet on Ubuntu. Which is to say I don't know where the config file is. The command line tool is killing me. I refuse to learn another mini-language! Do we really need mini-languages for every mundane tasks?

That said, you should do this. Hopefully I'll crack open the helpfiles soon and update this afterwards.

Setting up Django, etc

Now we are going to setup Django. We will first create a postgres user for our Django app: 

sudo su postgres
createuser -P pg_yourusername
# should not be a superuser
createdb --encoding=UNICODE db_yourtable -O pg_yourusername
exit

Hint: Write down the database table, user and password we'll be using them again in a couple of minutes, just far enough in the future to completely forget them all.

Now we need to give the www-data user access to our files (www-data is the user that runs the web server). 

sudo gpasswd -a www-data yourusername
chmod g+w ~

Then we'll want to create some folders for Django. You can feel free to play with the folder layout, its mostly a personal thing, but you'll have to keep any changes you make in mind when you follow the remaining instructions. 

cd
mkdir projects
mkdir apps
mkdir templates
mkdir src
cd src

Now we check out the Django source and link the checked out source into the site-packages so that the python interpreter can find it. 

sudo svn co http://code.djangoproject.com/svn/django/trunk 
sudo ln -s `pwd`/trunk/django /usr/lib/python2.5/site-packages/

Now we get to create our first Django project. 

cd ~/projects
~/src/trunk/django/bin/djang-admin.py startproject myproject

Now edit your newly minted settings.py file. 

emacs ~/projects/myproject/settings.py

You'll need to make a number of changes, these are the lines you will need to add (not alter) to your settings.py: 

CACHE_BACKEND = 'memcached://127.0.0.1:112211/'
CACHE_MIDDLEWARE_SECONDS = 60 * 60
CACHE_MIDDLEWARE_KEY_PREFIX = ''
CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True

I personally use a very long middleware cache because my pages don't change much (I am primarily serving a blog, and thus the caching entire pages is okay for my situation), the standard value is much lower, around 300 or so. The key prefix is used to distinguish caches between multiple Django projects using the same caching backend. If you are only planning to have one project using your caching backend then it is fine to leave the key as an empty string, if you do plan on ahving multiple projects each should have a unique key prefix. Finally, the anonymous only option means that logged in users will not recieve cached pages. For my application, where only the admin will ever be logged in, this is an appropriate setting, your milleage may vary.

And here are the already existing lines you will need to modify in settings.py: 

DATABASE_ENGINE = 'postgresql_psycopg2'
DATABASE_NAME = 'db_yourtable'
DATABASE_USER = 'pg_yourusername'
DATABASE_PASSWORD = 'whatever_you_chose'
MEDIA_ROOT = '/home/youruser/media/'
MEDIA_URL = 'http://yourdomain.com/media/'
ADMIN_MEDIA_PREFIX = '/media/admin/'

MIDDLEWARE_CLASSES = (
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.middleware.http.ConditionalGetMiddleware',
    'django.middleware.gzip.GZipMiddleware',
    'django.middleware.cache.CacheMiddleware',
    'django.middleware.doc.XViewMiddleware',
)

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.admin',
)

Extending this Django setup later: you will eventually want to add your own templates and your own media along with your own appl ication. To provide access to your media files you will want to create a symlink from wherever they are into your /var/www/yourdomain.com/media folder like this: 

sudo ln -s /home/yourusername/yourapp/media/yourapp /var/www/yourdomain.com/media/

For templates you'll just need to append a path to your template directory to the TEMPLATE_DIRS variable in the setting s.py file. Configuring Django applications and projects is sometimes more of an art than a science. Go try painting for a while, but feel free to ask for help if you need it.

After creating our Django project we need to finish setting up Apache. First we need to make a directory for our error logs: 

sudo mkdir /var/log/apache2/yourdomain.com

Then we'll need to edit our apache config file: 

sudo emacs /etc/apache2/sites-available/yourdomain.com

That will initially be an empty file, and you'll be adding this to it: 

  <VirtualHost *  > 
   ServerName www.yourdomain.com
   ServerAlias yourdomain.com


DocumentRoot /var/www/yourdomain.com

CustomLog /var/log/apache2/yourdomain.com/access.log combined
ErrorLog /var/log/apache2/yourdomain.com/error.log

SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE yourdjangoproject.settings
PythonDebug On
PythonPath "['/usr/lib/python2.5/site-packages/django'] + sys.path"

<Location "/media/">
SetHandler None
</Location>
</VirtualHost>

Finally a few more symlinks to seal the deal: 

sudo ln -s /etc/apache2/sites-available/yourdomain.com.com /etc/apache2/sites-enabled/yourdomain.com



sudo mkdir /var/www/yourdomain.com/media
   sudo ln -s /home/your-user-name-here/src/trunk/django/contrib/admin/media/ /var/www/yourdomain.com/media/admin

Go ahead and restart apache and a stock Django page should show up: 

sudo apache2ctl restart

Install Lighttpd

Basically I followed the instructions here. They are fantastic instructions, although I did end up playing with them a bit to get things working with my specific installation details.

First we open up the config file: 

sudo emacs /etc/lighttpd/lighttpd.conf

Next you need to uncomment line 60 

server.port = 81

Add the following (perhaps at line 118 like he does here): 

$HTTP["host"] =~ "yourdomain\.com" {
    evhost.path-pattern = "/var/www/yourdomain.com/media/"
}

Then we start the lighttpd server: 

sudo /etc/init.d/lighttpd start

And enable some apache2 mods... 

sudo ln -s /etc/apache2/mods-available/proxy.load /etc/apache2/mods-enabled/
sudo ln -s /etc/apache2/mods-available/proxy.conf /etc/apache2/mods-enabled/
sudo ln -s /etc/apache2/mods-available/proxy_connect.load /etc/apache2/mods-enabled/
sudo ln -s /etc/apache2/mods-available/proxy_http.load /etc/apache2/mods-enabled/

And finally edit the proxy.conf file 

sudo emacs /etc/apache2/mods-available/proxy.conf

You will need to modify the file to look like this (you will be changing the existing config file to look like what is below, not just appending the text below to the config file): 

<Proxy 127.0.0.1>
        AddDefaultCharset off
        Order deny,allow
        Deny from all
        Allow from 127.0.0.1
</Proxy>

You don't need to enable proxy routing/etc or modify any parts of the file outside of the tags

Next you need to modify the VirtualHost file: 

sudo emacs /etc/apache2/sites-available/yourdomain.com

Add these lines inside your VirtualHost (anywhere, except in the media location, is fine. I put them near the top): 

ProxyRequests Off
ProxyPreserveHost On
ProxyPass /web http://127.0.0.1:81/
ProxyPassReverse / http://127.0.0.1:81/

Load changes into Apache and restart it. 

sudo /etc/init.d/apache2 reload
sudo apache2ctl restart

The instructions I borrowed from recommended using curl to verify you are serving pages from lighttpd, but its easier (for me) to just use Firefox/Firebug and look at the response headers for the static media files you think lighttpd should be serving. Either way you should now be serving all files out of the /media folder using lighttpd.

Finished

At this point you should have a pretty kickin' Django server. You may want to spruce up the security a bit (I'll be looking into this and adding some more details), but I think this, security aside, is about as close to a production server as I can come. Mod_python for Django, lighttpd for static files, memcached for caching, Python 2.5 (2.4 would be a bit more stable, but at this point I think 2.5 is more than sufficiently mature), it works pretty well for me.

Let me know if you have any questions about any of the setup and I'd be glad to help. Also, if anyone has any ideas about improving this setup I'd really like to hear them.