# HG changeset patch # User Steve Losh # Date 1309401050 14400 # Node ID 16794453cea5d179723b1bd4d81d98199026dc92 # Parent 63e4adac659e1bdcd881374d818875bf71a5266e Add the Django entry. diff -r 63e4adac659e -r 16794453cea5 content/blog/2011/06/django-advice.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/content/blog/2011/06/django-advice.html Wed Jun 29 22:30:50 2011 -0400 @@ -0,0 +1,1055 @@ + {% extends "_post.html" %} + + {% hyde + title: "Django Advice" + snip: "Some useful things I've learned." + created: 2011-06-30 08:30:00 + flattr: true + %} + + {% block article %} + +For the past year and a half or so I've been working full-time at [Dumbwaiter +Design][] doing [Django][] development. I've picked up a bunch of useful tricks along +the way that help me work, and I figured I'd share them. + +I'm sure there are better ways to do some of the things that I mention. If you know +of any feel free to hit me up on [Twitter][] and let me know. + +Also: this entry was written over several months, so if there are inconsistencies let +me know and I'll try to fix them. + +[Dumbwaiter Design]: http://dwaiter.com/ +[Django]: {{links.django}} +[Twitter]: http://twitter.com/stevelosh + +[TOC] + +Vagrant +------- + +I used to develop Django sites by running them on my OS X laptop locally and +deploying to a Linode VPS. I had a whole section of this post written up about +tricks and tips for working with that setup. + +Then I found [Vagrant][]. + +I just deleted the entire section of this post I wrote. + +Vagrant gives you a better way of working. You need to use it. + +[Vagrant]: http://vagrantup.com/ + +### Why Vagrant? + +If you haven't used it before, Vagrant is basically a tool for managing +[VirtualBox][] VMs. It makes it easy to start, pause, and resume VMs. Instead of +installing Django in a virtualenv and developing against that, you run a VM which +runs your site and develop against that. + +This may not sound like much, but it's kind of a big deal. The critical difference +is that you can now develop against the same setup that you'll be using in +production. + +This cuts out a huge amount of pain that stems from OS differences. Here are a few +examples off the top of my head: + +* URLField and MacPorts Python 2.5 on OS X. There's a [bug][] where using + verify\_exists will crash your site every time you save a model, unless you set + a particular environment variable with no debug information. Yeah, I spent + a couple of hours tracking that one down at work. Awesome. +* Installing PIL on OS X is no picnic. [homebrew][] makes things better, if you use it, + so this one isn't a huge deal. +* Every time you update Python in-place on your local machines, ALL of your + virtualenvs break because the Python binaries inside are linked against global + Python library files. Have fun recreating them. I hope you froze your + `requirements.txt` files before you updated. + +Using Vagrant and VMs means you can just worry about ONE operating system and its +quirks. It saves you a ton of time. + +Aside from that, there's another benefit to using Vagrant: it strongly encourages you +to learn and use an automated provisioning system. Support for Puppet and Chef is +built in. I chose Puppet, but if you prefer Chef that's cool too. + +Because you're developing against a VM and deploying to a VM, you can reuse 90% of +the provisioning code across the two. + +When I make a new site, I do the following to initialize a new Vagrant VM: + +1. `vagrant up` (which runs Puppet to initialize the VM) +2. `fab dev bootstrap` + +When I'm ready to go live, I do the following: + +1. Buy a Linode VPS. +2. Run Puppet to initialize the VPS. +3. Enter the Linode info in my fabfile. +4. `fab prod bootstrap` + +No more screwing around with different paths, different versions of Nginx, different +versions of Python. When I'm developing something I can be pretty confident it will +"just work" in production without any major surprises. + +[VirtualBox]: http://www.virtualbox.org/ +[bug]: https://trac.macports.org/ticket/24421 +[homebrew]: http://mxcl.github.com/homebrew/ + +### Using Fabric to Stay Fast and Automate Everything + +One of the problems with this setup is that I can't just run `python manage.py +whatever` any more because I need it to run on the VM. + +To get around this I've created many simple [Fabric][] tasks to automate the common +things I need to do. Fabric is an awesome little Python utility for scripting tasks +(like deployments). We use it constantly at Dumbwaiter. Here are a few examples +from our fabfiles. + +[Fabric]: http://fabfile.org/ + +This first set is for running abitrary commands easily. + +`cmd` and `vcmd` will `cd` into the site directory on the VM and run a command of my +choosing. `vcmd` will prefix the command with the path to the virtualenv's `bin` +directory, so I can do something like `fab dev vcmd`, `pip install markdown`. + +The `sdo` commands do the same thing, but `sudo`'ed. + + :::python + def cmd(cmd=""): + '''Run a command in the site directory. Usable from other commands or the CLI.''' + require('site_path') + + + if not cmd: + sys.stdout.write(_cyan("Command to run: ")) + cmd = raw_input().strip() + + if cmd: + with cd(env.site_path): + run(cmd) + + def sdo(cmd=""): + '''Sudo a command in the site directory. Usable from other commands or the CLI.''' + require('site_path') + + if not cmd: + sys.stdout.write(_cyan("Command to run: sudo ")) + cmd = raw_input().strip() + + if cmd: + with cd(env.site_path): + sudo(cmd) + + def vcmd(cmd=""): + '''Run a virtualenv-based command in the site directory. Usable from other commands or the CLI.''' + require('site_path') + require('venv_path') + + if not cmd: + sys.stdout.write(_cyan("Command to run: %s/bin/" % env.venv_path.rstrip('/'))) + cmd = raw_input().strip() + + if cmd: + with cd(env.site_path): + run(env.venv_path.rstrip('/') + '/bin/' + cmd) + + def vsdo(cmd=""): + '''Sudo a virtualenv-based command in the site directory. Usable from other commands or the CLI.''' + require('site_path') + require('venv_path') + + if not cmd: + sys.stdout.write(_cyan("Command to run: sudo %s/bin/" % env.venv_path.rstrip('/'))) + cmd = raw_input().strip() + + if cmd: + with cd(env.site_path): + sudo(env.venv_path.rstrip('/') + '/bin/' + cmd) + +This next set is just some common commands that I need to run often. + + :::python + def syncdb(): + '''Run syncdb.''' + require('site_path') + require('venv_path') + + with cd(env.site_path): + run(_python('manage.py syncdb --noinput')) + + def collectstatic(): + '''Collect static media.''' + require('site_path') + require('venv_path') + + with cd(env.site_path): + sudo(_python('manage.py collectstatic --noinput')) + + def rebuild_index(): + '''Rebuild the search index.''' + require('site_path') + require('venv_path') + require('process_owner') + + with cd(env.site_path): + sudo(_python('manage.py rebuild_index')) + sudo('chown -R %s .xapian' % env.process_owner) + + def update_index(): + '''Update the search index.''' + require('site_path') + require('venv_path') + require('process_owner') + + with cd(env.site_path): + sudo(_python('manage.py update_index')) + sudo('chown -R %s .xapian' % env.process_owner) + +We also use Fabric to automate some of the more complex things we need to do. + +This task `curl`'s the site's home page to make sure we haven't completely borked +things. We use it in lots of other tasks as a sanity check. + + :::python + def check(): + '''Check that the home page of the site returns an HTTP 200.''' + require('site_url') + + print('Checking site status...') + + if not '200 OK' in local('curl --silent -I "%s"' % env.site_url, capture=True): + _sad() + else: + _happy() + +The `_happy` and `_sad` functions just print out some simple messages to get our +attention: + + :::python + from fabric.colors import red, green + + def _happy(): + print(green('\nLooks good from here!\n')) + + def _sad(): + print(red(r''' + ___ ___ + / /\ /__/\ + / /::\ \ \:\ + / /:/\:\ \__\:\ + / /:/ \:\ ___ / /::\ + /__/:/ \__\:\ /__/\ /:/\:\ + \ \:\ / /:/ \ \:\/:/__\/ + \ \:\ /:/ \ \::/ + \ \:\/:/ \ \:\ + \ \::/ \ \:\ + \__\/ \__\/ + ___ ___ ___ ___ + /__/\ / /\ / /\ / /\ ___ + \ \:\ / /::\ / /:/_ / /:/_ /__/\ + \ \:\ / /:/\:\ / /:/ /\ / /:/ /\ \ \:\ + _____\__\:\ / /:/ \:\ / /:/ /:/_ / /:/ /::\ \ \:\ + /__/::::::::\ /__/:/ \__\:\ /__/:/ /:/ /\ /__/:/ /:/\:\ \ \:\ + \ \:\~~\~~\/ \ \:\ / /:/ \ \:\/:/ /:/ \ \:\/:/~/:/ \ \:\ + \ \:\ ~~~ \ \:\ /:/ \ \::/ /:/ \ \::/ /:/ \__\/ + \ \:\ \ \:\/:/ \ \:\/:/ \__\/ /:/ __ + \ \:\ \ \::/ \ \::/ /__/:/ /__/\ + \__\/ \__\/ \__\/ \__\/ \__\/ + + + Something seems to have gone wrong! + You should probably take a look at that. + ''')) + +This one is for when `python manage.py reset APP` is broken because you've changed +some `db_column` names and Django chokes because of some constraits and you just want +to **reset the fucking app**. + +It's the "NUKE IT FROM ORBIT!!" option. + + :::python + def KILL_IT_WITH_FIRE(app): + require('site_path') + require('venv_path') + + with cd(env.site_path): + # Generate and download the reset SQL. + sudo(_python('manage.py sqlreset %s > reset.orig.sql' % app)) + get('reset.orig.sql') + + with open('reset.sql', 'w') as f: + with open('reset.orig.sql') as orig: + # Step through the first chunk of the file (the "drop" part). + line = orig.readline() + while not line.startswith('CREATE'): + if 'CONSTRAINT' in line: + # Don't write out CONSTRAINT lines. + # They're a problem when you change db_colum names. + pass + elif 'DROP TABLE' in line: + # Cascade drops. + # Hence with "with fire" part of this task's name. + line = line[:-2] + ' CASCADE;\n' + f.write(line) + else: + # Write other lines through untoched. + f.write(line) + line = orig.readline() + + # Write out the rest of the file untouched. + f.write(line) + f.write(orig.read()) + + # Upload the processed SQL file. + put('reset.sql', os.path.join(env.site_path, 'reset.ready.sql'), use_sudo=True) + + with cd(env.site_path): + # Use the SQL to reset the app, and fake a migration. + run(_python('manage.py dbshell < reset.ready.sql')) + sudo(_python('manage.py migrate --fake --delete-ghost-migrations ' + app)) + +This task uses Mercurial's local tags to add a `production` or `staging` tag in your local +repository, so you can easy see where the production/staging servers are at +compared to your local repo. + + :::python + def retag(): + '''Check which revision the site is at and update the local tag. + + Useful if someone else has deployed (which makes your production/staging local + tag incorrect. + ''' + require('site_path', provided_by=['prod', 'stag']) + require('env_name', provided_by=['prod', 'stag']) + + with cd(env.site_path): + current = run('hg id --rev . --quiet').strip(' \n+') + + local('hg tag --local --force %s --rev %s' % (env.env_name, current)) + +This task tails the Gunicorn logs on the server so you can quickly find out what's +happening when things blow up. + + :::python + def tailgun(follow=''): + """Tail the Gunicorn log file.""" + require('site_path') + + with cd(env.site_path): + if follow: + run('tail -f .gunicorn.log') + else: + run('tail .gunicorn.log') + +We've got a lot of other tasks but they're pretty specific to our setup. + +Wrangling Databases with South +------------------------------ + +If you're not using [South][], you need to start. Now. + +No, really, I'll wait. Take 30 minutes, try the [tutorial][Southtut], wrap your head +around it and come back. It's far more important than this blog post. + +[South]: http://south.aeracode.org/ +[Southtut]: http://south.aeracode.org/docs/tutorial/index.html + +### Useful Fabric Tasks + +South is awesome but its commands are very long-winded. Here's the set of fabric +tasks I use to save quite a bit of typing: + + :::python + def migrate(args=''): + '''Run any needed migrations.''' + require('site_path') + require('venv_path') + + with cd(env.site_path): + sudo(_python('manage.py migrate ' + args)) + + def migrate_fake(args=''): + '''Run any needed migrations with --fake.''' + require('site_path') + require('venv_path') + + with cd(env.site_path): + sudo(_python('manage.py migrate --fake ' + args)) + + def migrate_reset(args=''): + '''Run any needed migrations with --fake. No, seriously.''' + require('site_path') + require('venv_path') + + with cd(env.site_path): + sudo(_python('manage.py migrate --fake --delete-ghost-migrations ' + args)) + +Remember that running a migration without specifying an app will migrate everything, +so a simple `fab dev migrate` will do the trick. + +Watching for Changes +--------------------- + +When developing locally you'll want to make a change to your code and have the server +reload that code automatically. The Django development server does this, and we can +hack it into our Vagrant/Gunicorn setup too. + +First, add a `monitor.py` file at the root of your project (I believe I found this +code [here][monitor], but I may be wrong): + + :::python + import os + import sys + import time + import signal + import threading + import atexit + import Queue + + _interval = 1.0 + _times = {} + _files = [] + + _running = False + _queue = Queue.Queue() + _lock = threading.Lock() + + def _restart(path): + _queue.put(True) + prefix = 'monitor (pid=%d):' % os.getpid() + print >> sys.stderr, '%s Change detected to \'%s\'.' % (prefix, path) + print >> sys.stderr, '%s Triggering process restart.' % prefix + os.kill(os.getpid(), signal.SIGINT) + + def _modified(path): + try: + # If path doesn't denote a file and were previously + # tracking it, then it has been removed or the file type + # has changed so force a restart. If not previously + # tracking the file then we can ignore it as probably + # pseudo reference such as when file extracted from a + # collection of modules contained in a zip file. + + if not os.path.isfile(path): + return path in _times + + # Check for when file last modified. + + mtime = os.stat(path).st_mtime + if path not in _times: + _times[path] = mtime + + # Force restart when modification time has changed, even + # if time now older, as that could indicate older file + # has been restored. + + if mtime != _times[path]: + return True + except: + # If any exception occured, likely that file has been + # been removed just before stat(), so force a restart. + + return True + + return False + + def _monitor(): + while 1: + # Check modification times on all files in sys.modules. + + for module in sys.modules.values(): + if not hasattr(module, '__file__'): + continue + path = getattr(module, '__file__') + if not path: + continue + if os.path.splitext(path)[1] in ['.pyc', '.pyo', '.pyd']: + path = path[:-1] + if _modified(path): + return _restart(path) + + # Check modification times on files which have + # specifically been registered for monitoring. + + for path in _files: + if _modified(path): + return _restart(path) + + # Go to sleep for specified interval. + + try: + return _queue.get(timeout=_interval) + except: + pass + + _thread = threading.Thread(target=_monitor) + _thread.setDaemon(True) + + def _exiting(): + try: + _queue.put(True) + except: + pass + _thread.join() + + atexit.register(_exiting) + + def track(path): + if not path in _files: + _files.append(path) + + def start(interval=1.0): + global _interval + if interval < _interval: + _interval = interval + + global _running + _lock.acquire() + if not _running: + prefix = 'monitor (pid=%d):' % os.getpid() + print >> sys.stderr, '%s Starting change monitor.' % prefix + _running = True + _thread.start() + _lock.release() + +Next add a `post_fork` hook to your Gunicorn config file that uses the monitor to +watch for changes: + + :::python + def post_fork(server, worker): + import monitor + import local_settings + if local_settings.DEBUG: + server.log.info("Starting change monitor.") + monitor.start(interval=1.0) + +Now the Gunicorn server will automatically restart whenever code is changed. Use +whatever method for determining debug status that you like. We use +`local_settings.py` files which all have `DEBUG` variables, so that works for us. + +It will *not* restart when you add new code (e.g. when you install a new app), so +you'll need to handle that manually with `fab dev restart`, but that's not too bad! + +[monitor]: http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode + +### Using the Werkzeug Debugger with Gunicorn + +The final piece of the puzzle is being able to use the fantastic [Werkzeug +Debugger][debug] while running on the development VM with Gunicorn. + +To do this, create a `debug_wsgi.py` file at the root of your project: + + :::python + import os + import sys + import site + + parent = os.path.dirname + site_dir = parent(os.path.abspath(__file__)) + project_dir = parent(parent(os.path.abspath(__file__))) + + sys.path.insert(0, project_dir) + sys.path.insert(0, site_dir) + + site.addsitedir('VIRTUALENV_SITE_PACKAGES') + + from django.core.management import setup_environ + import settings + setup_environ(settings) + + import django.core.handlers.wsgi + application = django.core.handlers.wsgi.WSGIHandler() + + from werkzeug.debug import DebuggedApplication + application = DebuggedApplication(application, evalex=True) + + def null_technical_500_response(request, exc_type, exc_value, tb): + raise exc_type, exc_value, tb + from django.views import debug + debug.technical_500_response = null_technical_500_response + +Have Gunicorn use this file to run your development server with `gunicorn +debug_wsgi:application`. + +Make sure to replace `'VIRTUALENV_SITE_PACKAGES'` with the _full_ path to your +virtualenv's `site_packages` directory. You might want to make this a setting in +a machine-specific settings file. + +[debug]: http://werkzeug.pocoo.org/docs/debug/ + +### Pulling Uploads + +Once you give a client access to a site they'll probably be uploading images (through +Django's built-in file uploading features or with [django-filebrowser][]). + +When you're making changes locally it's often useful to have these uploaded files on +your local VM, otherwise you end up with a bunch of broken images. + +Here's a simple Fabric task that will pull down all the uploads from the server: + + :::python + def pull_uploads(): + '''Copy the uploads from the site to your local machine.''' + require('uploads_path') + + sudo('chmod -R a+r "%s"' % env.uploads_path) + + rsync_command = r"""rsync -av -e 'ssh -p %s' %s@%s:%s %s""" % ( + env.port, + env.user, env.host, + env.uploads_path.rstrip('/') + '/', + 'media/uploads' + ) + print local(rsync_command, capture=False) + +You might be wondering about the line that strips `/` characters and then adds them +back in. `rsync` does different things depending on whether you end a path with +a `/`, so this is actually pretty important. + +In your host task you'll need to set the `uploads_path` variable to something like +this: + + :::python + import os + env.site_path = os.path.join('var', 'www', 'myproject') + env.uploads_path = os.path.join(env.site_path, 'media', 'uploads') + +Now you can run `fab production pull_uploads` to pull down all the files people have +uploaded to the production server. + +[django-filebrowser]: http://code.google.com/p/django-filebrowser/ + +### Preventing Accidents + +Deploying to test and staging servers should be quick and easy. Deploying to +production servers should be harder to prevent people from accidentally doing it. + +I've created a little function that I call before deploying to production servers. +It forces me to type in random words from the system word list before proceeding to +make sure I *really* know what I'm doing: + + :::python + import os, random + + from fabric.api import * + from fabric.operations import prompt + from fabric.utils import abort + + WORDLIST_PATHS = [os.path.join('/', 'usr', 'share', 'dict', 'words')] + DEFAULT_MESSAGE = "Are you sure you want to do this?" + WORD_PROMPT = ' [%d/%d] Type "%s" to continue (^C quits): ' + + def prevent_horrible_accidents(msg=DEFAULT_MESSAGE, horror_rating=1): + """Prompt the user to enter random words to prevent doing something stupid.""" + + valid_wordlist_paths = [wp for wp in WORDLIST_PATHS if os.path.exists(wp)] + + if not valid_wordlist_paths: + abort('No wordlists found!') + + with open(valid_wordlist_paths[0]) as wordlist_file: + words = wordlist_file.readlines() + + print msg + + for i in range(int(horror_rating)): + word = words[random.randint(0, len(words))].strip() + p_msg = WORD_PROMPT % (i+1, horror_rating, word) + answer = prompt(p_msg, validate=r'^%s$' % word) + +You may need to adjust `WORDLIST_PATHS` if you're not on OS X. + +Working with Third-Party Apps +----------------------------- + +One of the best parts about working with Django is that many problems have already +been solved and the solutions have been released as open-source applications. + +We use quite a few open-source apps, and there are a couple of tricks I've learned to +make working with them easier. + +### Installing Apps from Repositories + +If I'm going to use an open-source Django app in a project I'll almost always install +it as an editable repository on the VM with `pip install -e`. + +Others may disagree with me on this, but I think it's the best way to work. + +Often I'll find a bug that I think may be in one of the third-party apps I'm using. +Installing the apps as repositories makes it easy to read their source and figure out +if the bug is really in the app. + +If the bug *is* in the third-party app having the app installed as a repository makes +it simple to fix the bug, fork the project on BitBucket or GitHub, send a pull +request, and get back to work. + +### Mirroring Repositories + +One problem we've run into at Dumbwaiter is that the repos for third-party apps we +use are scattered across GitHub, BitBucket, Google Code, and other servers. If any +one of these services goes down we're stuck waiting for it to come back up. + +A while ago I took half a day and consolidated all of these repos onto one of the +servers that we control. The basic process went like this: + +* Use [hg-git][] and [hgsubversion][] to convert the git and SVN repos to Mercurial + repos. +* Set up a master `mirror` Mercurial repo with all the app repos as subrepos. +* Push the master repo and all the subrepos up to one of our Linodes. + +Now we can use `-e ssh://hg@OUR_LINODE/mirror/APP@REV_THAT_WORKS#egg=APP` in our +`requirements.txt` files to install apps from our mirror. When we want to update our +dependencies we can simply pull from the upstream repos and commit in the mirror +repo. + +If our mirror goes down it's not a big deal, because we have far bigger problems to +worry about than new projects. + +I wrote a few scripts to automate updating apps and such, but they're extremely hacky +so I don't want ot post them here. Take half a day and write your own set -- it's +definitely worth it to have your own mirror of your specific dependencies. + +[hg-git]: http://hg-git.github.com/ +[hgsubversion]: https://bitbucket.org/durin42/hgsubversion/wiki/Home + +### Using BCVI to Edit Files + +I said that when I find a bug that I think is in a third-party app I'll poke around +with the app and try to figure it out. But since all the apps are installed in +a virtualenv on the Vagrant VM it might seem like it's a pain in the ass to edit +those files! + +Luckily [BCVI][] exists. It's a utility that opens a "back channel" to your local +machine when you SSH and lets you run `vi FILE` to open that file in +Vim/MacVim/GVim/etc on your *local* machine. + +It can be a bit tricky to set up, but it's worth it. Trust me. + +[BCVI]: http://sshmenu.sourceforge.net/articles/bcvi/ + +Improving the Admin Interface +----------------------------- + +I'm going to be honest: Django's admin interface is the main reason I'm still using +it. Other frameworks like [Flask][] are great, but Django's admin saves me +*ridiculous* amounts of time when I'm making simple CRUD sites for clients. + +That said, the Django admin isn't the prettiest thing around, but we can give it +a facelift. + +[Flask]: http://flask.pocoo.org/ + +### Enter Grappelli + +[Grappelli][] is a Django app that reskins the admin interface beautifully. It also +adds some functionality like drag-and-drop reordering of inlines, and allows you to +customize the dashboard to your liking. *Every* Django site I work on uses Grappelli +-- it's just that good. + +The downside of Grappelli is that it changes quite a lot and breaks backwards +compatibility at the drop of a hat. + +If you're going to use Grappelli you *must* freeze your requirements.txt files and +work with a single version at a time. Trying to always work from the trunk will make +you drink. + +[Grappelli]: http://django-grappelli.readthedocs.org/ + +### An Ugly Hack to Show Usable Foreign Key Fields + +A limitation of both Grappelli and the stock Django admin is that it seems like you +can't easily show fields from related models in the admin list view. + +For example, if you're new to Django you might expect this to work: + + :::python + class BlogEntryAdmin(admin.ModelAdmin): + list_display = ('title', 'author__name') + +Unfortunately Django chokes on the `author__name` lookup. You can *display* the name +without too much fuss: + + :::python + class BlogEntryAdmin(admin.ModelAdmin): + list_display = ('title', 'author_name') + + def author_name(self, obj): + return o.name + +That will display the name just fine. However, it won't be a fully-fledged column in +the Django admin because you can't sort on it. + +It may seem like this is the end -- if it could be a fully-functional field, why +wouldn't Django just let you use `author__name`? Luckily we can add one more line to +fix the problem: + + :::python + class BlogEntryAdmin(admin.ModelAdmin): + list_display = ('title', 'author_name') + + def author_name(self, obj): + return o.name + author_name.admin_order_field = 'author__name' + +Now the author name has all the functionality of a real `list_display` entry. + +Using Django-Annoying +--------------------- + +If you haven't heard of [django-annoying][] you should definitely check it out. It's +got a bunch of miscellaneous functions that fix some common, annoying parts of +Django. + +My two personal favorites from the package are a pair of decorators that help make +your views much, much cleaner. + +[django-annoying]: https://bitbucket.org/offline/django-annoying/wiki/Home + +### The render\_to Decorator + +The decorator is called `render_to` and it eliminates the ugly `render_to_response` +calls that Django normally forces you to use in every single view. + +Normally you'd use something like this: + + :::python + def videos(request): + videos = Video.objects.all() + return render_to_response('video_list.html', { 'videos': videos }, + context_instance=RequestContext(request)) + +With `render_to` your view gets much cleaner: + + :::python + @render_to('video_list.html') + def videos(request): + videos = Video.objects.all() + return { 'videos': videos } + +Less typing `context_instance=...` over and over, and less syntax to remember. + +### The ajax\_request Decorator + +The `ajax_request` decorator is like `render_to` for AJAX requests. You simply +return a Python dictionary from your view and the decorator handles the JSON encoding +and such: + + :::python + @ajax_request + def ajax_get_entries(request): + blog_entries = BlogEntry.objects.all() + return { 'entries': [(entry.title, entry.get_absolute_url()) + for entry in entries]} + +Templating Tricks +----------------- + +I'm not a frontend developer, but I've done my share of HTML hacking at Dumbwaiter. +Here are a few of the tricks I've learned. + +### Null Checks and Fallbacks + +A common pattern I see in Django templates looks like this: + + :::text + {% templatetag openblock %} if business.title {% templatetag closeblock %} + {% templatetag openvariable %} business.title {% templatetag closevariable %} + {% templatetag openblock %} else {% templatetag closeblock %} + {% templatetag openvariable %} business.short_title {% templatetag closevariable %} + {% templatetag openblock %} endif {% templatetag closeblock %} + +Here's a simpler way to do that: + + :::text + {% templatetag openblock %} firstof business.title business.short_title {% templatetag closeblock %} + +`firstof` will return the first non-Falsy item in its arguments. + +### Manipulating Query Strings + +Query strings are normally not a big deal, but every once in a while you'll have +a model listing page where you need to filter by category, and number of spaces, and +tags, etc all at once. + +If you're trying to manage GET queries manually it can get pretty hairy very fast. + +[This Django snippet][qstring] makes working with query strings in templates +a breeze. + +[qstring]: http://djangosnippets.org/snippets/2237/ + +### Satisfying Your Designer with Typogrify + +If you haven't heard of [Typogrify][] you should take a look at it. It makes it easy +to add all the typographic goodness your designers are looking for. + +[Typogrify]: http://code.google.com/p/typogrify/ + +The Flat Page Trainwreck +------------------------ + +Creating a site for a client is very different than creating a site for yourself. +For pretty much every client we've dealt with we've heard: "can't we just create +a new page at /drink-special/ for this special deal we're running?" + +Having clients go through you to make new pages is simply too much overhead. We +needed a way to let clients create new pages (like `/drink-special/`) on the fly, +without our intervention. + +Django has a "flatpages" app that solves this problem. Kind of. + +When using flat pages clients need to do two things that are often too much for +non-technical people: + +* Manage URLs manually. +* Write all content as raw HTML in a single text field. + +We've tried a lot of Django CMS apps at Dumbwaiter, and none of them made us happy. +They all seemed to have some or all of the following problems: + +* They take over your site and make you write a "Django-WhateverCMS site" instead of + a "Django site". +* They're extremely feature-rich and complicated with features like + internationalization, redirects, versions, and many others. This is great if you + need the flexibility, but bad if your clients just need to create a couple of + pages. +* They break `APPEND_TRAILING_SLASH` and make you clutter your `urls.py` files with + a bunch of extra code ot handle this. + +I finally got fed up and wrote my own Django CMS app: [Stoat][]. Stoat is designed +to be sleek, with only the features that our clients need. + +It's not officially version 1.0 yet, but we're using it for a few clients and it's +working well. Check it out if you're looking for a more lightweight CMS app. + +[Stoat]: http://stoat.rtfd.org/ + +Editing with Vim +---------------- + +I [use Vim][vimpost] to edit everything. Naturally I've found a bunch of plugins, +mappings and other tricks that make it even better when working on Django projects. + +[vimpost]: /blog/2010/09/coming-home-to-vim/ + +### Vim for Django + +There are a lot of ways to make Vim work with Django. I won't go into all of them in +this post, but a good place to start is [this Django wiki page][vimdjango]. + +[vimdjango]: https://code.djangoproject.com/wiki/UsingVimWithDjango + +### Filetype Mappings + +Most files in a Django project have one of two extensions: `.py` and `.html`. +Unfortunately these extensions aren't unique to Django, so Vim doesn't automatically +set the correct `filetype` when you open one. + +I've added a few mappings to my `.vimrc` to make it quick and easy to set the correct +`filetype`: + + :::text + nnoremap _dt :set ft=htmldjango + nnoremap _pd :set ft=python.django + +I also have a few autocommands that set the filetype for me when I'm editing a file +whose name "sounds like" a Django file: + + :::text + au BufNewFile,BufRead admin.py setlocal filetype=python.django + au BufNewFile,BufRead urls.py setlocal filetype=python.django + au BufNewFile,BufRead models.py setlocal filetype=python.django + au BufNewFile,BufRead views.py setlocal filetype=python.django + au BufNewFile,BufRead settings.py setlocal filetype=python.django + au BufNewFile,BufRead forms.py setlocal filetype=python.django + +### Python Sanity Checking + +Lets be honest here: it takes a lot of work to turn Vim into an "IDE", and even then +it doesn't reach the level of something like Eclipse for Java. Anyone who claims it +has the same levels of integration and functionality is simply lying. + +With that said I'll make an opinionated statement that is going to piss some of you +off. + +**I am a programmer, not an IDE operator.** + +I know Python. + +I know Django. + +I don't need to hit Cmd+Space twice for every line of code I write. + +When someone asks me "how do you run your site" I do **not** answer: "click the green +triangle in Eclipse". + +However, I am human. I do stupid things like forgetting a colon or forgetting an +import. To help me with those problems I've turned to Kevin Watters' [Pyflakes +plugin][] for Vim. + +Pyflakes doesn't have IDE-level integration with your code. It doesn't check that +whatever libraries you `import` actually exist. It simply checks that your files are +probably-valid Python, and tells you when they're not. + +This is enough for me. It catches the stupid mistakes I make. The less-stupid, +more-subtle mistakes slip by it, but to be fair many of them would have slipped by an +"IDE" as well. + +[Pyflakes plugin]: http://www.vim.org/scripts/script.php?script_id=2441 + +### Javascript Sanity Checking and Folding + +If you haven't used [Syntastic][], you definitely need to check it out. It's a Vim +plugin that adds on-the-fly syntax-checking for many different file formats: one of +which is JavaScript. It's not perfect but it *will* catch things like using trailing +commas in object literals. + +Some people like using CTags to get an overview of their code. I take a more +low-tech approach and am in love with code folding. When I fold my code +I automatically get an overview of everything in each file. + +By default Vim doesn't fold Javascript files, but you can add some basic, perfectly +serviceable folding with these two lines in your .vimrc: + + :::text + au FileType javascript setlocal foldmethod=marker + au FileType javascript setlocal foldmarker={,} + +[Syntastic]: http://www.vim.org/scripts/script.php?script_id=2736 + + +### Django Autocommands + +I *rarely* work with raw HTML files any more. Whenever I open a file ending in +`.html` it's almost always a Django template (or a [Jinja][] template, which has +a very similar syntax). I've added an autocommand to automatically set the correct +filetyle whenever I open a `.html` file: + +[Jinja]: http://jinja.pocoo.org/ + + :::text + au BufNewFile,BufRead *.html setlocal filetype=htmldjango + +I also have some autocommands that tweak how a few specific files are handled: + + :::text + au BufNewFile,BufRead urls.py setlocal nowrap + au BufNewFile,BufRead settings.py normal! zR + au BufNewFile,BufRead dashboard.py normal! zR + +This automatically unfolds `urls.py`, `dashboard.py` and `settings.py` (I prefer +seeing those unfolded) and unsets line wrapping for `urls.py` (lines in a `urls.py` +file can get long and are hard to read when wrapped). + +Conclusion +---------- + +I hope that this longer-than-expected blog entry has given you at least one or two +things to think about. + +I've learned a lot while working with Django for Dumbwaiter every day, but I'm sure +there's still a lot I've missed. If you see something I could be doing better please +let me know! + +{% endblock %}