# HG changeset patch # User Steve Losh # Date 1263604511 18000 # Node ID e7d27299ea5421a3809e09f45914a84dd20805f6 # Parent a6628e0d5710eb6b3ad9a33dd4d8a7b07564cbdd Add the Hyde Rewrite entry. diff -r a6628e0d5710 -r e7d27299ea54 content/blog/2010/01/moving-from-django-to-hyde.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/content/blog/2010/01/moving-from-django-to-hyde.html Fri Jan 15 20:15:11 2010 -0500 @@ -0,0 +1,338 @@ +{% extends "_post.html" %} + +{% hyde + title: "Moving from Django to Hyde" + snip: "Another year, another rewrite." + created: 2010-01-13 21:14:00 +%} + +{% block article %} + +Almost exactly one year ago I posted a blog entry about how I [rewrote this +site][rewrite] using [Django][]. It's a new year, and once again I've rewritten +the site. + +If you've visited my site before you may have noticed some small style tweaks. +I've cleaned up a few things and I think the site looks better overall. The +biggest change, however, is that I completely rewrote the core -- this time +with [Hyde][]. + +Hyde is a "static site generator" written in [Python][], similar to +[Jekyll][], [Blatter][] and [Pilcrow][]. I chose it over the others because +it's written in Python (unlike Jekyll), more flexible than Blatter and more +mature than Pilcrow. + +Rewriting an existing site that gets a decent amount of visitors is different +than creating a new site from scratch, so I decided to write this entry to +describe some of the issues I ran into and how I tackled them. + +[Django]: {{links.django}} +[rewrite]: /blog/2009/01/site-redesign/ +[Hyde]: {{links.hyde}} +[Python]: {{links.python}} +[Jekyll]: http://jekyllrb.com/ +[Blatter]: {{links.blatter}} +[Pilcrow]: http://inky.github.com/pilcrow/ + +[TOC] + +Why Static? +----------- + +There are a couple of reasons I decided to switch to a static site. + +### Less Memory Needed + +I use [WebFaction][] for my web hosting, and with my current plan I have a limit on how much memory I can use at any given time. Each Django site takes up around 20mb of memory on the server. + +By switching to a static set of HTML pages for my site I save those 20mb so I can use them for something else (like a staging site for another project). My site doesn't particularly *need* all of the functionality Django can provide, so I figured I'd switch to a static site and save the memory. + +[WebFaction]: {{links.webfaction}} + +### Static Pages are Faster + +My site doesn't get an enormous amount of traffic, so the Django instances weren't really working very hard. Still, serving a static page through Apache or nginx is always going to be faster than running it through Django. Faster-loading pages is always a good thing, even if the speed increase isn't very large. + +### Easier to Maintain + +This is the main reason I switched. Previously, to edit a blog entry I would log in through Django's admin interface and edit the entry. With Hyde, I simply edit a file (or create a new one) on my machine and then run a single command to publish it. + +Another problem with the old site is that I needed to be connected to the internet to get to the admin interface, so I couldn't update entries (or publish new ones) offline. I usually have an internet connection, but occasionally I don't. Now I can edit as much as I like on my own machine and only need a connection to publish the finished product. + +### Easier to Backup + +One more advantage of Hyde is that the site structure *and* content are all held in the same place. I keep everything in a [Mercurial][] repository, and so every time I push that repository somewhere it creates a full backup of the site's code *and* content. If WebFaction's server catches on fire I still have everything on my local machine (and on BitBucket). + +I've toyed around with backing up Django's database tables when I had my old site, but this new method is far less work. + +[Mercurial]: {{links.mercurial}} + +Getting Started +--------------- + +I wanted a fresh start when I was rewriting the site, so I went ahead and created a brand new folder and Mercurial repository for it. + +I used `hyde --init` to lay out a skeleton in the new folder. Then I stripped out a lot of the default items that get added with `--init` and created the directory structure I wanted to use, with stubs for each of the main content files I knew I'd need. + +Finally, I went ahead and filled some of the basic values in the `settings.py` file. + +Layout and Styling +------------------ + +Once I had a skeleton in place I started working on layout of the site. + +### Cleanup + +The templates created by `hyde --init` are functional, but when you look at the code they're a mess. The indentation is strange and inconsistent and there's trailing whitespace all over the place. I like clean code so I sat down and cleaned everything up before I started making any real changes. + +### Rewriting the Layout and Styles + +After I finished cleaning up the templates I duplicated the HTML structure and styles of the old site from scratch. The old site had gone through a bunch of iterations and I was a bit sloppy in my editing, so there was a lot of cruft that had snuck in. I wanted a truly *fresh* start for the site, so I buckled down and did it all again. + +### A New CSS Framework + +Previously I used the [Blueprint CSS framework][blueprint] to make laying out the site easier. Blueprint is a great framework, but it's more powerful than I need for a site as simple as this one. The site now uses [Aardvark Legs][aal], which is a much simpler framework that simply sets up a great vertical rhythm and leaves you free to lay out the horizontal structure yourself. + +### Pagination + +Before the rewrite the list of blog entries used to be paginated. Hyde supports pagination but I decided against using it because I simply don't write enough to make it necessary. All the blog entries are now listed on a single page, which means that you can use Cmd+F to find an article if you're looking for something specific. + +[blueprint]: http://www.blueprintcss.org/ +[aal]: {{links.aarkvarklegs}} + +Using Fabric to Type Less +------------------------- + +I realized very quickly that typing `hyde -g -s . && hyde -w -s .` would get old pretty quickly, so I installed [Fabric][] and wrote a fabfile. + +Fabric is a tool written in Python that lets you define tasks and execute them by running `fab taskname`. Fabfiles are pure Python, so you can build larger tasks out of smaller ones very easily and do just about anything you want. It's similar to [ant][], but without the excessive over-engineering. + +You can view the [current fabfile][fabfile] on BitBucket. At the time of this entry it looks like this: + + :::python + from fabric.api import * + import os + import fabric.contrib.project as project + + PROD = 'sjl.webfactional.com' + DEST_PATH = '/home/sjl/webapps/slc/' + ROOT_PATH = os.path.abspath(os.path.dirname(__file__)) + DEPLOY_PATH = os.path.join(ROOT_PATH, 'deploy') + + def clean(): + local('rm -rf ./deploy') + + def regen(): + clean() + local('hyde -g -s .') + + def serve(): + local('hyde -w -s .') + + def reserve(): + regen() + serve() + + def smush(): + local('smusher ./media/images') + + @hosts(PROD) + def publish(): + regen() + project.rsync_project( + remote_dir=DEST_PATH, + local_dir=DEPLOY_PATH.rstrip('/') + '/', + delete=True + ) + +The task I use most often while developing is `fab reserve`, which regenerates the site and then starts serving it so I can view the result in a browser. + +I use `fab smush` whenever I add new images. This runs [smusher][] on all of the images to optimize them without reducing quality. + +When I'm ready to publish changes to the live site I run `fab publish`, which will regenerate my local version and copy it up to the WebFaction server. + +[Fabric]: {{links.fabric}} +[ant]: http://ant.apache.org/ +[fabfile]: http://bitbucket.org/sjl/stevelosh/src/tip/fabfile.py +[smusher]: http://github.com/grosser/smusher + +Moving the Content +------------------ + +The content of the old site (blog entries, projects, and static pages like the [about page][]) was fairly easy to migrate over to the new one, because both version use [Markdown][] to format the text. + +First I created an empty file for each page with a filename that matched the "slug" (last part of the URL) of the old page. Then I manually copied over the title, creation time and content for every page. I could have written a script to do this, but I don't have enough pages on the site to make it worth the time. + +[about page]: /about/ +[Markdown]: {{links.markdown}} + +Converting the Comments +----------------------- + +At this point the new site was looking very much like the old one. The styles were (roughly) matching and the blog posts, entries, and static pages were all rendered nicely. + +The next big task was migrating all of the comments. Since the new site is static it can't handle dynamic content like comments on its own, so I decided to use [Disqus][]. I use Disqus for the comments on the [hg tip][] site and it's very nice. For that site I used it from the beginning, but for this rewrite I needed to somehow import all the old comments. + +To migrate the comments over I used [django-disqus][], but there was a small snag that I needed to deal with first. + +When I first wrote the old site I was just getting back into Django after not using it for a long time. I didn't know about Django's built-in comment models, so I created my own. They weren't as good as Django's, but they did the job and I didn't care enough to change them. + +This became a problem when it was time to import the old comments into Disqus. django-disqus only supports importing comments from Django's built-in comment models. To work around this I first had to convert the old comments into Django's built-in ones. I wrote a [small, hacky Python script][convert-comments] to do it: + + :::python + #!/usr/bin/env python + + from django.core.management import setup_environ + import settings + setup_environ(settings) + + from markdown import Markdown + from django.contrib.comments.models import Comment + from django.contrib.sites.models import Site + from stevelosh.blog.models import Comment as BlogComment + from stevelosh.projects.models import Comment as ProjectComment + + + mdown = Markdown() + + site = Site.objects.all()[0] + blog_comments = BlogComment.objects.filter(spam=False) + project_comments = ProjectComment.objects.filter(spam=False) + + for bc in blog_comments: + c = Comment() + c.content_object = bc.entry + c.user_name = bc.name + c.comment = mdown.convert(bc.body) + c.submit_date = bc.submitted + c.site = site + c.is_public = True + c.is_removed = False + c.save() + # print 'http://%s%s' % (site.domain, c.content_object.get_absolute_url()) + + for pc in project_comments: + c = Comment() + c.content_object = pc.project + c.user_name = pc.name + c.comment = mdown.convert(pc.body) + c.submit_date = pc.submitted + c.site = site + c.is_public = True + c.is_removed = False + c.save() + # print 'http://%s%s' % (site.domain, c.content_object.get_absolute_url()) + +Yes, it's ugly. No, I don't care. It was run once or twice locally and once on the live server. It worked and I'll never need to run it again. + +Once I had the comments converted I could use django-disqus to migrate them. The import went very smoothly -- I ran one command and after a couple of minutes everything was in Disqus. Once that finished it was just a matter of adding the Disqus JavaScript to one of the templates. + +[Disqus]: http://disqus.com/ +[hg tip]: {{links.hgtip}} +[django-disqus]: http://github.com/arthurk/django-disqus +[convert-comments]: http://bitbucket.org/sjl/stevelosh/src/da98105753a1/convert-comments.py + +Rewriting the Old URLs +---------------------- + +Since I was pretty much starting from scratch with this rewrite I decided to clean up the URL structure of my blog. Previously a blog entry's URL looked like this: + + :::text + http://stevelosh.com/blog/entry/2009/08/30/a-guide-to-branching-in-mercurial/ + +The `/entry/` part of the URL was useless -- it just took up space, so I got rid of it. The year and month are useful to get an idea of how old an entry is just by looking at the link, so I left them in. The day, however, probably doesn't matter that much, so I took it out. + +The new URLs look like this: + + :::text + http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/ + +The problem with rewriting the URL structure is that there are already links around the web pointing at my entries. I didn't want those old links to break, so I crafted an [`.htaccess` file][htaccess] that would rewrite the old URLs into the new ones: + + {% templatetag openblock %} if GENERATE_CLEAN_URLS {% templatetag closeblock %} + RewriteEngine on + RewriteBase {% templatetag openvariable %} node.site.settings.SITE_ROOT {% templatetag closevariable %} + + # Old URLs + RewriteRule ^blog/entry/(\d+)/(\d\d)/\d+/([^/]*)/?$ /blog/$1/$2/$3/ [R=301,L] + RewriteRule ^blog/entry/(\d+)/(\d)/\d+/([^/]*)/?$ /blog/$1/0$2/$3/ [R=301,L] + + {% templatetag openblock %} hyde_listing_page_rewrite_rules {% templatetag closeblock %} + + # listing pages whose names are the same as their enclosing folder's + RewriteCond %{REQUEST_FILENAME}/$1.html -f + RewriteRule ^([^/]*)/$ %{REQUEST_FILENAME}/$1.html + + # regular pages + RewriteCond %{REQUEST_FILENAME}.html -f + RewriteRule ^.*$ %{REQUEST_FILENAME}.html + + {% templatetag openblock %} endif {% templatetag closeblock %} + +Most of that file is the stock Hyde `.htaccess` file -- the two lines under `# Old URLs` are the ones that I added. It took me a while to get it right because I don't work with Apache's `mod_rewrite` very often, but it was worth it to avoid breaking all of the old links. + +[htaccess]: http://bitbucket.org/sjl/stevelosh/src/tip/content/.htaccess + +Creating an RSS Feed +-------------------- + +I had an RSS feed for the old site which Django made very easy to set up. I definitely needed a feed for the new site, and fortunately Hyde provides a simple sample template that can be used to make an ATOM feed. + +I cleaned up the whitespace and formatting of that template a bit, adjusted a few variables for my needs, put it on [FeedBurner][] and everything was ready. + +[FeedBurner]: http://www.feedburner.com/ + +Merging the Repositories +------------------------ + +The last step before I was finished was to merge the old and new repositories together so I wouldn't lose any of the history. It's probably not *too* important to keep the old site's history around, but I put a lot of work into it over the past year and it has some sentimental value. + +Fortunately it's very easy to [combine Mercurial repositories][combinerepos]. I just pulled the old repository into the new one, merged the old head into the new one while discarding all the changes, and pushed it up to BitBucket. + +[combinerepos]: http://hgtip.com/tips/advanced/2009-11-17-combining-repositories/ + +Still to Come +------------- + +At this point I felt the site was ready to be released, so I set up a new site on WebFaction and used Fabric to deploy it. + +I'm very happy with the result, but there are still a few things I'm going to fix/change in the future. + +### "Ago" Dates + +If you look at the top of each blog entry and project there's a line beneath the title that looks something like: + + :::text + Posted on Monday, November 16, 2009 (1 month, 3 weeks ago). + +The `(1 month, 3 weeks ago)` part of that line is something that I really appreciate on blogs. When they just list the date I always have to do the math in my head to figure out roughly how old something is. + +With a static site, however, those times will quickly become inaccurate if I don't regenerate and publish the site regularly. I'm still thinking about the best way to work this out. + +One option is to use a cron job on the WebFaction server to regenerate the site every day, which would keep the times *fairly* accurate. + +Another option would be to use a bit of JavaScript to calculate and render the time when the page is loaded. This would make it *completely* accurate but wouldn't work if someone is browsing with JavaScript turned off. + +### Categories + +Right now all the blog entries (and projects) are listed in a single chronological list. It would be great to break them up into categories so people can easily find the articles they're interested in. + +Hyde supports categories but I haven't spent the time to learn how to use them yet. I also need to figure out a way to work a list of categories into the design without cluttering things up too much. + +### Use LessCSS + +[LessCSS][] is a language that extends CSS with some useful features like variables, mixins, operations and nested rules. It can make the styles of a site much, much cleaner. + +Hyde includes a LessCSS "processor" that will automatically render your LessCSS files into normal CSS. I'm planning on rewriting the site's styles in LessCSS and using the processor once I get some free time. + +[lesscss]: {{links.lesscss}} + +View the Code +------------- + +The code is on [BitBucket][bb-repo] and [GitHub][gh-repo]. Feel free to poke around and see how I've set things up. If you have questions or suggestions I'd love to hear them! + +[bb-repo]: http://bitbucket.org/sjl/stevelosh/ +[gh-repo]: http://github.com/sjl/stevelosh/ + +{% endblock %} \ No newline at end of file