--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/content/blog/2010/01/moving-from-django-to-hyde.html Fri Jan 15 20:15:11 2010 -0500
@@ -0,0 +1,338 @@
+{% extends "_post.html" %}
+
+{% hyde
+ title: "Moving from Django to Hyde"
+ snip: "Another year, another rewrite."
+ created: 2010-01-13 21:14:00
+%}
+
+{% block article %}
+
+Almost exactly one year ago I posted a blog entry about how I [rewrote this
+site][rewrite] using [Django][]. It's a new year, and once again I've rewritten
+the site.
+
+If you've visited my site before you may have noticed some small style tweaks.
+I've cleaned up a few things and I think the site looks better overall. The
+biggest change, however, is that I completely rewrote the core -- this time
+with [Hyde][].
+
+Hyde is a "static site generator" written in [Python][], similar to
+[Jekyll][], [Blatter][] and [Pilcrow][]. I chose it over the others because
+it's written in Python (unlike Jekyll), more flexible than Blatter and more
+mature than Pilcrow.
+
+Rewriting an existing site that gets a decent amount of visitors is different
+than creating a new site from scratch, so I decided to write this entry to
+describe some of the issues I ran into and how I tackled them.
+
+[Django]: {{links.django}}
+[rewrite]: /blog/2009/01/site-redesign/
+[Hyde]: {{links.hyde}}
+[Python]: {{links.python}}
+[Jekyll]: http://jekyllrb.com/
+[Blatter]: {{links.blatter}}
+[Pilcrow]: http://inky.github.com/pilcrow/
+
+[TOC]
+
+Why Static?
+-----------
+
+There are a couple of reasons I decided to switch to a static site.
+
+### Less Memory Needed
+
+I use [WebFaction][] for my web hosting, and with my current plan I have a limit on how much memory I can use at any given time. Each Django site takes up around 20mb of memory on the server.
+
+By switching to a static set of HTML pages for my site I save those 20mb so I can use them for something else (like a staging site for another project). My site doesn't particularly *need* all of the functionality Django can provide, so I figured I'd switch to a static site and save the memory.
+
+[WebFaction]: {{links.webfaction}}
+
+### Static Pages are Faster
+
+My site doesn't get an enormous amount of traffic, so the Django instances weren't really working very hard. Still, serving a static page through Apache or nginx is always going to be faster than running it through Django. Faster-loading pages is always a good thing, even if the speed increase isn't very large.
+
+### Easier to Maintain
+
+This is the main reason I switched. Previously, to edit a blog entry I would log in through Django's admin interface and edit the entry. With Hyde, I simply edit a file (or create a new one) on my machine and then run a single command to publish it.
+
+Another problem with the old site is that I needed to be connected to the internet to get to the admin interface, so I couldn't update entries (or publish new ones) offline. I usually have an internet connection, but occasionally I don't. Now I can edit as much as I like on my own machine and only need a connection to publish the finished product.
+
+### Easier to Backup
+
+One more advantage of Hyde is that the site structure *and* content are all held in the same place. I keep everything in a [Mercurial][] repository, and so every time I push that repository somewhere it creates a full backup of the site's code *and* content. If WebFaction's server catches on fire I still have everything on my local machine (and on BitBucket).
+
+I've toyed around with backing up Django's database tables when I had my old site, but this new method is far less work.
+
+[Mercurial]: {{links.mercurial}}
+
+Getting Started
+---------------
+
+I wanted a fresh start when I was rewriting the site, so I went ahead and created a brand new folder and Mercurial repository for it.
+
+I used `hyde --init` to lay out a skeleton in the new folder. Then I stripped out a lot of the default items that get added with `--init` and created the directory structure I wanted to use, with stubs for each of the main content files I knew I'd need.
+
+Finally, I went ahead and filled some of the basic values in the `settings.py` file.
+
+Layout and Styling
+------------------
+
+Once I had a skeleton in place I started working on layout of the site.
+
+### Cleanup
+
+The templates created by `hyde --init` are functional, but when you look at the code they're a mess. The indentation is strange and inconsistent and there's trailing whitespace all over the place. I like clean code so I sat down and cleaned everything up before I started making any real changes.
+
+### Rewriting the Layout and Styles
+
+After I finished cleaning up the templates I duplicated the HTML structure and styles of the old site from scratch. The old site had gone through a bunch of iterations and I was a bit sloppy in my editing, so there was a lot of cruft that had snuck in. I wanted a truly *fresh* start for the site, so I buckled down and did it all again.
+
+### A New CSS Framework
+
+Previously I used the [Blueprint CSS framework][blueprint] to make laying out the site easier. Blueprint is a great framework, but it's more powerful than I need for a site as simple as this one. The site now uses [Aardvark Legs][aal], which is a much simpler framework that simply sets up a great vertical rhythm and leaves you free to lay out the horizontal structure yourself.
+
+### Pagination
+
+Before the rewrite the list of blog entries used to be paginated. Hyde supports pagination but I decided against using it because I simply don't write enough to make it necessary. All the blog entries are now listed on a single page, which means that you can use Cmd+F to find an article if you're looking for something specific.
+
+[blueprint]: http://www.blueprintcss.org/
+[aal]: {{links.aarkvarklegs}}
+
+Using Fabric to Type Less
+-------------------------
+
+I realized very quickly that typing `hyde -g -s . && hyde -w -s .` would get old pretty quickly, so I installed [Fabric][] and wrote a fabfile.
+
+Fabric is a tool written in Python that lets you define tasks and execute them by running `fab taskname`. Fabfiles are pure Python, so you can build larger tasks out of smaller ones very easily and do just about anything you want. It's similar to [ant][], but without the excessive over-engineering.
+
+You can view the [current fabfile][fabfile] on BitBucket. At the time of this entry it looks like this:
+
+ :::python
+ from fabric.api import *
+ import os
+ import fabric.contrib.project as project
+
+ PROD = 'sjl.webfactional.com'
+ DEST_PATH = '/home/sjl/webapps/slc/'
+ ROOT_PATH = os.path.abspath(os.path.dirname(__file__))
+ DEPLOY_PATH = os.path.join(ROOT_PATH, 'deploy')
+
+ def clean():
+ local('rm -rf ./deploy')
+
+ def regen():
+ clean()
+ local('hyde -g -s .')
+
+ def serve():
+ local('hyde -w -s .')
+
+ def reserve():
+ regen()
+ serve()
+
+ def smush():
+ local('smusher ./media/images')
+
+ @hosts(PROD)
+ def publish():
+ regen()
+ project.rsync_project(
+ remote_dir=DEST_PATH,
+ local_dir=DEPLOY_PATH.rstrip('/') + '/',
+ delete=True
+ )
+
+The task I use most often while developing is `fab reserve`, which regenerates the site and then starts serving it so I can view the result in a browser.
+
+I use `fab smush` whenever I add new images. This runs [smusher][] on all of the images to optimize them without reducing quality.
+
+When I'm ready to publish changes to the live site I run `fab publish`, which will regenerate my local version and copy it up to the WebFaction server.
+
+[Fabric]: {{links.fabric}}
+[ant]: http://ant.apache.org/
+[fabfile]: http://bitbucket.org/sjl/stevelosh/src/tip/fabfile.py
+[smusher]: http://github.com/grosser/smusher
+
+Moving the Content
+------------------
+
+The content of the old site (blog entries, projects, and static pages like the [about page][]) was fairly easy to migrate over to the new one, because both version use [Markdown][] to format the text.
+
+First I created an empty file for each page with a filename that matched the "slug" (last part of the URL) of the old page. Then I manually copied over the title, creation time and content for every page. I could have written a script to do this, but I don't have enough pages on the site to make it worth the time.
+
+[about page]: /about/
+[Markdown]: {{links.markdown}}
+
+Converting the Comments
+-----------------------
+
+At this point the new site was looking very much like the old one. The styles were (roughly) matching and the blog posts, entries, and static pages were all rendered nicely.
+
+The next big task was migrating all of the comments. Since the new site is static it can't handle dynamic content like comments on its own, so I decided to use [Disqus][]. I use Disqus for the comments on the [hg tip][] site and it's very nice. For that site I used it from the beginning, but for this rewrite I needed to somehow import all the old comments.
+
+To migrate the comments over I used [django-disqus][], but there was a small snag that I needed to deal with first.
+
+When I first wrote the old site I was just getting back into Django after not using it for a long time. I didn't know about Django's built-in comment models, so I created my own. They weren't as good as Django's, but they did the job and I didn't care enough to change them.
+
+This became a problem when it was time to import the old comments into Disqus. django-disqus only supports importing comments from Django's built-in comment models. To work around this I first had to convert the old comments into Django's built-in ones. I wrote a [small, hacky Python script][convert-comments] to do it:
+
+ :::python
+ #!/usr/bin/env python
+
+ from django.core.management import setup_environ
+ import settings
+ setup_environ(settings)
+
+ from markdown import Markdown
+ from django.contrib.comments.models import Comment
+ from django.contrib.sites.models import Site
+ from stevelosh.blog.models import Comment as BlogComment
+ from stevelosh.projects.models import Comment as ProjectComment
+
+
+ mdown = Markdown()
+
+ site = Site.objects.all()[0]
+ blog_comments = BlogComment.objects.filter(spam=False)
+ project_comments = ProjectComment.objects.filter(spam=False)
+
+ for bc in blog_comments:
+ c = Comment()
+ c.content_object = bc.entry
+ c.user_name = bc.name
+ c.comment = mdown.convert(bc.body)
+ c.submit_date = bc.submitted
+ c.site = site
+ c.is_public = True
+ c.is_removed = False
+ c.save()
+ # print 'http://%s%s' % (site.domain, c.content_object.get_absolute_url())
+
+ for pc in project_comments:
+ c = Comment()
+ c.content_object = pc.project
+ c.user_name = pc.name
+ c.comment = mdown.convert(pc.body)
+ c.submit_date = pc.submitted
+ c.site = site
+ c.is_public = True
+ c.is_removed = False
+ c.save()
+ # print 'http://%s%s' % (site.domain, c.content_object.get_absolute_url())
+
+Yes, it's ugly. No, I don't care. It was run once or twice locally and once on the live server. It worked and I'll never need to run it again.
+
+Once I had the comments converted I could use django-disqus to migrate them. The import went very smoothly -- I ran one command and after a couple of minutes everything was in Disqus. Once that finished it was just a matter of adding the Disqus JavaScript to one of the templates.
+
+[Disqus]: http://disqus.com/
+[hg tip]: {{links.hgtip}}
+[django-disqus]: http://github.com/arthurk/django-disqus
+[convert-comments]: http://bitbucket.org/sjl/stevelosh/src/da98105753a1/convert-comments.py
+
+Rewriting the Old URLs
+----------------------
+
+Since I was pretty much starting from scratch with this rewrite I decided to clean up the URL structure of my blog. Previously a blog entry's URL looked like this:
+
+ :::text
+ http://stevelosh.com/blog/entry/2009/08/30/a-guide-to-branching-in-mercurial/
+
+The `/entry/` part of the URL was useless -- it just took up space, so I got rid of it. The year and month are useful to get an idea of how old an entry is just by looking at the link, so I left them in. The day, however, probably doesn't matter that much, so I took it out.
+
+The new URLs look like this:
+
+ :::text
+ http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/
+
+The problem with rewriting the URL structure is that there are already links around the web pointing at my entries. I didn't want those old links to break, so I crafted an [`.htaccess` file][htaccess] that would rewrite the old URLs into the new ones:
+
+ {% templatetag openblock %} if GENERATE_CLEAN_URLS {% templatetag closeblock %}
+ RewriteEngine on
+ RewriteBase {% templatetag openvariable %} node.site.settings.SITE_ROOT {% templatetag closevariable %}
+
+ # Old URLs
+ RewriteRule ^blog/entry/(\d+)/(\d\d)/\d+/([^/]*)/?$ /blog/$1/$2/$3/ [R=301,L]
+ RewriteRule ^blog/entry/(\d+)/(\d)/\d+/([^/]*)/?$ /blog/$1/0$2/$3/ [R=301,L]
+
+ {% templatetag openblock %} hyde_listing_page_rewrite_rules {% templatetag closeblock %}
+
+ # listing pages whose names are the same as their enclosing folder's
+ RewriteCond %{REQUEST_FILENAME}/$1.html -f
+ RewriteRule ^([^/]*)/$ %{REQUEST_FILENAME}/$1.html
+
+ # regular pages
+ RewriteCond %{REQUEST_FILENAME}.html -f
+ RewriteRule ^.*$ %{REQUEST_FILENAME}.html
+
+ {% templatetag openblock %} endif {% templatetag closeblock %}
+
+Most of that file is the stock Hyde `.htaccess` file -- the two lines under `# Old URLs` are the ones that I added. It took me a while to get it right because I don't work with Apache's `mod_rewrite` very often, but it was worth it to avoid breaking all of the old links.
+
+[htaccess]: http://bitbucket.org/sjl/stevelosh/src/tip/content/.htaccess
+
+Creating an RSS Feed
+--------------------
+
+I had an RSS feed for the old site which Django made very easy to set up. I definitely needed a feed for the new site, and fortunately Hyde provides a simple sample template that can be used to make an ATOM feed.
+
+I cleaned up the whitespace and formatting of that template a bit, adjusted a few variables for my needs, put it on [FeedBurner][] and everything was ready.
+
+[FeedBurner]: http://www.feedburner.com/
+
+Merging the Repositories
+------------------------
+
+The last step before I was finished was to merge the old and new repositories together so I wouldn't lose any of the history. It's probably not *too* important to keep the old site's history around, but I put a lot of work into it over the past year and it has some sentimental value.
+
+Fortunately it's very easy to [combine Mercurial repositories][combinerepos]. I just pulled the old repository into the new one, merged the old head into the new one while discarding all the changes, and pushed it up to BitBucket.
+
+[combinerepos]: http://hgtip.com/tips/advanced/2009-11-17-combining-repositories/
+
+Still to Come
+-------------
+
+At this point I felt the site was ready to be released, so I set up a new site on WebFaction and used Fabric to deploy it.
+
+I'm very happy with the result, but there are still a few things I'm going to fix/change in the future.
+
+### "Ago" Dates
+
+If you look at the top of each blog entry and project there's a line beneath the title that looks something like:
+
+ :::text
+ Posted on Monday, November 16, 2009 (1 month, 3 weeks ago).
+
+The `(1 month, 3 weeks ago)` part of that line is something that I really appreciate on blogs. When they just list the date I always have to do the math in my head to figure out roughly how old something is.
+
+With a static site, however, those times will quickly become inaccurate if I don't regenerate and publish the site regularly. I'm still thinking about the best way to work this out.
+
+One option is to use a cron job on the WebFaction server to regenerate the site every day, which would keep the times *fairly* accurate.
+
+Another option would be to use a bit of JavaScript to calculate and render the time when the page is loaded. This would make it *completely* accurate but wouldn't work if someone is browsing with JavaScript turned off.
+
+### Categories
+
+Right now all the blog entries (and projects) are listed in a single chronological list. It would be great to break them up into categories so people can easily find the articles they're interested in.
+
+Hyde supports categories but I haven't spent the time to learn how to use them yet. I also need to figure out a way to work a list of categories into the design without cluttering things up too much.
+
+### Use LessCSS
+
+[LessCSS][] is a language that extends CSS with some useful features like variables, mixins, operations and nested rules. It can make the styles of a site much, much cleaner.
+
+Hyde includes a LessCSS "processor" that will automatically render your LessCSS files into normal CSS. I'm planning on rewriting the site's styles in LessCSS and using the processor once I get some free time.
+
+[lesscss]: {{links.lesscss}}
+
+View the Code
+-------------
+
+The code is on [BitBucket][bb-repo] and [GitHub][gh-repo]. Feel free to poke around and see how I've set things up. If you have questions or suggestions I'd love to hear them!
+
+[bb-repo]: http://bitbucket.org/sjl/stevelosh/
+[gh-repo]: http://github.com/sjl/stevelosh/
+
+{% endblock %}
\ No newline at end of file