Review: Hacking Vim

Introduction

Some of my faithful visitors may have noticed that I have a thing for Vim, one of the oldest and most powerful text editors in the world. In the past 15 or so years that I've been developing, I have spent quite a bit of time in several different text editors. It seemed like I was continually on the quest to find the fastest, most feature-packed editor out there, while still being cross-platform compatible and having it stay out of my way. Speed has always been very important to me.

I have been using Vi and Vim regularly since about 2000, when I began dabbling with Linux. I could certainly hold my ground in either of the two programs, but I was by no means proficient. The more appealing text editors for me offered syntax highlighting and code completion. At the time, I was under the impression that Vi/Vim didn't offer either of these two features. It wasn't until around the middle of last year, however, that I really started putting effort into learning and using Vim. After asking some of my Vim-savvy friends a lot of questions to get me kickstarted, I began to see the power that lies in Vim.

Before long, Vim had replaced all other text editors as my preferred editing environment. I learned that Vim could satisfy just able every single one of my personal qualifications for the perfect editor. I dumped all other editors in favor of Vim, and I even opted to use Vim over a several hundred dollar IDE at work.

Anyway. I received a review copy of Kim Schulz' "Hacking Vim: A cookbook to get the most out of the latest Vim editor" a couple of months ago and have been rummaging through it since then. I have learned a ton of fantastic tips from this little book! Being a cookbook, you're not expected to read the entire book start to finish. Rather, you can dig right into whatever section interests you and feel right at home.

Brief Overview

Packt Publishing printed this book back in 2007, but all of the tips are still very much up-to-date. The book starts off with the obligatory history lesson (which is actually quite interesting if you're a nerd like me), and the target audience is described as such:

New users join the Vim user community every day and want to use this editor in their daily work, and even though Vim sometimes can be complex to use, they still favor it above other editors. This is a book for these Vim users.

After the history lesson, chapter 2 of the book digs right into personalizing Vim to fit your own preferences. Topics covered include:

  • changing fonts
  • changing color schemes
  • personalizing highlighting
  • customizing the status line
  • toggling menus and toolbars in gvim
  • adding your own menu items and toolbar buttons
  • customizing your work area

Chapter 3 discusses better navigation techniques. Topics covered include:

  • faster navigation in a file
  • faster navigation in the Vim help system
  • faster navigation in multiple buffers
  • in-file searching
  • searching in multiple files or buffers
  • using marks and signs

Chapter 4, titled "Production Boosters" discusses the following:

  • templates using simple template file
  • templates using abbreviations
  • auto-completion using known words and tag lists
  • auto-completion using omni-completion
  • macros
  • sessions
  • registers and undo branches
  • folding
  • vimdiff
  • opening remote files using Netrw

Chapter 5 introduces some advanced formatting tips. You can learn how to put text into nicely-formatted paragraphs, aligning text, marking headlines, and creating lists. For code, this chapter discusses several different indentation options.

Vim scripting is the topic of chapter 6, and Schulz covers a wide variety of useful tips to get anyone started on scripting Vim to do their bidding. Tips include:

  • creating syntax-coloring scripts
  • how to install and use scripts
  • different types of scripts
  • basic syntax of Vim scripts
  • how to structure Vim scripts
  • debugging a Vim script
  • using other scripting languages (Perl, Python, Ruby)

Appendix A describes how Vim can be used for much more than just text editing. Several different games, including Tetris and a Rubik's Cube are briefly introduced, along with how to use Vim as a mail client or programmer's IDE. Appendix B suggests miscellaneous configuration script maintenance tips, such as how you can maintain the same configuration script across several different machines.

My Thoughts

I was very impressed with this book. I was afraid that, being published in 2007, it might be a little too out-of-date for my personal tastes. Since the book is about Vim, though, I wasn't overly concerned (the editor has been around for decades, and it doesn't change drastically from release to release anymore).

Just like the last book I reviewed, I found several typos in this book. A lot of the typos were in the first few pages of the actual content, and some were definitely more minor than others. This sort of thing doesn't really detract much from the material covered, but it sure does stand out as a distraction for people who pay attention to details.

Here are some of the things that I truly enjoyed reading and learning about (many of which actually made my jaw drop in awe of Vim)

  • Specifying multiple fonts for GVim, just in case your first choice isn't always available:

    :set guifont=Courier\ New\ 12, Arial\ 10
    
  • Specifying different font faces based on the extension of the file you're editing:

    :autocmd BufEnter *.txt set guifont=Arial\ 12
    
  • Highlighting the line your cursor is currently on, and the column the cursor is in:

    :set cursorline
    :set cursorcolumn
    
  • Limiting the number of suggestions that the spell checker offers:

    :set spellsuggest=5
    
  • Navigating to different words based on whitespace instead of "regular" word separators:

    • W to move to the beginning of the next word
    • B to move to the beginning of the previous word
    • E to move to the beginning of the previous word

    I knew about the lowercase variations of these commands, but not the uppercase.

  • Navigating up and down in the same long, wrapped line:

    gk
    gj
    
  • Opening a file that is referenced in the current buffer:

    gf
    

    I learned that this even works on Python imports! Just like the description says, it will work on the import module, not classes or other objects from inside the module. Not quite that intelligent!

  • Incremental searching:

    :set incsearch
    
  • Searching up/down in a buffer for any occurrence of the word under the cursor:

    g#
    g*
    

    I knew about the usual # and *, but those two will only match the same exact word. When they're prefixed with g, they will match any occurrence of the word, be it whole or part of another word. For example, hitting g* while the cursor is over the word foo would would match both food and foobar, while * would match neither.

  • Using markers to jump between specific points in different open buffers (mA through mZ)

  • Prepopulating empty files based on their extension:

    :autocmd BufNewFile * silent! 0r $VIMHOME/templates/%:e.tpl
    
  • Formatting a paragraph of text:

    gqap
    
  • Formatting all paragraphs of text in a file:

    1gqG
    
  • Smart indentation:

    :set smartindent
    
  • Enabling paste mode, so smartindent doesn't try to format code that you paste into your buffer:

    :set paste
    
  • Prettifying XML and HTML using Tidy:

    :autocmd FileType xml exe ":silent 1,$!tidy --input-xml true --indent yes -q"
    :autocmd FileType html,htm exe ":silent 1,$!tidy --indent yes -q"
    

Conclusion

All in all, this is a fantastic book. I will be keeping it near my workstation as a quick reference book when I want to do something crazy with Vim. I've already recommended the book to several of my friends and acquaintances, and I will make the same recommendation here. If you are mildly familiar with Vim and at all interested in getting more out of this fabulous editor, I highly recommend picking up a copy of this book.

How I Have A Mobile & Desktop Site With Django

Part of the latest version of my site included the deployment of a mobile-friendly site. Up until recently, I hadn't even attempted to create a mobile site because I thought it would take more time than it was worth. I wanted something beyond just using CSS to hide certain elements on the page. I wanted to be able to break down the content of my site into its most basic pieces and only include what was necessary. Also, I wanted to figure it out on my own (instead of reusing wheels other people had invented before me--horrible, I know).

With these requirements, I was afraid it would require more resources than I could spare on my shared Web host. My initial impression was that I would have to leverage the django.contrib.sites framework in a fashion that would essentially require two distinct instances of my site running in RAM. Despite these feelings, I decided to embark on a mission to create a mobile-friendly site while still offering a full desktop-friendly site. It was surprisingly simple. This may not be the best way to do it, but it sure works for me, and I'm very satisfied. So satisfied, in fact, that I am going to share my solution with all of my Django-loving friends.

The first step is to add a couple of new settings to your settings.py file:

import os
DIRNAME = os.path.abspath(os.path.dirname(__file__))

TEMPLATE_DIRS = (
    os.path.join(DIRNAME, 'templates'),
)

MOBILE_TEMPLATE_DIRS = (
    os.path.join(DIRNAME, 'templates', 'mobile'),
)
DESKTOP_TEMPLATE_DIRS = (
    os.path.join(DIRNAME, 'templates', 'desktop'),
)

For those of you not used to seeing that os.path.join stuff, it's just a (very efficient) way to make your Django project more portable between different computers and even operating systems. The new variables are MOBILE_TEMPLATE_DIRS and DESKTOP_TEMPLATE_DIRS, and their respective meanings should be fairly obvious. Basically, this tells Django that it can look for templates in your_django_project/templates, your_django_project/templates/mobile, and your_django_project/templates/desktop.

Next, we need to install a middleware that takes care of determining which directory Django should pay attention to when rendering pages, between mobile and desktop. You can put this into your_django_project/middleware.py:

from django.conf import settings

class MobileTemplatesMiddleware(object):
    """Determines which set of templates to use for a mobile site"""

    ORIG_TEMPLATE_DIRS = settings.TEMPLATE_DIRS

    def process_request(self, request):
        # sets are used here, you can use other logic if you have an older version of Python
        MOBILE_SUBDOMAINS = set(['m', 'mobile'])
        domain = set(request.META.get('HTTP_HOST', '').split('.'))

        if len(MOBILE_SUBDOMAINS & domain):
            settings.TEMPLATE_DIRS = settings.MOBILE_TEMPLATE_DIRS + self.ORIG_TEMPLATE_DIRS
        else:
            settings.TEMPLATE_DIRS = settings.DESKTOP_TEMPLATE_DIRS + self.ORIG_TEMPLATE_DIRS

Now you need to install the new middleware. Back in your settings.py, find the MIDDLEWARE_CLASSES variable, and insert a line like the following:

'your_django_project.middleware.MobileTemplatesMiddleware',

Finally, if you already have a base.html template in your your_django_project/templates directory, rename it to something else, such as site_base.html. Now create two new directories: your_django_project/templates/mobile and your_django_project/templates/desktop. In both of those directories, create a new base.html template that extends site_base.html.

Example site_base.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>{% block base_title %}Code Koala{% endblock %} - {% block title %}Welcome{% endblock %}</title>
<link href="{{ MEDIA_URL }}css/common.css" rel="stylesheet" type="text/css" media="screen" />
{% block extra-head %}{% endblock %}

</head>
<body>
<div id="page-wrapper">
    {% block header %}
    <div id="logo">
        <h1><a href="/">Code Koala</a></h1>
    </div>
    <div id="header">
        <div id="menu">
            <ul>
                <li><a href="/" class="first">Home</a></li>
                <li><a href="/blog/">Blog</a></li>
                <li><a href="/about/">About</a></li>
                <li><a href="/contact/">Contact</a></li>
            </ul>
        </div>
    </div>
    {% endblock %}
    <div id="page">
        <div id="content">
            {% block content %}{% endblock %}
        </div>
        <div id="sidebar">
            {% block sidebar %}
            Stuff
            {% endblock %}
        </div>
    </div>
    <div id="footer">
        {% block footer %}
        Footer stuff
        {% endblock %}
    </div>
</div>
</body>
</html>

Example desktop/base.html

{% extends 'site_base.html' %}

{% block extra-head %}
<!-- stylesheets -->
<link href="{{ MEDIA_URL }}css/desktop.css" rel="stylesheet" type="text/css" media="screen" />

<!-- JavaScripts -->
<script type="text/javascript" src="{{ MEDIA_URL }}js/jquery.js"></script>
{% endblock %}

Example mobile/base.html

{% extends 'site_base.html' %}

{% block extra-head %}
<!-- stylesheets -->
<link href="{{ MEDIA_URL }}css/mobile.css" rel="stylesheet" type="text/css" media="screen" />
{% endblock %}

{% block sidebar %}{% endblock %}

Please forgive me if the HTML or whatever is incorrect--I butchered the actual templates I use on Code Koala for the examples. There are some neat things you can do in your pages to make them more mobile friendly, such as including something like <meta name="viewport" content="width=device-width; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;" /> in your <head> tag. This is supposed to tell your visitor's browser to not scale your pages to make it all fit on the screen. You can find a lot of other such tips elsewhere on the interwebs, and I'm sure they'll be better explained elsewhere too. You can also find scripts to handle redirecting your visitors to a mobile site and whatnot. Google is your friend.

As for the Django side of things, that should be just about it. If you have other templates you want to customize based on the version of your site that visitors are viewing, simply add those templates to the your_django_project/templates/mobile or your_django_project/templates/desktop directories as necessary. For example, if you have an application called blog, and you want to override the entry_detail.html template for the mobile site, so it doesn't pull in a bunch of unnecessary information to save bandwidth, you could save your modified copy in your_django_project/templates/mobile/blog/entry_detail.html.

With this setup, all you have to do is point your main domain and a subdomain such as m.yourdomain.com to the same Django application, and the middleware will take care of the "heavy lifting". No need for an additional instance of your Django project just for the mobile site. No hackish hiding of elements using CSS. If you find this article useful and decide to use these techniques on your site, please let me know how it works in your environment and if you ran into any snags so I can update the information!

Auto-Generating Documentation Using Mercurial, ReST, and Sphinx

I often find myself taking notes about various aspects of my job that I feel I would forget as soon as I moved onto another project. I've gotten into the habit of taking my notes using reStructured Text, which shouldn't come as any surprise to any of my regular visitors. On several occasions, I had some of the other guys in the company ask me for some clarification on some things I had taken notes on. Lucky for me, I had taken some nice notes!

However, these individuals probably wouldn't appreciate reading ReST markup as much as I do, so I decided to do something nice for them. I setup Sphinx to prettify my documentation. I then wrote a small Web server using Python, so people within the company network could access the latest version of my notes without much hassle.

Just like I take notes to remind myself of stuff at work, I want to do that again for this automated ReST->HTML magic--I want to be able to do this in the future! I figured I would make my notes even more public this time, so you all can enjoy similar bliss.

Platform Dependence

I am writing this article with UNIX-like operating systems in mind. Please forgive me if you're a Windows user and some of this is not consistent with what you're seeing. Perhaps one day I'll try to set this sort of thing up on Windows.

Installing Sphinx

The first step that we want to take is installing Sphinx. This is the project that Python itself uses to generate its online documentation. It's pretty dang awesome. Feel free to skip this section if you have already installed Sphinx.

Depending on your environment of choice, you may or may not have a package manager that offers python-sphinx or something along those lines. I personally prefer to install it using pip or easy_install:

$ sudo pip install sphinx

Running that command will likely respond with a bunch of output about downloading Sphinx and various dependencies. When I ran it in my sandbox VM, I saw it install the following packages:

  • pygments
  • jinja2
  • docutils
  • sphinx

It should be a pretty speedy installation.

Installing Mercurial

We'll be using Mercurial to keep track of changes to our ReST documentation. Mercurial is a distributed version control system that is built using Python. It's wonderful! Just like with Sphinx, if you have already installed Mercurial, feel free to skip to the next section.

I personally prefer to install Mercurial using pip or easy_install--it's usually more up-to-date than what you would have in your package repositories. To do that, simply run a command such as the following:

$ sudo pip install mercurial

This will go out and download and install the latest stable Mercurial. You may need python-dev or something like that for your platform in order for that command to work. However, if you're on Windows, I highly recommend TortoiseHg. The installer for TortoiseHg will install a graphical Mercurial client along with the command line tools.

Create A Repository

Now let's create a brand new Mercurial repository to house our notes/documentation. Open a terminal/console/command prompt to the location of your choice on your computer and execute the following commands:

$ hg init mydox
$ cd mydox

Configure Sphinx

The next step is to configure Sphinx for our project. Sphinx makes this very simple:

$ sphinx-quickstart

This is a wizard that will walk you through the configuration process for your project. It's pretty safe to accept the defaults, in my opinion. Here's the output of my wizard:

$ sphinx-quickstart
Welcome to the Sphinx quickstart utility.

Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).

Enter the root path for documentation.
> Root path for the documentation [.]:

You have two options for placing the build directory for Sphinx output.
Either, you use a directory "_build" within the root path, or you separate
"source" and "build" directories within the root path.
> Separate source and build directories (y/N) [n]: y

Inside the root directory, two more directories will be created; "_templates"
for custom HTML templates and "_static" for custom stylesheets and other static
files. You can enter another prefix (such as ".") to replace the underscore.
> Name prefix for templates and static dir [_]:

The project name will occur in several places in the built documentation.
> Project name: My Dox
> Author name(s): Josh VanderLinden

Sphinx has the notion of a "version" and a "release" for the
software. Each version can have multiple releases. For example, for
Python the version is something like 2.5 or 3.0, while the release is
something like 2.5.1 or 3.0a1.  If you don't need this dual structure,
just set both to the same value.
> Project version: 0.0.1
> Project release [0.0.1]:

The file name suffix for source files. Commonly, this is either ".txt"
or ".rst".  Only files with this suffix are considered documents.
> Source file suffix [.rst]:

One document is special in that it is considered the top node of the
"contents tree", that is, it is the root of the hierarchical structure
of the documents. Normally, this is "index", but if your "index"
document is a custom template, you can also set this to another filename.
> Name of your master document (without suffix) [index]:

Please indicate if you want to use one of the following Sphinx extensions:
> autodoc: automatically insert docstrings from modules (y/N) [n]:
> doctest: automatically test code snippets in doctest blocks (y/N) [n]:
> intersphinx: link between Sphinx documentation of different projects (y/N) [n]:
> todo: write "todo" entries that can be shown or hidden on build (y/N) [n]:
> coverage: checks for documentation coverage (y/N) [n]:
> pngmath: include math, rendered as PNG images (y/N) [n]:
> jsmath: include math, rendered in the browser by JSMath (y/N) [n]:
> ifconfig: conditional inclusion of content based on config values (y/N) [n]:

A Makefile and a Windows command file can be generated for you so that you
only have to run e.g. `make html' instead of invoking sphinx-build
directly.
> Create Makefile? (Y/n) [y]:
> Create Windows command file? (Y/n) [y]: n

Finished: An initial directory structure has been created.

You should now populate your master file ./source/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
   make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.

If you followed the same steps I did (I separated the source and build directories), you should see three new files in your mydox repository:

  • build/
  • Makefile
  • source/

We'll do our work in the source directory.

Get Some ReST

Now is the time when we start writing some ReST that we want to turn into HTML using Sphinx. Open some file, like first_doc.rst and put some ReST in it. If nothing comes to mind, or you're not familiar with ReST syntax, try the following:

=========================
This Is My First Document
=========================

Yes, this is my first document.  It's lame.  Deal with it.

Save the file (keep in mind that it should be within the source directory if you used the same settings I did). Now it's time to add it to the list of files that Mercurial will pay attention to. While we're at it, let's add the other files that were created by the Sphinx configuration wizard:

$ hg add
adding ../Makefile
adding conf.py
adding first_doc.rst
adding index.rst
$ hg st
A Makefile
A source/conf.py
A source/first_doc.py
A source/index.rst

Don't worry that we don't see all of the directories in the output of hg st--Mercurial tracks files, not directories.

Automate HTML-ization

Here comes the magic in automating the conversion from ReST to HTML: Mercurial hooks. We will use the precommit hook to fire off a command that tells Sphinx to translate our ReST markup into HTML.

Edit your mydox/.hg/hgrc file. If the file does not yet exist, go ahead and create it. Add the following content to it:

[hooks]
precommit.sphinxify = ~/bin/sphinxify_docs.sh

I've opted to call a Bash script instead of using an inline Python call. Now let's create the Bash script, ~/bin/sphinxify_docs.sh:

#!/bin/bash
cd $HOME/mydox
sphinx-build source/ docs/

Notice that I used the $HOME environment variable. This means that I created the mydox directory at /home/myusername/mydox. Adjust that line according to your setup. You'll probably also want to make that script executable:

$ chmod +x ~/bin/sphinxify_docs.sh

Three, Two, One...

You should now be at a stage where you can safely commit changes to your repository and have Sphinx build your HTML documentation. Execute the following command somewhere under your mydox repository:

$ hg ci -m "Initial commit"

If your setup is anything like mine, you should see some output similar to this:

$ hg ci -m "Initial commit"
Making output directory...
Running Sphinx v0.6.4
No builder selected, using default: html
loading pickled environment... not found
building [html]: targets for 2 source files that are out of date
updating environment: 2 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... /home/jvanderlinden/mydox/source/first_doc.rst:: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [100%] index
writing additional files... genindex search
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded, 1 warning.
$ hg st
? docs/.buildinfo
? docs/.doctrees/environment.pickle
? docs/.doctrees/first_doc.doctree
? docs/.doctrees/index.doctree
? docs/_sources/first_doc.txt
? docs/_sources/index.txt
? docs/_static/basic.css
? docs/_static/default.css
? docs/_static/doctools.js
? docs/_static/file.png
? docs/_static/jquery.js
? docs/_static/minus.png
? docs/_static/plus.png
? docs/_static/pygments.css
? docs/_static/searchtools.js
? docs/first_doc.html
? docs/genindex.html
? docs/index.html
? docs/objects.inv
? docs/search.html
? docs/searchindex.js

If you see something like that, you're in good shape. Go ahead and take a look at your new mydox/docs/index.html file in the Web browser of your choosing.

Not very exciting, is it? Notice how your first_doc.rst doesn't appear anywhere on that page? That's because we didn't tell Sphinx to put it there. Let's do that now.

Customizing Things

Edit the mydox/source/index.rst file that was created during Sphinx configuration. In the section that starts with .. toctree::, let's tell Sphinx to include everything we ReST-ify:

.. toctree::
   :maxdepth: 2
   :glob:

   *

That should do it. Now, I don't know about you, but I don't really want to include the output HTML, images, CSS, JS, or anything in my documentation repository. It would just take up more space each time we change an .rst file. Let's tell Mercurial to not pay attention to the output HTML--it'll just be static and always up-to-date on our filesystem.

Create a new file called mydox/.hgignore. In this file, put the following content:

syntax: glob
docs/

Save the file, and you should now see something like the following when running hg st:

$ hg st
M source/index.rst
? .hgignore

Let's include the .hgignore file in the list of files that Mercurial will track:

$ hg add .hgignore
$ hg st
M source/index.rst
A .hgignore

Finally, let's commit one more time:

$ hg ci -m "Updating the index to include our .rst files"
Running Sphinx v0.6.4
No builder selected, using default: html
loading pickled environment... done
building [html]: targets for 1 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
writing additional files... genindex search
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded.

Tada!! The first_doc.rst should now appear on the index page.

Serving Your Documentation

Who seriously wants to have HTML files that are hard to get to? How can we make it easier to access those HTML files? Perhaps we can create a simple static file Web server? That might sound difficult, but it's really not--not when you have access to Python!

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from BaseHTTPServer import HTTPServer
from SimpleHTTPServer import SimpleHTTPRequestHandler

def main():
    try:
        server = HTTPServer(('', 80), SimpleHTTPRequestHandler)
        server.serve_forever()
    except KeyboardInterrupt:
        server.socket.close()

if __name__ == '__main__':
    main()

I created this simple script and put it in my ~/bin/ directory, also making it executable. Once that's done, you can navigate to your mydox/docs/ directory and run the script. Since I called the script webserver.py, I just do this:

$ cd ~/mydox/docs
$ sudo webserver.py

This makes it possible for you to visit http://localhost/ on your own computer, or to use your computer's IP in place of localhost to access your documentation from a different computer on your network. Pretty slick, if you ask me.

I suppose there's more I could add, but that's all I have time for tonight. Enjoy!

Review: Django 1.0 Web Site Development

Introduction

Several months ago, a UK-based book publisher, Packt Publishing contacted me to ask if I would be willing to review one of their books about Django. I gladly jumped at the opportunity, and I received a copy of the book a couple of weeks later in the mail. This happened at the beginning of September 2009. It just so happened that I was in the process of being hired on by ScienceLogic right when all of this took place. The subsequent weeks were filled to the brim with visitors, packing, moving, finding an apartment, and commuting to my new job. It was pretty stressful.

Things are finally settling down, so I've taken the time to actually review the book I was asked to review. I should mention right off the bat that this is indeed a solicited review, but I am in no way influenced to write a good or bad review. Packt Publishing simply wants me to offer an honest review of the book, and that is what I indend to do. While reviewing the book, I decided to follow along and write the code the book introduced. I made sure that I was using the official Django 1.0 release instead of using trunk like I tend to do for my own projects.

The title of the book is Django 1.0 Web Site Development, written by Ayman Hourieh, and it's only 250 pages long. Ayman described the audience of the book as such:

This book is for web developers who want to learn how to build a complete site with Web 2.0 features, using the power of a proven and popular development system--Django--but do not necessarily want to learn how a complete framework functions in order to do this. Basic knowledge of Python development is required for this book, but no knowledge of Django is expected.

Ayman introduced Django piece by piece using the end goal of a social bookmarking site, a la del.icio.us and reddit. In the first chapter of the book, Ayman discussed the history of Django and why Python and Django are a good platform upon which to build Web applications. The second chapter offers a brief guide to installing Python and Django, and getting your first project setup. Not much to comment on here.

Digging In

Chapter three is where the reader was introduced to the basic structure of a Django project, and the initial data models were described. Chapter four discussed user registration and management. We made it possible for users to create accounts, log into them, and log out again. As part of those additions, the django.forms framework was introduced.

In chapter five, we made it possible for bookmarks to be tagged. Along with that, we built a tag cloud, restricted access to certain pages, and added a little protection against malicious data input. Next up was the section where things actually started getting interesting for me: enhancing the interface with fancy effects and AJAX. The fancy effects include live searching for bookmarks, being able to edit a bookmark in place (without loading a new page), and auto-completing tags when you submit a bookmark.

This chapter really reminded me just how simple it is to add new, useful features to existing code using Django and Python. I was thoroughly impressed at how easy it was to add the AJAX functionality mentioned above. Auto-completing the tags as you type, while jQuery and friends did most of the work, was very easy to implement. It made me happy.

Chapter seven introduced some code that allowed users to share their bookmarks with others. Along with this, the ability to vote on shared bookmarks was added. Another feature that was added in this chapter was the ability for users to comment on various bookmarks.

The ridiculously amazing Django Administration utility was first introduced in chapter eight. It kinda surprised me that it took 150 pages before this feature was brought to the user's attention. In my opinion, this is one of the most useful selling points when one is considering a Web framework for a project. When I first encountered Django, the admin interface was one of maybe three deciding factors in our company's decision to become a full-on Django shop.

Bring on the Web 2.0

Anyway, in chapter nine, we added a handful of useful "Web 2.0" features. RSS feeds were introduced. We learned about pagination to enhance usability and performance. We also improved the search engine in our project. At this stage, the magical Q objects were mentioned. The power behind the Q objects was discussed very well, in my opinion.

In chapter 10, we were taught how we can create relationships between members on the site. We made it possible for users to become "friends" so they can see the latest bookmarks posted by their friends. We also added an option for users to be able to invite some of their other friends to join the site via email, complete with activation links. Finally, we improved the user interface by providing a little bit of feedback to the user at various points using the messages framework that is part of the django.contrib.auth package in Django 1.0.

More advanced topics, such as internationalization and caching, were discussed in chapter 11. Django's special unit testing features were also introduced in chapter 11. This section actually kinda frustrated me. Caching was discussed immediately before unit testing. In the caching section, we learned how to enable site-wide caching. This actually broke the unit tests. They failed because the caching system was "read only" while running the tests. Anyway, it's probably more or less a moot point.

Chapter 11 also briefly introduced things to pay attention to when you deploy your Django projects into a production environment. This portion was mildly disappointing, but I don't know what else would have made it better. There are so many functional ways to deploy Django projects that you could write books just to describe the minutia involved in deployment.

The twelfth and final chapter discussed some of the other things that Django has to offer, such as enhanced functionality in templates using custom template tags and filters and model managers. Generic views were mentioned, and some of the other useful things in django.contrib were brought up. Ayman also offered a few ideas of additional functionality that the reader can implement on their own, using the things they learned throughout the book.

Afterthoughts

Overall, I felt that this book did a great job of introducing the power that lies in using Django as your framework of choice. I thought Ayman managed to break things up into logical sections, and that the iterations used to enhance existing functionality (from earlier chapters) were superbly executed. I think that this book, while it does assume some prior Python knowledge, would be a fine choice for those who are curious to dig into Django quickly and easily.

Some of the beefs I have with this book deal mostly with the editing. There were a lot of strange things that I found while reading through the book. However, the biggest sticking point for me has to do with "pluggable" applications. Earlier I mentioned that the built-in Django admin was one of only a few deciding factors in my company's choice to become a Django shop. Django was designed to allow its applications to be very "pluggable."

You may be asking, "What do I mean by 'pluggable'?" Well, say you decide to build a website that includes a blog, so you build a Django project and create an application specific to blogging. Then, at some later time, you need to build another site that also has blog functionality. Do you want to rewrite all of the blogging code for the second site? Or do you want to use the same code that you used in the first site (without copying it)? If you're anything like me and thousands of other developers out there, you would probably rather leverage the work you had already done. Django allows you to do this if you build your Django applications properly.

This book, however, makes no such effort to teach the reader how to turn all of their hard work on the social bookmarking features into something they could reuse over and over with minimal effort in the future. Application-specific templates are placed directly into the global templates directory. Application-specific URLconfs are placed in the root urls.py file. I would have liked to see at least some effort to make the bookmarking application have the potential to be reused.

Finally, the most obvious gripe is that the book is outdated. That's understandable, though! Anything in print media will likely be outdated the second it is printed if the book has anything to do with computers. However, with the understanding that this book was written specifically for Django 1.0 and not Django 1.1 or 1.2 alpha, it does an excellent job at hitting the mark.

Even More New VIM Fun

It appears that my last article generated a bit of interest! I had one good friend share some enlightening tips with me after he read about my findings. He opened my eyes to the world of abbreviations in VIM. Oh boy, oh boy!!

I'll pretty much just continue today's earlier post, by showing another (super awesome) way to insert blocks into your Django templates. If you were to enter this line into your ~/.vimrc file...

ab bl <Esc>bi{% block <Esc>ea %}{% endblock %}<Esc>h%i

...you would be able to type this...

content bl

...while you're in insert mode. As soon as you hit your spacebar after the bl, VIM would turn it into...

{% block content %} {% endblock %}

...and put you in insert mode between the two tags. Perfect!

For those of you who are interested in understanding the abbreviation, first it will take you out of insert mode (<Esc>). Then it jumps to the beginning of the word before the bl abbreviation and drops back into insert mode (bi). Next it inserts {% block and hops out of insert mode long enough to move the cursor to the end of the word you typed before the bl abbreviation. It finishes inserting {% endblock %}, gets out of insert mode, goes back a character (to account for the space you typed after bl) (h), moves the cursor to the matching opening brace for the } at the end of {% endblock %} (%), and finally puts you back into insert mode (i). Whew! It's all the same old VIM stuff, just packed into one uber-powerful abbreviation!

Thank to Jonathan Geddes for the guidance on this! I find this method to be superior to the one I posted about earlier because you don't even have to leave insert mode to get the code block stuff inserted into your templates!

New Fun with VIM

Hey everyone! I apologize for my lack of writing lately. Our baby boy was born on August 8th, and I'm still trying to catch up to everything from April :)

However, today I figured out some neat magic with VIM that I just have to share. Part of it is also so I don't forget it in the future! That's how awesome it is.

As you well know, I'm a big fan of Django. And I'm a relatively recent convert to VIM. One thing that I find to be kind of a common thing for me when developing a Django site is adding new blocks in the templates. Typing the "boilerplate" code for these blocks is easy, but it takes time.

Today, while tinkering with VIM, I figured out a way to automate the insertion of the boilerplate code:

nnoremap <Leader>b i{% block  %}{% endblock %}<Esc>16hi

Inserting that in your ~/.vimrc will allow you to type \b in normal mode while in VIM, which will insert the text:

{% block  %}{% endblock %}

...and move your cursor to where you would type in the block's name. Finally, it puts you straight into insert mode so you can immediately type the block name and get to work. Alternatively, if you prefer to have the opening and closing block tags on separate lines, you could use:

nnoremap <Leader>b i{% block  %}<CR>{% endblock %}<Esc><Up>$2hi

This would insert:

{% block  %}
{% endblock %}

...and move your cursor to the same place as the other one, so you could start typing the block name. If you'd like to have access to both, you could change the letter that comes after <Leader> to whatever key you'd like. Pretty fancy stuff, eh?

I love it! I'm sure there are other, perhaps better, ways to accomplish the same thing. I am just excited that I figured it out on my own :)

Enjoy!

My VIM Adventures

Along with my recent adventures with Fedora 11, I decided to force myself to become more proficient with VIM. For those of you who do not know, VIM is based on perhaps one of the oldest surviving text editors around today. There are often religious-grade battles between those who believe in VIM and those who believe in Emacs, another long-surviving text editor. I'm not trying to get into any debates about which is better, and I'm not interested in why I should not be using VIM. If you still feel like I need to be set straight, please use the contact me form instead of the comments section.

Anyway, most people who use these editors fall into 1 of 3 categories (there are probably more categories actually):

  1. They're familiar with it enough to get the job done, but they're not exactly proficient. Therefore, they don't care about evangelizing the editor.
  2. They're proficient with the editor, but they're afraid of the politics involved in religious wars relating to text editors, so they don't evangelize.
  3. They're proficient with the editor and feel that the whole world would be better off if everyone used their preferred text editor. As such, they cannot shut up about the dang thing and drive all of their friends, coworkers, and acquaintances mad.

A few of you will probably agree with what I'm about to say. I fear I have transitioned from stage 1 to stage 3 fairly rapidly. I can't stop talking about VIM all of the sudden! You'd think it's the next best thing after sliced bread the way I've been blabbering about it. And here I am, writing an article about it. Hah.

Ever since I first started using Linux, I have been using vi to handle most of my text editing when I was in a terminal. I knew enough to get around. Basic things like navigation and inserting text were pretty much all I knew how to do. I dabbled with a tutorial here and there, but it wasn't long before the things I learned were lost, since I usually preferred a graphical text editor over VIM.

My recent experimentation with VIM has proved to be very fruitful, if I do say so myself. I am no longer tied down to some editor that is slow and bulky, I don't have much to worry about when I switch computers (chances are that VIM is on any computer I use regularly), and I don't even need to be sitting at the computer I'm using VIM on! In fact, today I was doing most of my work over an SSH session to my netbook. I felt more productive today than I have in a very long time.

It's been a long time since I've enjoyed using a mouse to perform basic tasks on my computer. Using VIM allows me to rid myself of the mouse entirely for my text editing tasks, and I don't feel at all limited in my capabilities. Things that used to be quite sketchy operations using my favorite graphical editors end up being very simple with VIM.

I also love the obscurity favor of it all.

Examples

I wish I could just keep adding stuff to this list! There are so many neat things I want to share with everyone about VIM! I'm sure there are more efficient ways to do some of the things I have been learning with VIM, but this works very well for me.

Laziness

I do a lot of reStructuredText for various things. In fact, I'm writing this article using VIM right now. ReST is fantastic, but it's horrible to do using an editor that is not set up with a mono spaced font. I like to see things nicely lined up (I'm a Python developer, after all). I also like to have my section headings have an underline that is as long as the heading itself. For example, the heading just above this looks like this:

Examples
========

In this particular instance, it's not a big deal to hold down the equals key long enough to underline the word "Examples". However, sometimes I get some pretty lengthy section titles. The lazy side of me doesn't want my finger to hang around on the same key for very long (or tap it dozens of times, for that matter). Also, trying to figure out how many characters are in a section title without a mono spaced font is very annoying.

The/a solution? Say I have a section heading that is 50 characters long. To underline it, all I have to do is type 50i= and hit the escape key.

Cutting Text Mid-Line

Another neat thing is being able to cut text from the cursor to a particular character somewhere later on (or earlier on!) in the same line. Say I have a hyperlink whose address I wish to change:

<a href="http://www.somelong.com/that/I/want/to/change/">Link Text</a>

Instead of using the mouse to highlight the href attribute's value (or highlight it using shift on the keyboard), I just position my cursor on the h in http and type dt". VIM will lop that address right out of there (and you can paste it elsewhere if you'd like). I used this particular shortcut countless times today as I replaced things like {% url some-named-url with,some,parameters %} with {{ some_object.get_absolute_url }} in some Django templates.

Search & Replace

And I cannot neglect the classic search and replace functionality in VIM. You can use fancy regular expressions in VIM to replace some text with something else. I was trying to do a little refactoring today, and I came up with a command like this:

:s/something/lambda (a,b,c): \0(a,b,c)/g

That sort of command works great to replace all occurrences of "something" on the current line with "lambda (a,b,c): something(a,b,c)". Fantastic. What about a global search and replace, instead of just the current line? Stash a % at the front of the command (:%s/something/lambda (a,b,c): \0(a,b,c)/g) and you're in business.

Now what if you only wanted to perform that search and replace over a certain group of lines instead of a single line or the whole file? This is one I'm particularly thrilled about:

:.,.+9 s/something/lambda (a,b,c): \0(a,b,c)/g

That little beauty will perform the search and replace on the current line and the following 9 lines. How awesome is that?

Moving & Deleting Words

Sometimes as I am writing something, I decide I would like to reword a sentence as I near the end. Sometimes this involves simply deleting a word or two. Sometimes it means chopping a few words out of the beginning part of a sentence to put them back at the end somewhere. Whatever the case, VIM seems to handle my needs perfectly well.

Say I have this sentence (from the Vimperator Web site): "Writing efficient user interfaces is the main maxim, here at Vimperator labs." If I want to move the "here at Vimperator labs" to the beginning of the sentence, assuming I just finished typing it, I would place my cursor over the period at the end, type dT,, hit ( to go to the beginning of the sentence, hit P to insert what I just copied, and then handle the rest of the clean up (capitalization, fixing the comma, etc). I could have also done something like, 4db instead of dT,.

If I want to cut/delete an entire word, or to the end of whatever word my cursor is currently on, I could use dw. For more than one word, just put a number before the command. It's great stuff!

Taking It Too Far

I've gotten so carried away with all of this VIM business. I really have. I installed vimperator in Firefox. This extension gives Firefox a VIM-like interface. Now I can do pretty much all of my regular surfing without using the mouse. Some may argue that this is absolutely impractical because it would take much longer to get to the right link on a page using the keyboard than it would with the mouse. That may be true. I dunno, but I still think it's awesome that I really don't need my mouse to browse the Internet now.

As I was playing with vimperator tonight, one of my buddies pointed out another useful extension called It's All Text. This extension allows you to use your preferred text editing program in regular old text boxes in Firefox. It is this extension which has just made writing my blog articles 200x more efficient. Now I can quickly and easily write my articles right here in VIM without having to copy and paste all over the place. Pretty dang incredible.

Oh yes, I'd like to thank Chad Hansen and Jonathan Geddes for helping me out as I explore the depths of VIM. You guys rock!

Django Tip: Application-Specific Templates

Today I have another Django goodie to share with you. For the past few days, I've been struggling to come up with a way to only load certain Django template tag libraries when a particular Django application is installed. There may well be other, more elegant solutions for this particular problem, but it can't hurt to add my findings to the pile.

We have several templates which need to display certain information only when a particular application (Satchmo, in this case) is installed in the site. A lot of these templates are global for our 100+ Django-powered sites, such as customized admin templates and the like. It's much easier for us to maintain our code this way, as opposed to overriding templates on a per-site basis.

The Problem

We have created template tags which allow us to render "Content A" if application foo is installed or render "Content B" if foo is not installed. This works great all the way up until you need to use template tags that are specific to foo. The reason for this is that all of the template nodes appear to be parsed before they're actually rendered. That means that if foo is not installed and one of your templates included a template tag from foo's template tag library, Django will complain because it cannot find that tag (since foo is not in your settings.INSTALLED_APPS).

I investigated several possible solutions to this, including a custom loadifapp tag. The idea was to only load a template tag library if the specified application exists in settings.INSTALLED_APPS. This proved to be an interesting and very, very hacky endeavor. In the end it didn't work, and it was taking much too long to get anywhere useful.

The Solution

The solution I came up with for this situation is to create an additional include tag. I basically copied the include tag from Django itself and hacked it a bit. The result:

from django import template
from django.core.exceptions import ImproperlyConfigured
from django.db.models import get_app
from django.template.loader_tags import ConstantIncludeNode, IncludeNode

register = template.Library()

def do_include_ifapp(parser, token):
    """
    Loads a template and renders it with the current context if the specified
    application is in settings.INSTALLED_APPS.

    Example::

        {% includeifapp app_label "foo/some_include" %}
    """
    bits = token.split_contents()
    if len(bits) != 3:
        raise TemplateSyntaxError, "%r tag takes two argument: the application label and the name of the template to be included" % bits[0]

    app_name, path = bits[1:]
    app_name = app_name.strip('"\'')
    try:
        models = get_app(app_name)
    except ImproperlyConfigured:
        return template.Node()

    if path[0] in ('"', "'") and path[-1] == path[0]:
        return ConstantIncludeNode(path[1:-1])
    return IncludeNode(path)
register.tag('includeifapp', do_include_ifapp)

The magic here is the return template.Node() if Django cannot load a particular application. This makes it so the template you would be including will not be parsed, and the invalid template tag errors disappear!

To use this tag in your Django-powered site, simple plug it into one of your template tag libraries and do something like this:

{% extends 'base.html' %}
{% load our_global_tags %}

{% block content %}
<h2>Global Content Header</h2>
Bla bla

{% includeifapp foo 'foo_specific_junk.html' %}
{% endblock %}

And within foo_specific_junk.html you would load whatever template tag libraries you need that would break your templates without foo being installed. This tag should work for any application. I would be interested to hear what you use it for in the comments!

Syntax Highlighting, ReST, Pygments, and Django

Some of you regulars out there may have noticed an interesting change in the presentation of some of my articles: source code highlighting. I've been interested in doing this for quite some time, I just never really got around to implementing it until last night.

I found this implementation process to be a bit more complicatd than I had anticipated. For my own benefit as well as for anyone else who wants to do the same thing, I thought I'd document my findings in a thorough article for how to add syntax highlighting to an existing Django- and reStructuredText-powered Web site.

The power behind the syntax highlighting is:

Python is a huge player in this feature because reStructuredText (ReST) was built for Python, Pygments is the source highlighter (written in Python), and Django is written in Python (and my site is powered by Django). Some of you may recall that I converted all of my articles to ReST not too long ago because it suited my needs better than Textile, my previous markup processor. At the time, I was not aware that the conversion to ReST would make it all the easier for me to implement the syntax highlighting, but last night I figured out that that conversion probably saved me a lot of frustration. Cascading Stylesheets (CSS) are responsible for making the source code actually look good, while Pygments takes care of assigning classes to various parts of the designated source code and generating the CSS.

So, the first set of requirements, which I will not document in this article, are that you already have a Django site up and running and that you're familiar with ReST syntax. If you have the django.contrib.flatpages application installed already, you can type up some ReST documents there and apply the concepts discussed in this article.

Next, you should ensure that you have Pygments installed. There are a variety of ways to install this. Perhaps the easiest and most platform-independent method is to use easy_install:

$ easy_install pygments

This command should work essentially the same on Windows, Linux, and Macintosh computers. If you don't have it installed, you can get it from its website. If you're using a Debian-based distribution of Linux, such as Ubuntu, you could do something like this:

$ sudo apt-get install python-pygments

...and it should take care of downloading and installing Pygments. Alternatively, you can download it straight from the PyPI page and install it manually.

Now we need to install the Pygments ReST directive. A ReST directive is basically like a special command to the ReST processor. I think this part was the most difficult aspect of the implementation, simply because I didn't know where to find the Pygments directive or how to write my own. Eventually, I ended up downloading the Pygments-1.0.tar.gz file from PyPI, opening the Pygments-1.0/external/rst-directive.py file from the archive, and copying the stuff in there into a new file within my site.

For my own purposes, I made some small adjustments to the directive over what come with the Pygments distribution. I think it would save us all a lot of hassle if I just copied and pasted the directive, as I currently have it, so you can see it first-hand.

"""
    The Pygments reStructuredText directive
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    This fragment is a Docutils_ 0.4 directive that renders source code
    (to HTML only, currently) via Pygments.

    To use it, adjust the options below and copy the code into a module
    that you import on initialization.  The code then automatically
    registers a ``code-block`` directive that you can use instead of
    normal code blocks like this::

    .. code:: python

            My code goes here.

    If you want to have different code styles, e.g. one with line numbers
    and one without, add formatters with their names in the VARIANTS dict
    below.  You can invoke them instead of the DEFAULT one by using a
    directive option::

    .. code:: python
       :number-lines:

            My code goes here.

    Look at the `directive documentation`_ to get all the gory details.

    .. _Docutils: http://docutils.sf.net/
    .. _directive documentation:
       http://docutils.sourceforge.net/docs/howto/rst-directives.html

    :copyright: 2007 by Georg Brandl.
    :license: BSD, see LICENSE for more details.
"""

# Options
# ~~~~~~~

# Set to True if you want inline CSS styles instead of classes
INLINESTYLES = False

from pygments.formatters import HtmlFormatter

# The default formatter
DEFAULT = HtmlFormatter(noclasses=INLINESTYLES)

# Add name -> formatter pairs for every variant you want to use
VARIANTS = {
    'linenos': HtmlFormatter(noclasses=INLINESTYLES, linenos=True),
}


from docutils import nodes
from docutils.parsers.rst import directives

from pygments import highlight
from pygments.lexers import get_lexer_by_name, TextLexer

def pygments_directive(name, arguments, options, content, lineno,
                       content_offset, block_text, state, state_machine):
    try:
        lexer = get_lexer_by_name(arguments[0])
    except ValueError:
        # no lexer found - use the text one instead of an exception
        lexer = TextLexer()
    # take an arbitrary option if more than one is given
    formatter = options and VARIANTS[options.keys()[0]] or DEFAULT
    parsed = highlight(u'\n'.join(content), lexer, formatter)
    parsed = '<div class="codeblock">%s</div>' % parsed
    return [nodes.raw('', parsed, format='html')]

pygments_directive.arguments = (1, 0, 1)
pygments_directive.content = 1
pygments_directive.options = dict([(key, directives.flag) for key in VARIANTS])

directives.register_directive('code-block', pygments_directive)

I won't explain what that code means, because, quite frankly, I'm still a little hazy on the inner workings of ReST directives myself. Suffice it to say that this snippet allows you to easily highlight blocks of code on ReST-powered pages.

The question now is: where do I put this snippet? As far as I'm aware, this code can be located anywhere so long as it is loaded at one point or another before you start your ReST processing. For the sake of simplicity, I just stuffed it in the __init__.py file of my Django site. This is the __init__.py file that lives in the same directory as manage.py and settings.py. Putting it in that file just makes sure it's loaded each time you start your Django site.

To make Pygments highlight a block of code, all you need to do is something like this:

.. code:: python

    print 'Hello world!'

...which would look like...

print 'Hello world!'

If you have a longer block of code and would like line numbers, use the :number-lines: option:

.. code:: python
    :number-lines:

    for i in range(100):
        print i

...which should look like this...

for i in range(100):
    print i

That's all fine and dandy, but it probably doesn't look like the code is highlighted at all just yet (on your site, not mine). It's just been marked up by Pygments to have some pretty CSS styles applied to it. But how do you know which styles mean what?

Luckily enough, Pygments takes care of generating the CSS files for you as well. There are several attractive styles that come with Pygments. I would recommend going to the Pygments demo to see which one suits you best. You can also roll your own styles, but I haven't braved that yet so I'll leave that for another day.

Once you choose a style (I chose native for Code Koala), you can run the following commands:

$ pygmentize -S native -f html > native.css
$ cp native.css /path/to/site/media/css

(obviously, you'd want to replace native with the name of the style you like the most) Finally, add a line to your HTML templates to load the newly created CSS file. In my case, it's something like this:

<link rel="stylesheet" type="text/css" href="/static/styles/native.css" />

Now you should be able to see nicely-formatted source code on your Web pages (assuming you've already got ReST processing your content).

If you haven't been using ReST to generate nicely-formatted pages, you should make sure a couple of things are in place. First, you must have the django.contrib.markup application installed. Second, your templates should be setup to process ReST markup into HTML. Here's a sample templates/flatpages/default.html:

{% extends 'base.html' %}
{% load markup %}

{% block title %}{{ flatpage.title }}{% endblock %}

{% block content %}
<h2>{{ flatpage.title }}</h2>

{{ flatpage.content|restructuredtext }}
{% endblock %}

So that short template should allow you to use ReST markup for your flatpages, and it should also take care of the magic behind the .. code:: python directive.

I should also note that Pygments can handle a TON of languages. Check out the Pygments demo for a list of languages it knows how to highlight.

I think that about does it. Hopefully this article will help some other poor chap who is currently in the same situation as I was last night, and hopefully it will save you a lot more time than it took me to figure out all this junk. If it looks like I've missed something, or maybe that something needs further clarification, please comment and I'll see what I can do.

Adding Captcha To Django's Built-in Comments

I recently had a good friend of mine ask for some help in adding a captcha field to his comments form. I gave him some pointers, but before he could put them into action he had to leave for a Thanksgiving roadtrip home. I didn't give much mind to the idea of putting captchas on my own site since it's not all that popular amongst spammers yet. When I woke up this morning, however, I found myself with a few spare minutes to see if my pointers were correct.

Some of the ideas I shared with my friend turned out to not work very well. As I tinkered about trying to get things to work on my own site, I think I came up with a relatively efficient way of doing things.

Installing The Captcha

The captcha field I use is quite simple and effective. I originally got it from http://django.agami.at/media/captcha/, but the project seems to be unmaintained now. Along the road to Django 1.0, some changes were made to the way form fields work, and there is a minor change required in the base code for this captcha field if you want it to work. Alternatively, you can use a copy of the field that I'm currently using.

All you need to do is extract the captcha directory somewhere on your PYTHONPATH. The author recommends putting it in django.contrib, but I usually just place it straight on the PYTHONPATH so all I need to do is from captcha import CaptchaField instead of from django.contrib.captcha import CaptchaField. Minor details...

Adding The Captcha

The first thing you'll want to do after installing the captcha field is add the field itself to your comments form. Instead of subclassing the built-in django.contrib.comments.forms.CommentForm form, I simply decorated the constructor of the form as such:

1
2
3
4
5
6
7
8
9
from django.contrib.comments.forms import CommentForm
from captcha import CaptchaField

def add_captcha(func):
    def wrapped(self, *args, **kwargs):
        func(self, *args, **kwargs)
        self.fields['security_code'] = CaptchaField()
    return wrapped
CommentForm.__init__ = add_captcha(CommentForm.__init__)

This adds a field called security_code to the CommentForm, and it works the same way as if you had done something like this:

1
2
3
4
5
6
7
from django import forms
from captcha import CaptchaField

class MyCommentForm(forms.Form):
    name = forms.CharField()
    ...
    security_code = CaptchaField()

You can put the decorating snippet from above anywhere you'd like so long as the module you put it in is loaded at some point in your project. I usually put this sort of magic in my main urls.py file so it's harder to forget about when I debug things.

Fixing the Form

The first problem with this little trick seems to be that the CaptchaField is rendered as unsafe HTML in the default form.html template in the built-in comments application. That just means that, instead of seeing the captcha, you will see the HTML necessary to render the CaptchaField directly on the page, like this:

<input type="hidden" name="security_code" value="captcha.caZ1SqQ" />
<img src="/static/captchas/caZ1SqQ/0656f09d3974850397dd4c4974f23a35.gif"
 alt="" /><br /><input type="text" name="security_code"
 id="id_security_code" />

To fix that, you can apply the safe filter to the field and make the template look something like this:

{% load comments i18n %}
<form action="{% comment_form_target %}" method="post">
<table>
{% for field in form %}
    {% if field.is_hidden %}
    {{ field }}
    {% else %}
    <tr{% ifequal field.name "honeypot" %} style="display:none;"{% endifequal %}>
        <th>{{ field.label_tag }}:</th>
        <td {% if field.errors %} class="error"{% endif %}>
            {{ field|safe }} &nbsp;
            {% if field.errors %}{{ field.errors|join:"; " }}{% endif %}
        </td>
    </tr>
    {% endif %}
{% endfor %}
</table>
<p class="submit">
    <input type="submit" name="post" class="submit-post"
     value="{% trans "Post" %}" />
    <input type="submit" name="preview" class="submit-preview"
     value="{% trans "Preview" %}" />

</form>

Notice the {{ field|safe }} in there. Also note that I prefer the table layout for the comment form over the default mode. If you change your form template as I have done, you should put the updated copy in your own project's template directory. It belongs in templates/comments/form.html, assuming that your templates directory is called templates. You'll probably also want to check out the preview.html template for the django.contrib.comments application. I changed mine to look like this:

{% extends "comments/base.html" %}
{% load i18n %}

{% block title %}{% trans "Preview your comment" %}{% endblock %}

{% block content %}
    {% load comments %}
    {% if form.errors %}
    <h1>{% blocktrans count form.errors|length as counter %}Please
     correct the error below{% plural %}Please correct the errors below
     {% endblocktrans %}</h1>
    {% else %}
    <h1>{% trans "Preview your comment" %}</h1>
        <blockquote>{{ comment|linebreaks }}</blockquote>

        {% trans "and" %} <input type="submit" name="submit"
         class="submit-post" value="{% trans "Post your comment" %}"
         id="submit" /> {% trans "or make changes" %}:

    {% endif %}

    {% include 'comments/form.html' %}
{% endblock %}

See how I just use the include tag to pull in the comments/form.html template I mentioned above? Saves a lot of typing and potential for problems... If you update the preview.html template, you should save your copy in templates/comments/preview.html, assuming your templates directory is called templates.

Testing It Out

At this point, you should be able to try out your newly installed captcha-fied comments. If it doesn't work, please comment on this article and perhaps we can figure out the problem!