Some of you regulars out there may have noticed an interesting change in the presentation of some of my articles: source code highlighting. I've been interested in doing this for quite some time, I just never really got around to implementing it until last night.

I found this implementation process to be a bit more complicatd than I had anticipated. For my own benefit as well as for anyone else who wants to do the same thing, I thought I'd document my findings in a thorough article for how to add syntax highlighting to an existing Django- and reStructuredText-powered Web site.

The power behind the syntax highlighting is:

• Python
• reStructuredText
• Pygments
• Cascading Stylesheets
• Django

Python is a huge player in this feature because reStructuredText (ReST) was built for Python, Pygments is the source highlighter (written in Python), and Django is written in Python (and my site is powered by Django). Some of you may recall that I converted all of my articles to ReST not too long ago because it suited my needs better than Textile, my previous markup processor. At the time, I was not aware that the conversion to ReST would make it all the easier for me to implement the syntax highlighting, but last night I figured out that that conversion probably saved me a lot of frustration. Cascading Stylesheets (CSS) are responsible for making the source code actually look good, while Pygments takes care of assigning classes to various parts of the designated source code and generating the CSS.

So, the first set of requirements, which I will not document in this article, are that you already have a Django site up and running and that you're familiar with ReST syntax. If you have the django.contrib.flatpages application installed already, you can type up some ReST documents there and apply the concepts discussed in this article.

Next, you should ensure that you have Pygments installed. There are a variety of ways to install this. Perhaps the easiest and most platform-independent method is to use easy_install:

$ easy_install pygments

This command should work essentially the same on Windows, Linux, and Macintosh computers. If you don't have it installed, you can get it from its website. If you're using a Debian-based distribution of Linux, such as Ubuntu, you could do something like this:

$ sudo apt-get install python-pygments

...and it should take care of downloading and installing Pygments. Alternatively, you can download it straight from the PyPI page and install it manually.

Now we need to install the Pygments ReST directive. A ReST directive is basically like a special command to the ReST processor. I think this part was the most difficult aspect of the implementation, simply because I didn't know where to find the Pygments directive or how to write my own. Eventually, I ended up downloading the Pygments-1.0.tar.gz file from PyPI, opening the Pygments-1.0/external/rst-directive.py file from the archive, and copying the stuff in there into a new file within my site.

For my own purposes, I made some small adjustments to the directive over what come with the Pygments distribution. I think it would save us all a lot of hassle if I just copied and pasted the directive, as I currently have it, so you can see it first-hand.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

"""
    The Pygments reStructuredText directive
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    This fragment is a Docutils_ 0.4 directive that renders source code
    (to HTML only, currently) via Pygments.

    To use it, adjust the options below and copy the code into a module
    that you import on initialization.  The code then automatically
    registers a ``code-block`` directive that you can use instead of
    normal code blocks like this::

    .. code:: python

            My code goes here.

    If you want to have different code styles, e.g. one with line numbers
    and one without, add formatters with their names in the VARIANTS dict
    below.  You can invoke them instead of the DEFAULT one by using a
    directive option::

    .. code:: python
       :number-lines:

            My code goes here.

    Look at the `directive documentation`_ to get all the gory details.

    .. _Docutils: http://docutils.sf.net/
    .. _directive documentation:
       http://docutils.sourceforge.net/docs/howto/rst-directives.html

    :copyright: 2007 by Georg Brandl.
    :license: BSD, see LICENSE for more details.
"""

# Options
# ~~~~~~~

# Set to True if you want inline CSS styles instead of classes
INLINESTYLES = False

from pygments.formatters import HtmlFormatter

# The default formatter
DEFAULT = HtmlFormatter(noclasses=INLINESTYLES)

# Add name -> formatter pairs for every variant you want to use
VARIANTS = {
    'linenos': HtmlFormatter(noclasses=INLINESTYLES, linenos=True),
}


from docutils import nodes
from docutils.parsers.rst import directives

from pygments import highlight
from pygments.lexers import get_lexer_by_name, TextLexer

def pygments_directive(name, arguments, options, content, lineno,
                       content_offset, block_text, state, state_machine):
    try:
        lexer = get_lexer_by_name(arguments[0])
    except ValueError:
        # no lexer found - use the text one instead of an exception
        lexer = TextLexer()
    # take an arbitrary option if more than one is given
    formatter = options and VARIANTS[options.keys()[0]] or DEFAULT
    parsed = highlight(u'\n'.join(content), lexer, formatter)
    parsed = '<div class="codeblock">%s</div>' % parsed
    return [nodes.raw('', parsed, format='html')]

pygments_directive.arguments = (1, 0, 1)
pygments_directive.content = 1
pygments_directive.options = dict([(key, directives.flag) for key in VARIANTS])

directives.register_directive('code-block', pygments_directive)

I won't explain what that code means, because, quite frankly, I'm still a little hazy on the inner workings of ReST directives myself. Suffice it to say that this snippet allows you to easily highlight blocks of code on ReST-powered pages.

The question now is: where do I put this snippet? As far as I'm aware, this code can be located anywhere so long as it is loaded at one point or another before you start your ReST processing. For the sake of simplicity, I just stuffed it in the __init__.py file of my Django site. This is the __init__.py file that lives in the same directory as manage.py and settings.py. Putting it in that file just makes sure it's loaded each time you start your Django site.

To make Pygments highlight a block of code, all you need to do is something like this:

.. code:: python

    print 'Hello world!'

...which would look like...

print 'Hello world!'

If you have a longer block of code and would like line numbers, use the :number-lines: option:

.. code:: python
    :number-lines:

    for i in range(100):
        print i

...which should look like this...

1
2

for i in range(100):
    print i

That's all fine and dandy, but it probably doesn't look like the code is highlighted at all just yet (on your site, not mine). It's just been marked up by Pygments to have some pretty CSS styles applied to it. But how do you know which styles mean what?

Luckily enough, Pygments takes care of generating the CSS files for you as well. There are several attractive styles that come with Pygments. I would recommend going to the Pygments demo to see which one suits you best. You can also roll your own styles, but I haven't braved that yet so I'll leave that for another day.

Once you choose a style (I chose native for Code Koala), you can run the following commands:

$ pygmentize -S native -f html > native.css
$ cp native.css /path/to/site/media/css

(obviously, you'd want to replace native with the name of the style you like the most) Finally, add a line to your HTML templates to load the newly created CSS file. In my case, it's something like this:

<link rel="stylesheet" type="text/css" href="/static/styles/native.css" />

Now you should be able to see nicely-formatted source code on your Web pages (assuming you've already got ReST processing your content).

If you haven't been using ReST to generate nicely-formatted pages, you should make sure a couple of things are in place. First, you must have the django.contrib.markup application installed. Second, your templates should be setup to process ReST markup into HTML. Here's a sample templates/flatpages/default.html:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

{% extends 'base.html' %}
{% load markup %}

{% block title %}{{ flatpage.title }}{% endblock %}

{% block content %}
<h2>{{ flatpage.title }}</h2>

{{ flatpage.content|restructuredtext }}
{% endblock %}

So that short template should allow you to use ReST markup for your flatpages, and it should also take care of the magic behind the .. code:: python directive.

I should also note that Pygments can handle a TON of languages. Check out the Pygments demo for a list of languages it knows how to highlight.

I think that about does it. Hopefully this article will help some other poor chap who is currently in the same situation as I was last night, and hopefully it will save you a lot more time than it took me to figure out all this junk. If it looks like I've missed something, or maybe that something needs further clarification, please comment and I'll see what I can do.

With the recent release of Python 3.0 final, I've had a crazy itch that needed to be scratched. That itch, my friends, was to do a write-up of how to install Python 3.0 alongside my existing Python 2.5.2 installation without borking things up. The reason I thought it would be useful is that I'm running a Debian-based distribution of Linux called sidux right now, and neither Python 2.6 nor Python 3.0 are in the package repositories. I assume that Ubuntu and other Debian-based distributions might be the same way, and that there are others like me who would like to tinker with these new releases.

Just before attempting to install Python 3.0 on my computer, I started getting ready to write this article. Before I knew it, Python 3.0 was installed on my system, and I had no notes to share with you! That is how stinkin easy it is to install Python 3.0 without interfering with an existing install. I mentioned to a good friend that I wasn't sure the process was worth writing an article because it was so simple, but he encouraged me to carry on. Thanks bro.

So, without any further ado, here is what I did to install Python 3.0 from source alongside my Python 2.5 installation:

Note: If you're using a Mac or Windows, you should be able to simply install the packages for your platform to accomplish this same feat. Once the packages are in the repositories it will be just as easy for Linux.

• Download Python 3.0 from the official Web site: http://python.org/download/releases/3.0/

  I downloaded it this way:

$ wget http://python.org/ftp/python/3.0/Python-3.0.tar.bz2

• Unpack the archive:

$ tar jxf Python-3.0.tar.bz2

• Descend into the newly extracted Python-3.0 directory:

$ cd Python-3.0

• Install libreadline-dev. This step is necessary for the arrow keys to work, as pointed out by jazevec below in the comments. If you are on a Debian-based system, you can execute a command such as this:

  $ sudo apt-get install libreadline-dev

Other distributions may have different package names, such as ``readline-dev``.  If neither one of those package names work for your distribution, try searching your package manager for ``readline`` and install the development files.  Alternatively, you should be able to manually install what you need by installing `readline itself <http://directory.fsf.org/project/readline/>`_.

• Configure Python 3.0 for your computer:

$ ./configure

• Compile Python 3.0:

  $ make


**UPDATE** (9 Dec): Depending on your setup, you may or may not see a message such as this after executing the ``make`` command (thanks again to ``jazevec`` for pointing out that this can happen)::

  Failed to find the necessary bits to build these modules:
  _dbm               _gdbm              _hashlib
  _sqlite3           _ssl               _tkinter
  bz2                zlib
  To find the necessary bits, look in setup.py in detect_modules() for the module's name.

If you need any of those capabilities, you should install the appropriate development files for the missing module(s).  For example, above, we installed the ``libreadline-dev`` package.  To resolve the missing module problem for each one listed above (except ``_dbm`` because it's apparently borked on Debian right now... possibly other distros too), install these packages:

  * ``tk-dev`` to satisfy ``_tkinter``
  * ``libsqlite3-dev`` to satisfy ``_sqlite3``
  * ``libbz2-dev`` to satisfy ``bz2``
  * ``zlib1g-dev`` to satisfy ``zlib``
  * ``libssl-dev`` to satisfy ``_ssl`` and ``_hashlib``
  * ``libgdbm-dev`` to satisfy ``_gdbm``

• Install Python 3.0:

  $ sudo make install

or:

# make install

• Test Python 3.0:

$ python3.0
Python 3.0 (r30:67503, Dec  5 2008, 11:05:45)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'Hi'
File "<stdin>", line 1
    print 'Hi'
            ^
SyntaxError: invalid syntax
>>> print('Hi')
Hi
>>>

• Make sure that your old version of Python is still around:

$ python
Python 2.5.2 (r252:60911, Sep 29 2008, 21:15:13)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'Hi'
Hi
>>>

• Rejoice

That's about it folks! Extremely simple and fast, compared to what it could have been.

If you find yourself in a situation where you don't have access to sudo or straight root-level access, you can install Python 3.0 locally by doing something like this:

• Configure Python 3.0 for your computer:

$ ./configure --prefix=$HOME/local/

• Compile Python 3.0:

$ make

• Install Python 3.0:

  $ make install

Not that I omitted the ``sudo`` part of the command here.

• Symlink to Python 3.0, assuming you have a bin/ directory in your home directory (i.e. /home/[yourusername]/bin), and that said bin directory is on your PATH:

$ ln -s ~/local/bin/python3.0 ~/bin

• Test your locally installed Python 3.0:

  $ python3.0

or, if your local ``bin`` directory isn't on your ``PATH``:

$ ~/bin/python3.0

• Do the dance.

Please comment with any problems you find with this process, or any additional advice you can offer to newbies!

I've found myself in many situations where I have several images that need a watermark applied to them. Applying a watermark to more than 5 images at a time is annoying at best, even if you do have macros to do the mundane stuff for you. I don't like to do things that annoy me.

Recently, I found myself yet again confronted with a "please watermark these images" sort of situation, and I decided to take a more logical approach to the problem: program it. The result of my efforts is called django-watermark and is available on Google Code and PyPI. To be perfectly honest, I actually ganked most of the useful code for this application from a generous soul on the Internet.

This seems to be working fine for me. I think the positioning for watermarks might need more work, but there are several options to hold people over for the time being. Please let me know what your thoughts are.

• http://code.google.com/p/django-watermark/
• http://pypi.python.org/pypi/django-watermark/

Here's yet another application to help you make your site more useful to your users. django-reploc is a "representative locator" for Django-powered Web sites. It uses the Google Maps API to offer your users an interactive map of your representatives, vendors, friends, houses, etc.

I built this application to be a dealer locator for one of my clients. The application provides a way for you to create "representatives," and each representative may have one or more locations. For example, say you are a producer of clothing and apparel, ranging from shoes to pants to t-shirts to sweaters to hats to bags and so on. Perhaps your bigger representatives (or vendors) carry your products in all of their retail branches. reploc gives you a way to display each of these locations for the one vendor on a map for your users.

In addition to that, while the bigger vendors might carry your products in all locations, perhaps some locations only carry particular lines of your products. For example, a vendor location in Miami, Florida might carry a good deal of your sandals and none of your sweatshirts, while a vendor location in Bangor, Maine might not bother with any sandals but keeps sweatshirts in stock year-round. reploc offers a way for you to describe which product lines each location carries by the use of "attributes".

However, it can be used for much more than just that. Pretty much any location that you can show on Google Maps (is there anything that can't be shown on Google Maps anymore?) can be displayed with this application.

Furthermore, you can search for locations within a certain radius of an address that your users specify! A nice circle is drawn on the map, making it visually easier to understand what is and isn't within range. To make things even easier, the locations that do not fall within the radius of the given address are hidden from the map.

Finally, once a user finds a location within the radius of the address they specify, they are able to get driving directions to help them get right where they need to go!

w00t!

To learn more, go to the project page: http://code.google.com/p/django-reploc/

As some of you may have noticed, my Web host is having brain farts all over the place lately. None of my sites are very usable thanks to the poor quality of service and support on the part of my host right now. The problems seem to have begun around noon on Friday, September 19th. The symptoms include random HTTP 500 (internal server error) messages or extremely long load times.

I'm in the process of switching to a host that is more reliable. This new host is perhaps one of the best in the industry according to all of the reviews that I've read. Let's hope I have better luck there.

The other day at work I was involved in a humorous discussion about having our organization's Web site change colors depending on what day of the year it was. We threw around all sorts of fun ideas, knowing full well that it would never happen for our organization. It would be too unprofessional.

That night my allergies were bothering me to the point where I couldn't sleep, so I decided to see what I could do about making our discussion more possible, at least for others who can take a little more creative license than us. By morning, I had a working application that could effectively change the colors on a Web page depending on the date.

Today I released the source code on Google Code, hoping that someone more creative than me might take it and experiment to make something both attractive and useful.

You can learn more at the project's homepage: http://code.google.com/p/django-seasonal-stylesheets/

This article is a related to my previously posted article about installing Django, an advanced Web framework for perfectionists, on your own computer. Now we will learn how to install Django on a shared hosting account, using Site5 and fastcgi as an example. Depending on your host, you may or may not have to request additional privileges from the support team in order to execute some of these commands.

Note: Django requires at least Python 2.3. Newer versions of Python are preferred.

Note: This HOWTO assumes familiarity with the UNIX/Linux command line.

Note: If the wget command doesn't work for you (as in you don't have permission to run it), you might try curl [url] -O instead. That's a -O as in upper-case o.

mkdir ~/downloads
cd ~/downloads
wget http://peak.telecommunity.com/dist/virtual-python.py

python -V

python ~/downloads/virtual-python.py

export PATH=$HOME/bin:$PATH

source ~/.bashrc

which python

cd ~/downloads
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py

svn co http://code.djangoproject.com/svn/django/trunk ~/downloads/django-trunk

ln -s ~/downloads/django-trunk/django ~/lib/python2.4/site-packages
ln -s ~/downloads/django-trunk/django/bin/django-admin.py ~/bin

python -c "import django; print django.get_version()"

Traceback (most recent call last):
    File "<string>", line 1, in ?
ImportError: No module named django

easy_install -f http://www.pythonware.com/products/pil/ Imaging
easy_install mysql-python
easy_install flup

cd ~/downloads
wget http://effbot.org/downloads/Imaging-1.1.6.tar.gz
tar zxf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6
ln -s ~/downloads/Imaging-1.1.6/PIL ~/lib/python2.4/site-packages

python -c "import PIL"

mkdir -p ~/projects/django
cd ~/projects/django
django-admin.py startproject mysite
cd mysite
mkdir media templates

ln -s ~/projects/django/mysite/media ~/public_html/mysite/static
ln -s ~/lib/python2.4/site-packages/django/contrib/admin/media ~/public_html/mysite/media

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

#!/home/[your username]/bin/python
import sys, os

# Add a custom Python path.
sys.path.insert(0, "/home/[your username]/projects/django")

# Switch to the directory of your project. (Optional.)
os.chdir("/home/[your username]/projects/django/mysite")

# Set the DJANGO_SETTINGS_MODULE environment variable.
os.environ['DJANGO_SETTINGS_MODULE'] = "mysite.settings"

from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")

1
2
3
4
5
6

RewriteEngine On
RewriteBase /
RewriteRule ^(media/.*)$ - [L]
RewriteRule ^(static/.*)$ - [L]
RewriteCond %{REQUEST_URI} !(django.fcgi)
RewriteRule ^(.*)$ django.fcgi/$1 [L]

chmod 755 ~/public_html/mysite/django.fcgi

That's right folks! The amazing Django team has announced the immediate availability of Django 1.0 final!

This release is perhaps one of the most highly anticipated events in the Python community. More than three years in the making, the Django Web framework is something that perfectionist developers love and enjoy using. I've been using it for just over a year now, and I can't even remember doing things any other way! It's absolutely amazing!!!

So go ahead and download and install it today! What are you waiting for?!

Last November, a former co-worker and I began brainstorming for a particular site. This site was intended to give people a way to remember quotes that they heard/read/smelled/absorbed/etc without much effort. Soon thereafter, QuoteBoards.org was born.

The site has been pretty boring until recently, though it has been performing its purpose since its first launch. Up until a couple weeks ago, we only had one truly "faithful" member. This member, dirid51, added quotes to his QuoteBoard regularly. He held the site record of 52 quotes until one day another member (Puck) blew is 52 quotes out of the water with nearly 150 (genuine, I might add) quotes in one day.

I got in touch with Puck, and he shared some invaluable feedback with me concerning QuoteBoards.org. With this feedback, I was able to make several improvements that helped Puck along the way as he added his collection of quotes. Currently, he as 1,002 quotes. Most of these were added within a two or three day window. The reason he's stopped adding quotes is because he's tired of transcribing them from printed documents.

With that in mind, I decided it was finally time to begin working on the Firefox extension that I had originally planned to build for QuoteBoards.org when my co-worker and I were initially brainstorming. The purpose of this extension was to make adding quotes from the Web even easier. So, with a few minutes here and there between laundry and such, I was able to build my very first Firefox extension: Quote Grabber.

Right now, the extension is marked as "experimental" and requires people to log into the Mozilla Add-ons site in order to install and use it. However, if you're feeling adventurous, you can bypass that requirement by installing it directly from QuoteBoards.org. There are also directions for using it there.

I hope that those of you who are reading this find the extension to be beneficial and useful. I had a good time developing it nonetheless.

I'm proud to announce the official release of yet another one of my side projects. I call it django-tracking, and it is a simple project capable of telling you how many users are currently on your site. If you look at the bottom of any page on codekoala.com, you will see a demonstration of what this project can be used for.

django-tracking also offers basic blacklisting capabilities. I had a chap who apparently setup a script to spam my blog with rubbish comments every 11 minutes recently. I devised a way to stop the comments from being posted, but I noticed from my logs that the script was still hitting my website consistently over the following days. As a result, I implemented the "Banned IP" feature in django-tracking, and any visitor (or script) accessing my site from that IP address receives an error page. Yay!

If you have "Big Brother"-like tendencies when it comes to your visitors, there is also a way for you to have a "live feed" of active users on your site. Works pretty well, if I do say so myself :)

For more information, hop on over to django-tracking's homepage. There you will also find installation instructions and details for configuration and usage.

Have fun!