Python Tip: Conditional Expressions

I learned something new today about Python 2.5 and newer. I thought it was so nifty that I decided to write a blog article about it. Hopefully someone out there finds it as interestingly useful as I do.

One of the things I find myself doing quite often in my code (be it Python, PHP, Java, or what have you) is a simple conditional assignment like so:

if foo:
    bar = 'baz'
else:
    bar = 'qux'

In a lot of languages these days, you can use a ternary operator as follows to turn these four lines into a one-liner:

<?php
$bar = $foo ? 'baz' : 'qux';
?>

However, this ternary operator does not exist in Python. I have used various means in the past to accomplish the same task without using code like we saw in the first code block above, but they were all very hackish. Today I was happy to learn that there is an official way to do it in Python:

bar = ('baz' if foo else 'qux')

I'm not sure which one is more confusing to newbies: Python's conditional expressions or other languages' ternary operator. Personally, I prefer constructs like these to make my code more concise. I my mind, they also make the code more readable. I have heard some folks argue that ternary operators and the like obfuscate the code more than necessary, so they discourage the use of such tactics and recommend using the classic approach featured in the first example.

For those who are interested, I learned about it and a few other neat things in Python 2.5 and newer at http://www.python.org/doc/2.5/whatsnew/pep-308.html.

Miscellaneous Site Updates

I figured I should probably post something since I haven't done so yet this year. I've been making several changes to the site lately. Most of them are pretty subtle, but I hope they're useful to you. Updates include

  • You can now send an article to some friends by clicking on the "envelope" icon in the top-right corner of each article. It's a pretty simple mechanism.

  • You can save any of my articles to your hard drive for later consumption in the form of a PDF. Just click the "save" button in the top right corner of any article, and you will be able to view/save the latest revision of the respective article as a PDF. One thing to note on this, though, is that the program I use to generate the PDFs does not support the line numbers in code blocks. Sorry folks.

    For those of you who are interested, I'm using rst2pdf to generate the PDFs from my reStructuredText-based articles.

  • I've removed the calendar from the sidebar and replaced it with my newest useless side project: django-bibliophile. It allows me to share my reading progress with my visitors, because I know you all care that much. I plan on officially releasing the project in the near future.

  • Pagination has been implemented in parts of the article archive.

  • I've added an "article distribution" chart when looking at a year's blog articles.

  • Other random improvements.

GIT-SVN on Slackware 12.2

With all of the hype that git has been receiving lately, I started playing with it a while back to see if it suited me and my wants/needs. I found it to be an interesting utility. I won't go into any details simply because I'm not really all that knowledgeable about all the ins and outs of version control systems, but I will say that I have decided I like it. I'm still not sure whether I prefer GIT over SVN or SVN over GIT.

My problem is that basically all of my projects are based on SVN repositories. I don't want to have to start up a new GIT repository for each of my past projects. Fortunately, there is an interface for GIT to use SVN repositories called git-svn. I use this utility primarily on my EeePC because it saves a good amount of space on my small disk (the git-svn versions of the working copies are typically about half the size of their svn counterparts). Sometimes it's a little wacky, but it works well enough for my needs.

I started using this git-svn utility on a Debian-based distribution. That meant it was insanely simple to get up and running: sudo apt-get install git-svn. I recently installed Slackware 12.2, and I was surprised to find out that the git-svn utility wasn't immediately available to me.

I did some googling to see if others had encountered the same problem. There were several accurate hits, but I couldn't quite find the solution I needed. In the end, I finally got things working. The following information describes what I did to achieve this monumental success.

Trying git svn

The first roadblock that I encountered, obviously, was finding out that git-svn didn't work on my shiny new Slackware installation. After doing a bit of research, I learned that I could substitute the familiar git-svn command with git svn and continue using it as I previously had.

Installing Dependencies

Once I learned about git svn and tried it out, I got a nasty error about Alien/SVN. I've lost track of the original error, and for that I apologize. Doing a little bit of research led me to execute this command as root:

cpan Alien::SVN

I'm not sure exactly whether that step is required, but you might as well do it :).

Next, I downloaded a couple SlackBuilds to create my own Slackware packages suited for my computer.

For each SlackBuild, you must download the original source code along with the actual SlackBuild itself. For example, when retrieving the necessary files for swig, I must download both swig-1.3.35.tar.gz and swig.tar.gz from the link specified. Here are some example commands, which should be run as root:

mkdir -p ~/downloads/slackbuilds; cd ~/downloads/slackbuilds
wget http://slackbuilds.org/slackbuilds/12.2/development/swig.tar.gz
tar zxf swig.tar.gz
cd swig/
wget http://downloads.sourceforge.net/swig/swig-1.3.35.tar.gz
./swig.SlackBuild
installpkg /tmp/swig-1.3.35-i486-1_SBo.tgz

The commands above should create a new directory in /root/ called downloads/slackbuilds. Next, the SlackBuild for swig will be downloaded and extracted, after which the swig source code will be downloaded. The SlackBuild is executed, rendering an installable Slackware package. Finally, the package is installed onto the system.

The process is basically the same for the subversion-bindings SlackBuild. On my system, however, I had to modify the stock SlackBuild slightly. I didn't install Apache on my EeePC because I don't use it and it would just be taking up space. When I tried to execute the SlackBuild for subversion-bindings straight from the archive, it complained about a missing apxs file, which has something to do with Apache.

To avoid the error, I modified the subversion-bindings.SlackBuild script to ignore the apxs thingy. The original ./configure section looked like this:

CFLAGS="$SLKCFLAGS" \
./configure \
  --prefix=/usr \
  --mandir=/usr/man \
  --enable-shared \
  --disable-static \
  --with-apr=/usr \
  --with-apr-util=/usr \
  --with-apxs=/usr/sbin/apxs \
  --with-neon=/usr \
  --with-zlib=/usr \
  --with-pic \
  --with-ssl \
  --build=$ARCH-slackware-linux

I just removed the line that says --with-apxs=/usr/sbin/apxs \ and ran the SlackBuild script again. Worked like a charm.

At this point everything appeared to be able to work properly. Running git svn from the command line no longer spit out that nasty error I mentioned earlier. Instead it gave me the options I would expect to see.

That's when I tried to update an existing working copy of an SVN repository. It gave me this error:

$ git svn rebase
Authentication realm: <http://special.domain.com:80> Subversion - code
Password for 'myuser': Can't locate Term/ReadKey.pm in @INC (@INC contains:
/usr/lib/perl5/site_perl/5.10.0/i486-linux-thread-multi /usr/lib/perl5
/site_perl/5.10.0 /usr/lib/perl5/5.10.0/i486-linux-thread-multi /usr/lib
/perl5/5.10.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.10.0
/i486-linux-thread-multi /usr/lib/perl5/vendor_perl/5.10.0 /usr/lib/perl5
/vendor_perl .) at /usr/libexec/git-core/git-svn line 3071.

That's not very nice, now is it? The solution was fairly simple: install Perl's Term::ReadKey module. As root, execute the following command:

cpan Term::ReadKey

After doing that I was able to happily update my working copy and move on.

I don't envision that this article will be the all-knowing, all-powerful resource for how to use git-svn on Slackware, but I sure hope it will help some other folks who run into the same problems as me.

Installing Slackware 12.2 On Your EeePC (701 4G, in my case)

Welcome to my second article about installing Slackware on an Asus EeePC. This is a follow-up article to the one I posted in May 2008 soon after Slackware 12.1 was released. In this article, I will assume that you're doing a fresh installation of Slackware 12.2 and that you have access to an external USB CD/DVD ROM drive.

In all honesty, the installation process is extremely similar to what I did with 12.1. However, looking back at my previous article, I realize that my steps may not have been the most useful in the world. This time around I will try to be more helpful.

Getting Slackware

The first, and most obvious step, is to get a copy of Slackware. Simply head on over to http://www.slackware.com/getslack/ and retrieve the appropriate ISO(s) using whichever method you prefer. I downloaded the DVD version of Slackware. If you download the CD ISOs, you really only need the first 3 ISOs. The remaining 3 are source packages for the binary packages you install from the first three discs. Rarely do you need the source code for these packages.

After retrieving the Slackware ISO(s), you must burn them to a disc of some sort: ISOs that are ~650MB should be burned to CDs and anything larger should (obviously) be burned to a DVD. Be sure you burn each ISO using the "burn disc image" functionality in your disc writing software--simply burning the ISO file onto the disc in a regular data session will not do what we need.

Booting The Install Disc

After you have a good copy of the installation disc (the DVD or the first of the CDs), put the disc into your CD/DVD ROM drive and reboot your computer. To ensure that your computer boots from the disc rather than the hard drive, hit F2 when you see the initial boot screen. Then go to the "Boot" tab and verify that your external CD/DVD drive takes precedence over the internal SSD. While we're in the BIOS, let's hop over to the "Advanced" tab and set "OS Installation" to "Start". This will increase the chances that your external drive will be recognized or something.... mine didn't work until I made that change. When you're all done with that, exit your BIOS, saving your changes.

The computer will reboot, and it should access your installation disc immediately after the initial boot screen disappears. Once you boot from the installation disc, you should be presented with a screen which allows you to pass some settings to the installation kernel.

The installation boot screen

To make the installation go faster, use the following boot string:

hugesmp.s hdc=noprobe

This makes it so the installation will see the internal SSD as /dev/sda instead of /dev/hdc, which also boosts the read/write times by about 13 times.

During the boot process you will be asked to specify your keyboard map. Unless you want something special here, just hit the enter key to proceed.

Partition Your SSD

Next you will need to login as root and partition your SSD. You can do this using one of the following two commands:

fdisk /dev/sda
cfdisk /dev/sda

Here are some steps in case you're not familiar with these utilities:

  1. Remove all partitions (unless you know what you're doing)
    1. fdisk: d to delete (you may have to select multiple partitions to delete if you have more than one for some reason)
    2. cfdisk: Select all partitions individually with up/down arrow keys and use the left/right arrow keys to select delete from the menu at the bottom. Hit enter to run the delete command when it's highlighted.
  2. Create one partition that takes the whole SSD (again, unless you know what you're doing)
    1. fdisk: n (for new); enter; p (for primary); enter; 1 (for the first primary partition); enter; enter (to start at the beginning of the drive); enter (to select the end of the drive)
    2. cfdisk: Select the new command with the left/right arrow keys and hit enter when it's selected. Make it a primary partition, and have it take the whole SSD (3997.49MB in my case).
  3. Set the type of the new partition to be Linux
    1. fdisk: t (for type); enter; 83 (for Linux); enter
    2. cfdisk: Use the left/right arrow keys to select the type command at the bottom and hit enter when it's selected. Choose 83.
  4. Set the new partition (or the first, if you decided to make more than one) to be bootable
    1. fdisk: a (for bootable); enter; 1 (for primary partition 1); enter
    2. cfdisk: Select the bootable command from the bottom using the left/right arrow keys. Hit enter when it's selected.
  5. Write the changes to the partition table and quit
    1. fdisk: w
    2. cfdisk: Use the left/right arrow keys to select the write command from the bottom. Hit enter when it's selected. Type 'yes' to verify your intent, acknowledging that your previous data will be "gone". Then select the quit command.

Installing Slackware

As soon as your partitioning has finished, go ahead and run setup to begin the actual installation program.

The first screen of the installation program

Since we don't have a swap partition, can jump straight to the TARGET option. Use the arrow keys to highlight this option and hit enter. Select /dev/sda1 from the list, and format it with ext2. On the EeePC, most people prefer this format since it is a non-journaling filesystem. That means fewer writes to the SSD, which supposedly translates to a longer lifetime.

After the SSD is formatted, you will be asked to select the installation source. Again, I'm assuming that you want to use your fresh Slackware 12.2 disc, but you are free to choose what you want if you know what you're doing.

Selecting the installation source

I went with the default "Install from a Slackware CD or DVD" and told it to auto scan for my disc drive. It was found at /dev/sr0.

Choosing Your Packages

Next, you are given the opportunity to tweak the package series which will be installed on your EeePC. I chose the following series: A, AP, K, L, N, TCL, X, and XAP. I planned on using XFCE instead of KDE on my EeePC simply because it is much more light-weight and still capable of what I need. If you want KDE, be sure to check the appropriate series.

Selecting the packages to install

Once you mark each of the package series you wish to install, hit the "OK" button. You'll then have to choose which prompting mode to use. I chose menu, simply to be a little more picky about which packages I wanted installed. Installation took approximately 28 minutes with my package selection and setup.

Configuring Your System

When all of the packages are done being installed, you will be presented with some other screens to finish up the installation process.

  1. Choose whether or not you want to make a bootable USB... I skipped it.
  2. Choose how you wish to install LILO. I chose simple.
  3. Choose your frame buffer mode for the console. I chose 640x480x256.
  4. Specify any optional kernel parameters. Ensure that the hdc=noprobe from earlier is here to speed up your system considerably.
  5. Specify whether you wish to use UTF-8 on the console. I chose no.
  6. Specify where to install LILO. I chose MBR.
  7. Specify your mouse type. I chose imps2.
  8. Specify whether or not you wish to have gpm run at boot, which allows you to use your mouse in the console. I chose yes.
  9. Configure your network.
  10. Give your EeePC a hostname. This can be whatever you'd like.
  11. Specify the domain for your network. This can be whatever you'd like as well.
  12. Configure your IP address information. I just chose DHCP.
  13. Set the DHCP hostname. I left this blank.
  14. Review and confirm your network settings.
  15. Choose which services you wish to have running immediately after booting.
  16. See if you want to try custom screen fonts. I usually don't bother.
  17. Specify whether your hardware clock is set to local time or UTC.
  18. Choose your timezone.
  19. Select your preferred window manager. I chose XFCE.
  20. Set the root password.

At this point Slackware has been installed on your EeePC and you can exit the setup menu and hit Ctrl-Alt-Delete to reboot your computer.

First Boot

You should now go back into your BIOS and set "OS Installation" back to "Finished", exit and save changes, and reboot again.

Slackware's default LILO boot screen

You should then see the Slackware boot screen. By default, it has a 2-minute timeout, which seems absolutely absurd to me, so we'll change that later. Just hit enter for now and watch your new Slackware boot. The first boot will usually take a bit longer than subsequent reboots because all sorts of things need to generate their first configuration file.

When your system is ready, you'll be presented with a login prompt. Just login as root, using the password you specified in the last step of the installation process.

Tweaking Your Slackware

Here are some of the first things I do when I install a new copy of Slackware:

Add An Unprivileged User

This step is very important, because one thing that sets Linux apart from other operating systems is security ;). If you run your Linux system as root all the time, you're begging for problems.

To create a new unprivileged user, I use the adduser command. It walks you through the process of creating a user. This is the user you should use to do your day-to-day computing. Only use the root user when performing system administration tasks. Trust me :)

Tell X Windows to Start Automatically

I have no problem with the command line interface in Linux. I actually enjoy it quite a bit. However, on a device such as the EeePC, not having a GUI just doesn't seem all that practical. It's also not very impressive to your potential converts when they look over your shoulder and see that your tiny gadget just displays a black and white screen when you turn it on...

So, to help ourselves be a little more productive and to impress our followers, let's tell X Windows to start up automatically when we turn on the computer. To do that, we want to edit /etc/inittab and change the following line:

id:3:initdefault:

to be:

id:4:initdefault:

You can use whatever program you feel comfortable with, such as vi or nano. The next time you reboot your computer, you should see a GUI as soon as all of the services are fully loaded.

Along with this step, I suppose we can mention the configuration of X Windows. I usually run xorgsetup as root to get things up and running. Usually there is also a bit of tweaking to get things like the scroll wheel on the mouse to function. This part in particular took quite some time for me to figure out.

Enable The Scroll Wheel on the Trackpad

Some of you might be able to live without being able to scroll a page or whatever without using the scroll feature on most mouse devices these days, but I'm not one of them. Here is my entire /etc/X11/xorg.conf file:

Section "ServerLayout"
    Identifier     "X.org Configured"
    Screen      0  "Screen0" 0 0
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "SynapticMouse" "AlwaysCore"
    InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
    RgbPath      "/usr/share/X11/rgb"
    ModulePath   "/usr/lib/xorg/modules"
    FontPath     "/usr/share/fonts/TTF"
    FontPath     "/usr/share/fonts/OTF"
    FontPath     "/usr/share/fonts/Type1"
    FontPath     "/usr/share/fonts/misc"
    FontPath     "/usr/share/fonts/75dpi/:unscaled"
EndSection

Section "Module"
    Load  "xtrap"
    Load  "GLcore"
    Load  "record"
    Load  "dri"
    Load  "dbe"
    Load  "extmod"
    Load  "glx"
    Load  "freetype"
    Load  "type1"
    Load  "synaptics"
EndSection

Section "InputDevice"
    Identifier  "Keyboard0"
    Driver      "kbd"
    Option       "XkbModel"  "pc104"
    Option       "XkbLayout"  "us"
EndSection

Section "InputDevice"
    Identifier  "Mouse0"
    Driver "mouse"
    Option "Device" "/dev/input/mice"
    Option "Protocol" "IMPS/2"
    Option "Buttons" "5"
    Option "zAxisMapping" "4 5"
    Option "SHMConfig" "on"
EndSection

Section "InputDevice"
    Identifier "SynapticMouse"
    Driver "synaptics"
    Option "Device" "/dev/input/mice"
    Option "Protocol" "auto-dev"
    Option "SHMConfig" "on"
EndSection

Section "Monitor"
    Identifier   "Monitor0"
    VendorName   "Monitor Vendor"
    ModelName    "Monitor Model"
EndSection

Section "Device"
        ### Available Driver options are:-
        ### Values: <i>: integer, <f>: float, <bool>: "True"/"False",
        ### <string>: "String", <freq>: "<f> Hz/kHz/MHz"
        ### [arg]: arg optional
        #Option     "NoAccel"               # [<bool>]
        #Option     "SWcursor"              # [<bool>]
        #Option     "ColorKey"              # <i>
        #Option     "CacheLines"            # <i>
        #Option     "Dac6Bit"               # [<bool>]
        #Option     "DRI"                   # [<bool>]
        #Option     "NoDDC"                 # [<bool>]
        #Option     "ShowCache"             # [<bool>]
        #Option     "XvMCSurfaces"          # <i>
        #Option     "PageFlip"              # [<bool>]
    Identifier  "Card0"
    Driver      "intel"
    VendorName  "Intel Corporation"
    BoardName   "Mobile 915GM/GMS/910GML Express Graphics Controller"
    BusID       "PCI:0:2:0"
EndSection

Section "Screen"
    Identifier "Screen0"
    Device     "Card0"
    Monitor    "Monitor0"
    DefaultDepth 24
    SubSection "Display"
        Viewport   0 0
        Depth     1
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     4
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     8
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     15
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     16
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     24
    EndSubSection
EndSection

A lot of that stuff might not be necessary, but it's what works for me. Normally the process for enabling the scroll wheel is pretty easy, but something seems to have changed in this respect with the release of Slackware 12.2. I had to edit the /etc/modprobe.d/psmouse script to make this line:

options psmouse proto=imps

look like:

options psmouse proto=any

After making that change, things seemed to work a lot better.

Make LILO to Boot Faster

There are a couple tricks we can use to make LILO boot our EeePC slightly faster. The first is to add the compact option somewhere, and the second is to decrease the menu timeout.

Open up /etc/lilo.conf with a text editor of your choosing as root. Add a single line with the word compact somewhere. I put it under the line that says boot = /dev/sda so the top of lilo.conf looks like this:

# LILO configuration file
# generated by 'liloconfig'
#
# Start LILO global section
# Append any additional kernel parameters:
append="hdc=noprobe vt.default_utf8=8"
boot = /dev/sda
compact

I also changed the line that said timeout = 1200 to be timeout = 50 to make LILO only hang around for 5 seconds instead of 2 minutes.

After making these changes, we must reinstall LILO to the MBR with the new settings:

lilo -v

Here's my /etc/lilo.conf with most of the commented lines removed:

# LILO configuration file
# generated by 'liloconfig'
#
# Start LILO global section
# Append any additional kernel parameters:
append="hdc=noprobe vt.default_utf8=0"
boot = /dev/sda
compact

# Boot BMP Image.
# Bitmap in BMP format: 640x480x8
bitmap = /boot/slack.bmp
bmp-colors = 255,0,255,0,255,0
bmp-table = 60,6,1,16
bmp-timer = 65,27,0,255

prompt
timeout = 50
change-rules
reset
vga = normal
# End LILO global section
# Linux bootable partition config begins
image = /boot/vmlinuz
root = /dev/sda1
label = Linux
read-only
# Linux bootable partition config ends

Network Tweaking

While the wireless adapter seemed to work great for me out of the box this time, the ethernet adapter is still not functional. I compiled and installed the atl2 driver to solve the problem. You can get it from http://people.redhat.com/csnook/atl2/atl2-2.0.4.tar.bz2. Here are the steps I took to install it:

wget http://people.redhat.com/csnook/atl2/atl2-2.0.4.tar.bz2
tar jxf atl2-2.0.4.tar.bz2
cd atl2-2.0.4
make
cp atl2.ko /lib/modules/`uname -r`/kernel/drivers/net/
depmod -a
modprobe atl2
ifconfig

The next tweak I added for networking was to boost boot times... The DHCP address request hangs the entire boot process out of the box if you don't have an ethernet cable plugged in while booting. To remedy this problem, add the following line to the first section of your /etc/rc.d/rc.inet1.conf:

DHCP_TIMEOUT[0]="5"

This will tell your computer to continue booting if an IP address hasn't been assigned after 5 seconds of waiting.

Enable Frequency Scaling

We all like out battery to last a long time, right? Well, the EeePC 701 doesn't have the greatest battery in the world, but we can help increase the battery life by enabling the CPU frequency modules. I put this stuff in my /etc/rc.d/rc.local script:

#!/bin/sh
#
# /etc/rc.d/rc.local:  Local system initialization script.
#
# Put any local startup commands in here.  Also, if you have
# anything that needs to be run at shutdown time you can
# make an /etc/rc.d/rc.local_shutdown script and put those
# commands in there.

modprobe p4-clockmod
modprobe cpufreq_ondemand
modprobe cpufreq_conservative
modprobe cpufreq_powersave
modprobe cpufreq_performance

cpufreq-set -g ondemand -d 450Mhz -u 900Mhz

Add Your SD Card to /etc/fstab

I have an SD card that I leave in my EeePC all the time, and it's formatted with ext2 just like the internal SSD. Without this tweak, I have to mount the SD card each time I turn on the computer, which gets bothersome. My fix is to add the SD card to /etc/fstab, which takes care of mounting the device at boot.

First, you should make a directory that will be used to mount the device. I made one as such:

mkdir /mnt/sd

Now you need to determine your SD card's UUID. I started out by unmounting my SD card and taking it out of the slot. Then I executed this command:

ls /dev/disk/by-uuid

Next, I popped the SD card back in and executed that command again. The UUID that appears the second time but not the first time is your SD card's UUID.

It's time to add the magic line to your /etc/fstab. Add a line such as:

UUID=[your SD card's UUID] /mnt/sd ext2 defaults,noatime 1 1

somewhere in the file. While we're digging around in /etc/fstab, we might as well add the noatime option to the internal SSD to help reduce disk writes. Save the file and exit the editor. Then mount everything (using mount -a) or just the SD card (using mount /mnt/sd).

For posterity's sake, here's my entire /etc/fstab file:

/dev/sda1        /                ext2        defaults,noatime         1   1
UUID=30293ff4-5bee-457a-8528-ec296f099e9a /mnt/sd ext2 defaults,noatime 1 1
#/dev/cdrom      /mnt/cdrom       auto        noauto,owner,ro  0   0
/dev/fd0         /mnt/floppy      auto        noauto,owner     0   0
devpts           /dev/pts         devpts      gid=5,mode=620   0   0
proc             /proc            proc        defaults         0   0
tmpfs            /dev/shm         tmpfs       defaults         0   0

Preventing Shutdown Hangs

Sometimes the sound card seems to make Slackware hang when you're shutting down. Everything seems to turn off fine, but the little green power LED still shines bright. The solution to this problem appears to be adding the following line:

modprobe -r snd_hda_intel

to /etc/rc.d/rc.6 right before the "Unmounting local file systems." line (around line 195).

Enable Volume Hotkeys and Sleeping

Slackware 12.2 is already listening for ACPI events by default, so we just need to insert our custom stuff into /etc/acpi/acpi_handler.sh:

#!/bin/sh

IFS=${IFS}/
set $@

#logger "ACPI Event $1, $2, $3, $4, $5"

case "$1" in
    button)
        case "$2" in
            power) /sbin/init 0;;
            sleep) /etc/acpi/actions/lid.sh;;
            lid)
                if grep -q closed /proc/acpi/button/lid/LID/state
                then
                    /etc/acpi/actions/lid.sh
                fi
                ;;
            *) logger "ACPI action $2 is not defined";;
        esac
        ;;
    hotkey)
        case "$3" in
            # Fn+F2 Wireless/Bluetooth button
            # Fn+F7 Mute button
            00000013) amixer set Master toggle;;
            # Fn+F8 Volume down
            00000014) amixer set Master 10%-;;
            # Fn+F9 Volume up
            00000015) amixer set Master 10%+;;
        esac
        ;;
    *) logger "ACPI group $1 / action $2 is not defined";;
esac

And to handle the closing of the lid or pressing the sleep button, we need to create a new script in /etc/acpi/actions/ called lid.sh:

#!/bin/sh
# script by Fluxx from linuxquestions slackware forum
# discover video card's ID
ID=`/sbin/lspci | grep VGA | awk '{ print $1 }' | sed -e 's@:@/@'`

# securely create a temporary file
TMP_FILE=`mktemp /tmp/video_state.XXXXXX`
trap 'rm -f $TMP_FILE' 0 1 15

# switch to virtual terminal 1 to avoid graphics
# corruption in X
chvt 1

/sbin/hwclock --systohc

# remove the webcam module
rmmod uvcvideo

# write all unwritten data (just in case)
sync

# dump current data from the video card to the
# temporary file
cat /proc/bus/pci/$ID > $TMP_FILE

# suspend-to-ram
# (samwise) not using this it stuffs up the screen brightness
echo -n mem > /sys/power/state

# suspend-to-disk
#echo -n disk > /sys/power/state

# standby
#echo -n standby > /sys/power/state

# force on for now...
xset dpms force on

/sbin/hwclock --hctosys

# restore the webcam module
modprobe uvcvideo

# restore video card data from the temporary file
# on resume
cat $TMP_FILE > /proc/bus/pci/$ID

# switch back to virtual terminal 2 (running X)
chvt 6; sleep 2
chvt 2

# remove temporary file
rm -f $TMP_FILE

And we need to make sure the script is executable:

chmod +x /etc/acpi/actions/lid.sh

These scripts should enable us to use the mute key, the increase/decrease volume keys, and the sleep key. They should also allow us to close the lid of the EeePC to put it to sleep. Occasionally, when you wake up the computer, you will just see a blank black screen. To get around this, switch back to VT2 by using the keystroke Ctrl+Alt+F2.

Install Special Packages

Slackware comes with a lot of awesome stuff right out of the box, but it is missing some very important utilities at the same time. Included in this list, for me, is a program called wicd, or a network connectivity manager. This is similar to the "Network Manager" utility found in other mainstream distributions like Ubuntu, Fedora, and openSuSE. Slackware has yet to include such a utility by default.

Anyway, wicd can be found in the extra directory on the Slackware DVD or the 3rd (?) CD. To install it, find the package on the disc (or download it from the Internet) and execute the following command:

installpkg wicd-1.5.6-noarch-2.tgz

Be sure to check out the extra directory on the Slackware install disc. There are some neat tools in there. Some excellent resources for Slackware packages include:

There are some utilities out there to help you in your quest to resolve package dependencies. Two of the major ones that I've used in the past are swaret and slapt-get.

Using Slackware 12.2

My Slackware 12.2-powered EeePC 701 4G

I have to give the Linux kernel hackers props--the 2.6.27.7 kernel is amazingly fast! I'm sure the fact that I'm running a fairly stock Slackware installation (as opposed to something like Ubuntu) helps the speed quite a bit too. This past semester I had Linux Mint 5 (XFCE edition) installed on my EeePC, and that seemed fairly responsive. Slackware blew me away though, and I can still do everything I want to do!

The webcam and sound card work out of the box, just like the wireless. I rarely use the webcam, but it's fun to play with, and my mom appreciates seeing me on Skype occasionally. The wireless connection quality exceeds what it was with the madwifi driver I was using with Slackware 12.1 and other distros like Linux Mint. Programs are ultra speedy and responsive, even with the processor clocked at 450Mhz. I love it!!!

Boot times could be better, but I'm not too concerned with it. My setup takes approximately 50 seconds from boot to a useable desktop interface. Not horrible by any means, but perhaps not the best for a netbook when all you want to do is check your e-mail.

I would like to see the Network Manager that so many other distributions offer in Slackware some day. The wicd application is nice, but it's not nearly as intuitive as Network Manager, and it seems to be relatively limited in its capabilities in comparison. I know I'm not alone in my desire to see Network Manager included, or at least available, for Slackware. It would be tremendously beneficial in a world where wireless networking and laptops are more and more pervasive. Using the command line to adjust your wireless connection settings each time you have to hop to a new access point is just annoying.

In the end, I'm excited to have Slackware on my EeePC once again. I think it will be around for quite a while this time.

Please comment with any advice or problems that you have in regards to installing Slackware 12.2 on an EeePC.

Slackware 12.2, Gmail Tasks and SMS

I just had to come up with a blog post about these newly released bits of awesomeness!!

  • Slackware 12.2 was announced this morning
  • Google has launched two new Labs features for Gmail: a simple task list and the ability to send SMS messages within Gmail itself. w00t

I wish I had more time to report other important releases, but I must study!!!

In PHP's Defense

So the other day I wrote up an article that sarcastically compared PHP and Python syntax a little. While I am completely serious when I say that I prefer Python's syntax a heck of a lot more than that of PHP, I thought it might be a good thing for me to demonstrate that the code I posted before could have been more appealing had it been thought out a little more. After a solid year of not really dealing with anything besides Python, I will share a feeble attempt at cleaning up/optimizing the PHP code.

Let's start with this snippet:

<?php
$this->result=(isset($tmp["UMstatus"])?$tmp["UMstatus"]:"Error");
$this->resultcode=(isset($tmp["UMresult"])?$tmp["UMresult"]:"E");
$this->authcode=(isset($tmp["UMauthCode"])?$tmp["UMauthCode"]:"");
$this->refnum=(isset($tmp["UMrefNum"])?$tmp["UMrefNum"]:"");
$this->batch=(isset($tmp["UMbatch"])?$tmp["UMbatch"]:"");
$this->avs_result=(isset($tmp["UMavsResult"])?$tmp["UMavsResult"]:"");
$this->avs_result_code=(isset($tmp["UMavsResultCode"])?$tmp["UMavsResultCode"]:"");
$this->cvv2_result=(isset($tmp["UMcvv2Result"])?$tmp["UMcvv2Result"]:"");
$this->cvv2_result_code=(isset($tmp["UMcvv2ResultCode"])?$tmp["UMcvv2ResultCode"]:"");
$this->vpas_result_code=(isset($tmp["UMvpasResultCode"])?$tmp["UMvpasResultCode"]:"");
$this->convertedamount=(isset($tmp["UMconvertedAmount"])?$tmp["UMconvertedAmount"]:"");
$this->convertedamountcurrency=(isset($tmp["UMconvertedAmountCurrency"])?$tmp["UMconvertedAmountCurrency"]:"");
$this->conversionrate=(isset($tmp["UMconversionRate"])?$tmp["UMconversionRate"]:"");
$this->error=(isset($tmp["UMerror"])?$tmp["UMerror"]:"");
$this->errorcode=(isset($tmp["UMerrorcode"])?$tmp["UMerrorcode"]:"10132");
$this->custnum=(isset($tmp["UMcustnum"])?$tmp["UMcustnum"]:"");

$this->avs=(isset($tmp["UMavsResult"])?$tmp["UMavsResult"]:"");
$this->cvv2=(isset($tmp["UMcvv2Result"])?$tmp["UMcvv2Result"]:"");
?>

We see a lot of duplicate code in this chunk of code. The only thing that really changes much are the variable names and associative array keys. If we had defined a function that looked something like this...

1
2
3
4
5
<?php
function assign($member, $arr, $key, $default='') {
    $this->$member = isset($arr[$key]) ? $arr[$key] : $default;
}
?>

...things might just look a bit better. Let's see what the snippet might look like with this function defined in the same class:

<?php
$this->assign("result", $tmp, "UMstatus", "Error");
$this->assign("resultcode", $tmp, "UMresult", "E");
$this->assign("authcode", $tmp, "UMauthCode");
$this->assign("refnum", $tmp, "UMrefNum");
$this->assign("batch", $tmp, "UMbatch");
$this->assign("avs_result", $tmp, "UMavsResult");
$this->assign("avs_result_code", $tmp, "UMavsResultCode");
$this->assign("cvv2_result", $tmp, "UMcvv2Result");
$this->assign("cvv2_result_code", $tmp, "UMcvv2ResultCode");
$this->assign("vpas_result_code", $tmp, "UMvpasResultCode");
$this->assign("convertedamount", $tmp, "UMconvertedAmount");
$this->assign("convertedamountcurrency", $tmp, "UMconvertedAmountCurrency");
$this->assign("conversionrate", $tmp, "UMconversionRate");
$this->assign("error", $tmp, "UMerror");
$this->assign("errorcode", $tmp, "UMerrorcode", "10132");
$this->assign("custnum", $tmp, "UMcustnum");

$this->assign("avs", $tmp, "UMavsResult");
$this->assign("cvv2", $tmp, "UMcvv2Result");
?>

In my opinion, this still isn't as appealing as the Python solution, but I'd take it over the original code. It's a lot easier to read. This may or may not be the best solution on any level of scrutiny--feel free to comment with any suggestions for ways to further improve things.

The second snippet from my original post could use a lot more help than the first one. I don't know who these guys are who wrote the PHP USA ePay module, but I think they could use a little assistance. No offense if you're reading this article--just some friendly constructive criticism. I would expect no less from anyone else who was examining my code and found ways to improve its efficiency.

Here's the original:

<?php
switch(substr($ccnum,0,1))
{
    case 2: //enRoute - First four digits must be 2014 or 2149. Only valid length is 15 digits
        if((substr($ccnum,0,4) == "2014" || substr($ccnum,0,4) == "2149") && strlen($ccnum) == 15) return 20;
        break;
    case 3: //JCB - Um yuck, read the if statement below, and oh by the way 300 through 309 overlaps with diners club.  bummer.
        if((substr($ccnum,0,4) == "3088" || substr($ccnum,0,4) == "3096" || substr($ccnum,0,4) == "3112" || substr($ccnum,0,4) == "3158" || substr($ccnum,0,4) == "3337" ||
            (substr($ccnum,0,8) >= "35280000" ||substr($ccnum,0,8) <= "358999999")) && strlen($ccnum)==16)
        {
            return 28;
        } else {
            switch(substr($ccnum,1,1))
            {
                case 4:
                case 7: // American Express - First digit must be 3 and second digit 4 or 7. Only Valid length is 15
                    if(strlen($ccnum) == 15) return 3;
                    break;
                    case 0:
                case 6:
                case 8: //Diners Club/Carte Blanche - First digit must be 3 and second digit 0, 6 or 8. Only valid length is 14
                    if(strlen($ccnum) == 14) return 4;
                    break;
            }
        }
        break;
    case 4: // Visa - First digit must be a 4 and length must be either 13 or 16 digits.
        if(strlen($ccnum) == 13 || strlen($ccnum) == 16)
        {
            return 2;
        }
        break;

    case 5: // Mastercard - First digit must be a 5 and second digit must be int the range 1 to 5 inclusive. Only valid length is 16
        if((substr($ccnum,1,1) >=1 && substr($ccnum,1,1) <=5) && strlen($ccnum) == 16)
        {
            return 1;
        }
        break;
case 6: // Discover - First four digits must be 6011. Only valid length is 16 digits.
        if(substr($ccnum,0,4) == "6011" && strlen($ccnum) == 16) return 10;
}
?>

The first, and most obvious, improvement I would make to this code is to cram the substr($ccnum,0,4) junk into its own variable. It's used 8 different times up there. While substring operations might not be the most costly of functions out there, there's no need to repeatedly call the same function to get the same value that many times in the same block of code.

Similar to how I wrote the Python version, I would also throw the things that are repeatedly compared to the substr($ccnum,0,4) into an array and use the in_array function to increase readability. Oh, and consistent indentation (and not just because I like Python--it's good style to align things).

<?php
$four = substr($ccnum, 0, 4);
switch (substr($ccnum, 0, 1)) {
    case 2:
        /* enRoute - First four digits must be 2014 or 2149. Only valid
           length is 15 digits */
        if (in_array($four, array("2014", "2149")) && strlen($ccnum) == 15) return 20;
        break;
    case 3:
        /* JCB - Um yuck, read the if statement below, and oh by the way
           300 through 309 overlaps with diners club.  bummer. */
        if (in_array($four, array("3088", "3096", "3112", "3158", "3337")) ||
            in_array(substr($ccnum, 0, 8), array("35280000", "358999999")) &&
            strlen($ccnum) == 16) {
            return 28;
        } else {
            switch (substr($ccnum, 1, 1)) {
                case 4:
                case 7:
                    /* American Express - First digit must be 3 and second
                       digit 4 or 7. Only Valid length is 15 */
                    if(strlen($ccnum) == 15) return 3;
                    break;
                case 0:
                case 6:
                case 8:
                    /* Diners Club/Carte Blanche - First digit must be 3
                       and second digit 0, 6 or 8. Only valid length is 14 */
                    if(strlen($ccnum) == 14) return 4;
                    break;
            }
        }
        break;
    case 4:
        /* Visa - First digit must be a 4 and length must be either 13 or
           16 digits. */
        if (strlen($ccnum) == 13 || strlen($ccnum) == 16) {
            return 2;
        }
        break;
    case 5:
        /* Mastercard - First digit must be a 5 and second digit must be
           int the range 1 to 5 inclusive. Only valid length is 16 */
        if ($ccnum[1] >= 1 && $ccnum[1] <= 5 && strlen($ccnum) == 16) {
            return 1;
        }
        break;
    case 6:
        /* Discover - First four digits must be 6011. Only valid length
           is 16 digits. */
        if ($four == "6011" && strlen($ccnum) == 16) return 10;
}
?>

That just feels better to me. It should work exactly the same as the original snippet (though I admit I haven't tested it--don't even have PHP installed these days), but it just looks a heck of a lot better to me. Again, it might not be the most efficient way of accomplishing the desired task, but I consider these minor changes to make all the difference when you're required to maintain the code you wrote :)

You might notice that my version of the PHP is 10 lines longer than the original. That's mostly due to the fact that I try to respect the 80-character margin by wrapping lines before reaching that point. I believe this also adds to the pleasing appearance, but I realize that's more of a subjective thing these days.

Flame away folks!

Installing Django on Shared Hosting (Site5)

This article is a related to my previously posted article about installing Django, an advanced Web framework for perfectionists, on your own computer. Now we will learn how to install Django on a shared hosting account, using Site5 and fastcgi as an example. Depending on your host, you may or may not have to request additional privileges from the support team in order to execute some of these commands.

Note: Django requires at least Python 2.3. Newer versions of Python are preferred.

Note: This HOWTO assumes familiarity with the UNIX/Linux command line.

Note: If the wget command doesn't work for you (as in you don't have permission to run it), you might try curl [url] -O instead. That's a -O as in upper-case o.

Install Python

Site5 (and many other shared hosting providers that offer SSH access) already has Python installed, but you will want to have your own copy so you can install various tools without affecting other users. So go ahead and download virtual python:

mkdir ~/downloads
cd ~/downloads
wget http://peak.telecommunity.com/dist/virtual-python.py

Virtual Python will make a local copy of the installed Python in your home directory. Now you want to make sure you execute this next command with the newest version of Python available on your host. For example, Site5 offers both Python 2.3.4 and Python 2.4.3. We want to use Python 2.4.3. To verify the version of your Python, execute the following command:

python -V

If that displays Python 2.3.x or anything earlier, try using python2.4 -V or python2.5 -V instead. Whichever command renders the most recent version of Python is the one you should use in place of python in the next command. Since python -V currently displays Python 2.4.3 on my Site5 sandbox, I will execute the following command:

python ~/downloads/virtual-python.py

Again, this is just making a local copy of the Python installation that you used to run the virtual-python.py script. Your local installation is likely in ~/lib/python2.4/ (version could vary).

Make Your Local Python Be Default

To reduce confusion and hassle, let's give our new local installation of Python precedence over the system-wide Python. To do that, open up your ~/.bashrc and make sure it contains a line similar to this:

export PATH=$HOME/bin:$PATH

If you're unfamiliar with UNIX-based text editors such as vi, here is what you would type to use vi to make the appropriate changes:

  • vi ~/.bashrc to edit the file
  • go to the end of the file by using the down arrow key or the j key
  • hit o (the letter) to tell vi you want to start typing stuff on the next line
  • type export PATH=$HOME/bin:$PATH
  • hit the escape key
  • type :x to save the changes and quit. Don't forget the : at the beginning. Alternatively, you can type :wq, which works exactly the same as :x.

Once you've made the appropriate changes to ~/.bashrc, you need to make those changes take effect in your current SSH session:

source ~/.bashrc

Now we should verify that our changes actually took place. Type the following command:

which python

If they output of that command is not something like ~/bin/python or /home/[your username]/bin/python, something probably didn't work. If that's the case, you can try again, or simply remember to use ~/bin/python instead of python throughout the rest of this HOWTO.

Install Python's setuptools

Now we should install Python's setuptools to make our lives easier down the road.

cd ~/downloads
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py

This gives us access to a script called easy_install, which makes it easy to install many useful Python tools. We will use this a bit later.

Download Django

Let's now download the most recent development version of Django. SSH into your account and execute the following commands (all commands shall be executed on your host).

svn co http://code.djangoproject.com/svn/django/trunk ~/downloads/django-trunk

Now we should make a symlink (or shortcut) to Django and put it somewhere on the Python Path. A sure-fire place is your ~/lib/python2.4/site-packages/ directory (again, that location could vary from host to host):

ln -s ~/downloads/django-trunk/django ~/lib/python2.4/site-packages
ln -s ~/downloads/django-trunk/django/bin/django-admin.py ~/bin

Now verify that Django is installed and working by executing the following command:

python -c "import django; print django.get_version()"

That command should return something like 1.0-final-SVN-8964. If you got something like that, you're good to move onto the next section. If, however, you get something more along the lines of...

Traceback (most recent call last):
    File "<string>", line 1, in ?
ImportError: No module named django

...then your Django installation didn't work. If this is the case, make sure that you have a ~/downloads/django-trunk/django directory, and also verify that ~/lib/python2.4/site-packages actually exists.

Installing Dependencies

In order for your Django projects to become useful, we need to install some other packages: PIL (Python Imaging Library, required if you want to use Django's ImageField), MySQL-python (a MySQL database driver for Python), and flup (a utility for fastcgi-powered sites).

easy_install -f http://www.pythonware.com/products/pil/ Imaging
easy_install mysql-python
easy_install flup

Sometimes, using easy_install to install PIL doesn't go over too well because of your (lack of) permissions. To circumvent this situation, you can always download the actual PIL source code and install it manually.

cd ~/downloads
wget http://effbot.org/downloads/Imaging-1.1.6.tar.gz
tar zxf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6
ln -s ~/downloads/Imaging-1.1.6/PIL ~/lib/python2.4/site-packages

And to verify, you can try this command:

python -c "import PIL"

If that doesn't return anything, you're good to go. If it says something about "ImportError: No module named PIL", it didn't work. In that case, you have to come up with some other way of installing PIL.

Setting Up A Django Project

Let's attempt to setup a sample Django project.

mkdir -p ~/projects/django
cd ~/projects/django
django-admin.py startproject mysite
cd mysite
mkdir media templates

If that works, then you should be good to do the rest of your Django development on your server. If not, make sure that ~/downloads/django-trunk/django/bin/django-admin.py exists and that it has a functioning symlink (shortcut) in ~/bin. If not, you'll have to make adjustments according to your setup. Your directory structure should look something like:

  • projects
    • django
      • mysite
        • media
        • templates
        • __init__.py
        • manage.py
        • settings.py
        • urls.py

Making A Django Project Live

Now we need to make your Django project accessible from the Web. On Site5, I generally use either a subdomain or a brand new domain when setting up a Django project. If you plan on having other projects accessible on the same hosting account, I recommend you do the same. Let's assume you setup a subdomain such as mysite.mydomain.com. On Site5, you would go to ~/public_html/mysite for the next few commands. This could differ from host to host, so I won't go into much more detail than that.

Once you're in the proper place, you need to setup a few things: two symlinks, a django.fcgi, and a custom .htaccess file. Let's begin with the symlinks.

ln -s ~/projects/django/mysite/media ~/public_html/mysite/static
ln -s ~/lib/python2.4/site-packages/django/contrib/admin/media ~/public_html/mysite/media

This just makes it so you can have your media files (CSS, images, javascripts, etc) in a different location than in your public_html.

Now for the django.fcgi. This file is what tells the webserver to execute your Django project.

#!/home/[your username]/bin/python
import sys, os

# Add a custom Python path.
sys.path.insert(0, "/home/[your username]/projects/django")

# Switch to the directory of your project. (Optional.)
os.chdir("/home/[your username]/projects/django/mysite")

# Set the DJANGO_SETTINGS_MODULE environment variable.
os.environ['DJANGO_SETTINGS_MODULE'] = "mysite.settings"

from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")

And finally, the .htaccess file:

1
2
3
4
5
6
RewriteEngine On
RewriteBase /
RewriteRule ^(media/.*)$ - [L]
RewriteRule ^(static/.*)$ - [L]
RewriteCond %{REQUEST_URI} !(django.fcgi)
RewriteRule ^(.*)$ django.fcgi/$1 [L]

The .htaccess file makes it so that requests to http://mysite.mydomain.com/ are properly directed to your Django project. So, now you should have a directory structure that something that looks like this:

  • public_html
    • mysite
      • media
      • static
      • .htaccess
      • django.fcgi

If that looks good, go ahead and make the django.fcgi executable and non-writable by others:

chmod 755 ~/public_html/mysite/django.fcgi

After that, head over to http://mysite.mydomain.com/ (obviously, replace the mydomain accordingly). If you see a page that says you've successfully setup your Django site, you're good to go!

Afterthoughts

I've noticed that I need to "restart" my Django sites on Site5 any time I change the .py files. There are a couple methods of doing this. One includes killing off all of your python processes (killall ~/bin/python) and the other simply updates the timestamp on your django.fcgi (touch ~/public_html/mysite/django.fcgi). I find the former to be more destructive and unreliable than the latter. So, my advice is to use the touch method unless it doesn't work, in which case you can try the killall method.

Good luck!

Project Release: django-tracking 0.1

I'm proud to announce the official release of yet another one of my side projects. I call it django-tracking, and it is a simple project capable of telling you how many users are currently on your site. If you look at the bottom of any page on codekoala.com, you will see a demonstration of what this project can be used for.

django-tracking also offers basic blacklisting capabilities. I had a chap who apparently setup a script to spam my blog with rubbish comments every 11 minutes recently. I devised a way to stop the comments from being posted, but I noticed from my logs that the script was still hitting my website consistently over the following days. As a result, I implemented the "Banned IP" feature in django-tracking, and any visitor (or script) accessing my site from that IP address receives an error page. Yay!

If you have "Big Brother"-like tendencies when it comes to your visitors, there is also a way for you to have a "live feed" of active users on your site. Works pretty well, if I do say so myself :)

For more information, hop on over to django-tracking's homepage. There you will also find installation instructions and details for configuration and usage.

Have fun!

Using Django to Design Your Database Schema

Last night I had a buddy of mine ask me how I would approach a particular database design problem. I get similar questions quite often from my peers--suggests there is something important lacking from the database classes out there. Instead of answering him directly, I decided to come up with this tutorial for using Django to design your database schema.

For those of you new to Django, this article might seem a bit advanced. In time I will have more introductory-level Django tutorials, but I hope this one is easy enough.

Create a Django Project

The first step is to create a Django project. If you already have a project that you can play with, you can skip this step. To create a project, go to a place where you want to keep your code (like C:\projects or /home/me/projects) in a command prompt/terminal and run the following command:

django-admin.py startproject myproject

This will create a new directory in your current location called myproject (you can replace myproject with whatever you'd like so long as you're consistent). This new directory will contain a few files:

  • __init__.py
  • manage.py
  • settings.py
  • urls.py

If you get an error message when running the above command, you might not have Django installed properly. See Step-by-Step: Installing Django for details on installing Django.

Create An Application

Once you have a Django project setup, you should create a new application.

Note: If you're using Windows, you will probably need to omit the ./ on the ./manage.py commands. I will include them here for everyone else who's using Linux or a Mac.

cd myproject
./manage.py startapp specialapp

This will create a new directory in your myproject directory. This new directory will contain three files: __init__.py, models.py, and views.py. We are only concerned with the models.py file in this article.

Create Your Models

Models are usually a direct representation of what your database will be. Django makes creating these models extremely easy, and Python's syntax makes them quite readable. The Django framework asks for models to be defined in the models.py file that was created in the last step. Here's an example (for my buddy who prompted the creation of this article):

from django.db import models

class Component(models.Model):
    item_number = models.CharField(max_length=20)
    name = models.CharField(max_length=50)
    size = models.CharField(max_length=10)
    quantity = models.IntegerField(default=1)
    price = models.DecimalField(max_digits=8, decimal_places=2)

class Project(models.Model):
    name = models.CharField(max_length=50)
    components = models.ManyToManyField(Component)
    instructions = models.TextField()

(for more information about models, see the Django Model API Reference)

I don't know about you, but that code seems pretty straightforward to me. I'll spare you all the details about what's going on (that can be a future article).

Install Your New Application

Once you have your models setup, we need to add our specialapp to our list of INSTALLED_APPS in order for Django to register these models. To do that, open up settings.py in your myproject directory, go to the bottom of the file, until you see something like

1
2
3
4
5
6
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
)

When you find that, add your specialapp to the list

1
2
3
4
5
6
7
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'specialapp'
)

Setup Your Database

Now you need to let Django know what kind of database you're using. Django currently supports MySQL, SQLite3, PostgreSQL, and Oracle natively, but you can get third-party tools that allow you to use other database (like SQL Server).

Still in your settings.py, go to the top until you see DATABASE_ENGINE and DATABASE_NAME. Set that to whatever type of database you are using:

DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = 'myproject.db'

Save your settings.py and go back to your command prompt/terminal.

Get Django's Opinion For Your Schema

Make sure you're in your myproject directory and run the following command:

./manage.py sqlall specialapp

This command will examine the models that we created previously and will generate the appropriate SQL to create the tables for your particular database. For SQLite, we get something like this for output:

BEGIN;
CREATE TABLE "specialapp_component" (
      "id" integer NOT NULL PRIMARY KEY,
      "item_number" varchar(20) NOT NULL,
      "name" varchar(50) NOT NULL,
      "size" varchar(10) NOT NULL,
      "quantity" integer NOT NULL,
      "price" decimal NOT NULL
)
;
CREATE TABLE "specialapp_project" (
      "id" integer NOT NULL PRIMARY KEY,
      "name" varchar(50) NOT NULL,
      "instructions" text NOT NULL
)
;
CREATE TABLE "specialapp_project_components" (
      "id" integer NOT NULL PRIMARY KEY,
      "project_id" integer NOT NULL REFERENCES "specialapp_project" ("id"),
      "component_id" integer NOT NULL REFERENCES "specialapp_component" ("id"),
      UNIQUE ("project_id", "component_id")
)
;
COMMIT;

Notice how Django does all sorts of nifty things, like wrapping the table creation queries in a transaction, setting up indexes, unique keys, and defining relationships between tables. The output also offers a solution to the original problem my buddy had: an intermediate table that just keeps track of relationships between projects and components (the specialapp_project_components table).

Notice that the SQL above may not work with database servers other than SQLite.

Enhancing The Intermediate Table

After my buddy reviewed this article, he asked a very interesting and valid question: What if a project needs 3 of one component? In response, I offer the following models (this requires a modern version of Django--it doesn't work on Django 0.96.1 or earlier):

from django.db import models

class Component(models.Model):
    item_number = models.CharField(max_length=20)
    name = models.CharField(max_length=50)
    size = models.CharField(max_length=10)
    quantity = models.IntegerField(default=1)
    price = models.DecimalField(max_digits=8, decimal_places=2)

class Project(models.Model):
    name = models.CharField(max_length=50)
    components = models.ManyToManyField(Component, through='ProjectComponent')
    instructions = models.TextField()

class ProjectComponent(models.Model):
    project = models.ForeignKey(Project)
    component = models.ForeignKey(Component)
    quantity = models.PositiveIntegerField()

    class Meta:
        unique_together = ['project', 'component']

Running ./manage.py sqlall specialapp now generates the following SQL:

BEGIN;
CREATE TABLE "specialapp_component" (
    "id" integer NOT NULL PRIMARY KEY,
    "item_number" varchar(20) NOT NULL,
    "name" varchar(50) NOT NULL,
    "size" varchar(10) NOT NULL,
    "quantity" integer NOT NULL,
    "price" decimal NOT NULL
)
;
CREATE TABLE "specialapp_project" (
    "id" integer NOT NULL PRIMARY KEY,
    "name" varchar(50) NOT NULL,
    "instructions" text NOT NULL
)
;
CREATE TABLE "specialapp_projectcomponent" (
    "id" integer NOT NULL PRIMARY KEY,
    "project_id" integer NOT NULL REFERENCES "specialapp_project" ("id"),
    "component_id" integer NOT NULL REFERENCES "specialapp_component" ("id"),
    "quantity" integer unsigned NOT NULL,
    UNIQUE ("project_id", "component_id")
)
;
CREATE INDEX "specialapp_projectcomponent_project_id" ON "specialapp_projectcomponent" ("project_id");
CREATE INDEX "specialapp_projectcomponent_component_id" ON "specialapp_projectcomponent" ("component_id");
COMMIT;

As you can see, most of the SQL is the same. The main difference is that the specialapp_project_components table has become specialapp_projectcomponent and it now has a quantity column. This can be used to keep track of the quantity of each component that a project requires. You can add however many fields you want to this new intermediate table's model.

Using This SQL

There are several ways you can use the SQL generated by Django. If you want to make your life really easy, you can have Django create the tables for you directly. Assuming that you have specified all of the appropriate database information in your settings.py file, you can simply run the following command:

./manage.py syncdb

This will execute the queries generated earlier directly on your database, creating the tables (if they don't already exist). Please note that this command currently will not update your schema if the table exists but is missing a column or two. You must either do that manually or drop the table in question and then execute the syncdb command.

Another option, if you want to keep your DDL(Data Definition Language) in a separate script (maybe if you want to keep it in some sort of version control) is something like:

./manage.py sqlall specialapp > specialapp-ddl-080813.sql

This just puts the output of the sqlall command into a file called specialapp-ddl-080813.sql for later use.

Benefits of Using Django To Create Your Schema

  • Simple: I personally find the syntax of Django models to be very simple and direct. There is a comprehensive API that explains and demonstrates what Django models are capable of.
  • Fast: Being that the syntax is so simple, I find that it makes designing and defining your schema much faster than trying to do it with raw SQL or using a database administration GUI.
  • Understandable: Looking at the model code in Django is not nearly as intimidating as similar solutions in other frameworks (think about Java Persistence API models).
  • Intelligent: Using the same model code, Django can generate proper Data Definition Language SQL for several popular database servers. It handles indexes, keys, relationships, transactions, etc. and can tell the difference between server types.

Downfalls of Using Django To Create Your Schema

  • The Table Prefix: Notice how all of the tables in the SQL above were prefixed with specialapp_. That's Django's safe way of making sure models from different applications in the same Django project do not interfere with each other. However, if you don't plan on using Django for your end project, the prefix could be a major annoyance. There are a couple solutions:
    • A simple "search and replace" before executing the SQL in your database
    • Define the db_table option in your models
  • Another Technology: Django (or even Python) may or may not be in your organization's current development stack. If it's not, using the methods described in this article would just become one more thing to support.

Other Thoughts

I first thought about doing the things mentioned in this application when I was working on a personal Java application. I like to use JPA when developing database-backed applications in Java because it abstracts away a lot of the database operations. However, I don't like coming up with the model classes directly, so I usually reverse engineer them from existing database tables.

Before thinking about the things discussed in this article, I created the tables by hand, making several modifications to the schema before I was satisfied with my JPA models. This proved to be quite bothersome and time-consuming.

After using Django to develop my tables, the JPA models turned out to be a lot more reliable, and they were usually designed properly from the get-go. I haven't created tables manually ever since.

If you find yourself designing database schemas often, and you find that you have to make several changes to your tables before you/the project requirements are satisfied, you might consider using Django to do the grunt work. It's worked for me, and I'm sure it will work for you too.

Good luck!

Step-by-Step: Installing Django

Being the Django and Python zealot that I am, I often find myself trying to convert those around me to this awesome development paradigm. Once I break them, these people often ask me a lot of questions about Django. Over the past few months I've noticed that one of the biggest sticking points for people who are new to Django is actually getting it up and running to begin with. In response, this is the first in a series of articles dedicated to getting Django up and running.

What is Django?

The Django Web site describes Django as "a high-level Python Web framework that encourages rapid development and clean, pragmatic design." Basically, Django is just about the most amazing thing for Web development. I have tinkered with several different Web technologies, but nothing seems to even come close to what Django can do for me.

What is Python?

Python is a programming language used in numerous aspects of computing these days. It has a very simple yet powerful syntax. It's an easy language for beginners to pick up, but it provides adequate levels of power for the more experienced developers out there. If you have never programmed anything before, or you have dabbled with something like BASIC, Python should be fairly straightforward. If you are a programming veteran, but have only worked with languages like C, C++, Java, etc, you might struggle a bit with the syntax of the language. It's not difficult to overcome the differences in a couple hours of hands-on development.

Let's get started.

Installing Python...

Having Python installed is critical--Django does not work without Python. I'm guessing that you're relatively familiar with the procedures for installing software packages on your particular operating system. However, I will share a few notes to point you in the proper direction if you're lost. If nothing else, just head over to the Python download page to download anything you need to install Python. I whole-heartedly recommend using the latest stable version of Python for Django, but you should be able to get by with as early a version as 2.3.

...On Windows

Simply grab the latest version of the Python installer. It is currently version 2.5.2. Once the installer has downloaded successfully, just run through the installation wizard like any other setup program.

...On Mac OS X

Recent Mac OS X computers come with Python pre-installed. To determine whether or not you actually have it, launch the Terminal (Applications > Utilities > Terminal) and type python -c "import sys; print sys.version". If Python is already installed, you will see the version you have installed. If you have a version that is less than 2.3, you should download the newest version. If you don't have Python installed, you will get a "command not found" error. If you're in this boat, just download the latest version of the Python Universal installer and install it.

...On Linux

Most Linux distributions also have Python pre-installed. Just like with Mac OS X, you can check to see by opening up a terminal/konsole session and running the command python -c "import sys; print sys.version". If you have Python installed, you will see its version. If you get an error message when running that command, or you have a version earlier than 2.3, you need to download and install the latest version of Python.

If you're running a Debian-based distribution (like Ubuntu, sidux, MEPIS, KNOPPIX, etc), you can probably use sudo apt-get install python to get Python. If you're running an RPM-based Distribution, you can probably use something like Yum or YaST to install Python.

A sure-fire way to install Python on any Linux system, however, is to install from source. If you need to do this, you simply:

  1. download the source for the latest version of Python
  2. extract it: tar jxf Python-2.5.2.tar.bz2
  3. go into the newly-extracted directory: cd python-2.5.2
  4. configure it: ./configure
  5. compile it: make
  6. install it: make install

(I've only installed Python from source one time, so I might be wrong)

Setting Up Your PYTHONPATH...

Generally speaking, if you didn't have Python installed before starting this tutorial, you will need to setup your PYTHONPATH environment variable. This is a variable that lets Python know where to find useful things (like Django).

...On Windows

  • Open up your System Properties (Win+Break or right click on "My Computer" on your desktop and select Properties)
  • Go to the "Advanced" tab
  • Click the "Environment Variables" button
  • If you have permission to change system variables, click the "New" button in the bottom pane. Otherwise, create the PYTHONPATH variable for your user account using the "New" button in the top (User variables for [username]) pane.
  • Set the variable name to PYTHONPATH
  • Set the variable value to C:\Python25\Lib\site-packages (replace C:\Python25\ with whatever it is on your system if needed)
  • Save it

You may also need to add the python executable to your PATH. If you can successfully run python from a command prompt window, you don't need to worry about it.

If you can't run python from a command prompt, follow the procedure above, but use the PATH variable instead of PYTHONPATH. PATH most likely already exists, so you just need to append/prepend the existing value with something like C:\Python25\ (again, this might need to change depending on where you installed Python)

...On Mac OS X

Your PYTHONPATH should already be setup for you.

...On Linux

Usually you just need to edit your ~/.bash_rc script to setup your PYTHONPATH environment variable. Go ahead and open that up in your preferred text editor and make sure there's something in it like:

export PYTHONPATH=/usr/lib/python2.5/site-packages:$PYTHONPATH

Save any changes necessary and run the following command:

source ~/.bash_rc

This will take care of updating your current session with any changes you made to your ~/.bash_rc.

Installing Django

Once you have Python and have verified that you have version 2.3 or later, you are ready to install Django. Currently, the latest stable release is 0.96.1, but this is grossly out-dated. Django 1.0 will be released on September 2nd 2008, so the "unstable" copy of Django is pretty close to what 1.0 will have to offer. There are some incredibly useful improvements in the unstable version that I don't think I could do without anymore, so that's what I'll talk about installing here.

First, you need to have a subversion client. On Windows, the most popular one is called TortoiseSVN. On Mac OS X, I have played with a few, but I think Versions is a pretty decent one. Linux also has several to choose from, but if you're using Linux, you're probably going to use the command line anyway (right?).

For brevity, I will just use the subversion commands necessary to accomplish this task (instead of discussing all GUI interfaces to subversion).

The exact location that Django should be installed differs from system to system, but here are some guidelines for typical setups:

  • Windows: C:\Python25\Lib\site-packages
  • Linux: /usr/lib/python2.5/site-packages
  • Mac OS X: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages

If you want a definite location, run the following command:

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Once you know that location, go there in your command prompt or terminal session. Then execute this command svn co http://code.djangoproject.com/svn/django/trunk/django django. You will see loads of output, showing all of the files that you are downloading as you install Django.

As soon as that process completes, you should run python -c "import django" to make sure everything worked properly. If the command doesn't display an ImportError, you're good. Otherwise, you need to try again.

Getting Access to Django Scripts...

Once you can successfully import django, you might want to make sure you can run the django-admin.py script that comes with Django.

...On Windows

This process is very similar to what we did with the PYTHONPATH environment variable earlier.

  • Open your System Properties again
  • Go to the Advanced tab
  • Click the Environment Variables button
  • Find your PATH environment variable (either for your user or system-wide)
  • Make sure that the variable value contains something like C:\Python25\Lib\site-packages\django\bin
  • Save any changes
  • Open a fresh command prompt
  • Try to run django-admin.py. If you're successful, you're ready to get started with Django. Otherwise, you need to fix your path to django/bin or just call the django-admin.py script using an absolute path when needed.

...On Mac OS X

You can run a command similar to this:

sudo ln -s /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/bin/django-admin.py /usr/local/bin

...On Linux

If you have "root" privileges on your Linux system, you can execute a command like:

sudo ln -s /usr/lib/python2.5/site-packages/django/bin/django-admin.py /usr/local/bin

If you don't have "root" privileges, you can setup your own /usr/local/bin:

mkdir ~/bin

Make sure your ~/.bash_rc contains something like:

export PATH=$HOME/bin:$PATH

Then update your current session with any changes you made to ~/.bash_rc by running this command:

source ~/.bash_rc

And that should do it! Now you should be ready to get started with Django.

Feel free to leave a comment if you're having problems installing Django. Good luck!

Check out Installing Django on Shared Hosting.