Django Projects

Over the past 6 years, I've built a lot of things with Django. It has treated me very well, and I have very much enjoyed seeing it progress. I got into Django when I helped the company I was working for transition away from a homegrown PHP framework toward something more reliable and flexible. It was very exciting to learn more about Django at a time when the ecosystem was very young.

When I started with Django, there weren't a lot of pluggable apps to fill the void for things like blogs, event calendars, and other useful utilities for the kinds of sites I was building. That has changed quite a bit since then. The ecosystem has evolved and progressed like mad, and it's wonderful. We have so many choices for simple things to very complex things. It's amazing!

Unfortunately, during this whole time period, my development efforts have shifted from creating my own open source projects to share with the world toward more proprietary solutions for my employers. If it's not obvious to you from my blog activity in recent years, I've become very busy with family life and work. I have very little time to give my open source projects the attention they deserve.

For at least 4 years, I've been telling myself that I'd have/make time to revamp all of my projects. To make them usable with what Django is today instead of what it was when I built the projects. Yeah, this time has never showed up. Take a look at the last time I wrote a blog article!

I have decided to disown pretty much all of my open source Django projects. I've basically done this with one or two of the more popular projects already--let someone else take the reigns while I lurk in the background and occasionally comment on an issue here or there. I truly appreciate those who have taken the initiative here. But there are still plenty of projects that people may find useful that need some attention. I'm putting it up to the community to take these projects over if you find them useful so they can get the love and attention they need.

Here is a list of Django projects that anyone is free to assume responsibility for. Most of them are silly and mostly useless now. Some are unpleasant to look at and could use an entire rewrite.

  • django-articles - Blogging engine, which I wrote for my own site many years ago. This one could probably use a full rewrite. [already maintained by John Leith].
  • django-tracking - Visitor tracking to see what page a user is on and their general location based on IP. Update 2015-07-28: Adopted by bashu.
  • django-axes - Track failed login attempts and ban users. This one is already pretty much owned by aclark4life
  • django-reploc - Designed to show representative/retailer locations. I only used it on one site, but it was designed to be generic.
  • django-smileys - To easily replace things like :) and B-) with emoticons or whatever. Pretty lame. Update 2015-07-28: Adopted by bashu.
  • django-pendulum - A very basic time-tracking app. I believe this has already been forked and used for some more interesting commercial projects.
  • django-bibliophile - A way to let your visitors know what you're reading.
  • django-watermark - Some utilities to add watermarks to images. Update 2015-07-28: Adopted by bashu.
  • Any other Django-related project you see at https://github.com/codekoala?tab=repositories or https://bitbucket.org/codekoala/

The fact that I'm giving up these projects does not mean I'm giving up on Django. On the contrary, I'm still using it quite heavily. I'm just doing it in such a way that I can't necessarily post my work for everyone to use. I honestly don't expect much of this disowning effort, since the projects are mostly stale and incompatible with recent versions of Django. But please let me know if you do want to take over one of my projects and care for it.

Arduino-Powered Webcam Mount

Earlier this month, I completed yet another journey around the biggest star in our galaxy. Some of my beloved family members thought this would be a good occasion to send me some cash, and I also got a gift card for being plain awesome at work. Even though we really do need a bigger car and whatnot, my wife insisted that I only spend this money on myself and whatever I wanted.

Little did she know the can of worms she just opened up.

I took pretty much all of the money and blew it on stuff for my electronics projects. Up to this point, my projects have all been pretty boring simply because nothing ever moved--it was mostly just lights turning on and off or changing colors. Sure, that's fun, but things really start to get interesting when you actually interact with the physical world. With the birthday money, I was finally able to buy a bunch of servos to begin living out my childhood dream of building robots.

My first project since getting all of my new toys was a motorized webcam mount. My parents bought me a Logitech C910 for my birthday because they were tired of trying to see their grandchildren with the crappy webcam that is built into my laptop. It was a perfect opportunity to use SparkFun's tutorial for some facial tracking (thanks to OpenCV) using their Pan/Tilt Servo Bracket.

It took a little while to get everything setup properly, but SparkFun's tutorial explains perfectly how you can get everything setup if you want to repeat this project.

The problem I had with the SparkFun tutorial, though, is that it basically only gives you a standalone program that does the facial tracking and displays your webcam feed. What good is that? I actually wanted to use this rig to chat with people!! That's when I set out to figure out how to do this.

While the Processing sketch ran absolutely perfect on Windows, it didn't want to work on my Arch Linux system due to some missing dependencies that I didn't know how/care to satisfy. As such, I opted to rewrite the sketch using Python so I could do the facial tracking in Linux.

This is still a work in progress, but here's the current facial tracking program which tells the Arduino where the webcam should be pointing, along with the Arduino sketch.

Now that I could track a face and move my webcam in Linux, I still faced the same problem as before: how can I use my face-tracking, webcam-moving program during a chat with my mom? I had no idea how to accomplish this. I figured I would have to either intercept the webcam feed as it was going to Skype or the Google Talk Plugin, or I'd have to somehow consume the webcam feed and proxy it back out as a V4L2 device that the Google Talk Plugin could then use.

Trying to come up with a way of doing that seemed rather impossible (at least in straight Python), but I eventually stumbled upon a couple little gems.

So the GStreamer tutorial walks you step-by-step through different ways of using a gst-launch utility, and I found this information very useful. I learned that you can use tee to split a webcam feed and do two different things with it. I wondered if it would be possible to split one webcam feed and send it to two other V4L2 devices.

Enter v4l2loopback.

I was able to install this module from Arch's AUR, and using it was super easy (you should be root for this):

modprobe v4l2loopback devices=2

This created two new /dev/video* devices on my system, which happened to be /dev/video4 and /dev/video5 (yeah... been playing with a lot of webcams and whatnot). One device, video4, is for consumption by my face-tracking program. The other, video5, is for VLC, Skype, Google+ Hangouts, etc. After creating those devices, I simply ran the following command as a regular user:

gst-launch-0.10 v4l2src device=/dev/video1 ! \
    'video/x-raw-yuv,width=640,height=480,framerate=30/1' ! \
    tee name=t_vid ! queue ! \
    v4l2sink sync=false device=/dev/video4 t_vid. ! \
    queue ! videorate ! 'video/x-raw-yuv,framerate=30/1' ! \
    v4l2sink device=/dev/video5

There's a whole lot of stuff going on in that command that I honestly do not understand. All I know is that it made it so both my face-tracking Python program AND VLC can consume the same video feed via two different V4L2 devices! A co-worker of mine agreed to have a quick Google+ Hangout with me to test this setup under "real" circumstances (thx man). It worked :D Objective reached!

I had really hoped to find a way to handle this stuff inside Python, but I have to admit that this is a pretty slick setup. A lot of things are still hardcoded, but I do plan on making things a little more generic soon enough.

So here's my little rig (why yes, I did mount it on top of an old Kool-Aid powder thingy lid):

And a video of it in action. Please excuse the subject of the webcam video, I'm not sure where that guy came from or why he's playing with my webcam.

2Ze.us Updates

There has been quite a bit of recent activity in my 2ze.us project since I first released it nearly a year ago. My intent was not to become a competitor with bit.ly, is.gd, or anyone else in the URL-shortening arena. I created the site as a way for me to learn more about Google's AppEngine. It didn't take very long to get it up and running, and it seemed to work fairly well.

AppEngine and Extensions

I was able to basically leave the site alone on AppEngine for several months--through about September 2009. In that time, I came up with a Firefox extension to make its use more convenient.

The extension allows you to quickly get a shortened URL for the page you're currently looking at, and a couple of context menu items let you get a short URL for things like specific images on a page. Also included in the extension is a preview for 2ze.us links. The preview can tell you the title and domain of the link's target. It can tell you how much smaller the 2ze.us URL is compared to the full URL. Finally, it displays how many times that particular 2ze.us link has been clicked.

That as all fine and dandy. It was the second Firefox extension I had ever written, and it's still running strong. In June or July of 2009, I started working on a little program to make it easier for me to interact with Twitter the way I wanted to. This was a great opportunity for me to incorporate 2ze.us into the application so any URL I wanted to post to Twitter would automatically be shortened for me, using my own shortener.

Porting to WebFaction And PHP

Anyway, around the end of September 2009, I noticed that there were a lot of problems with 2ze.us. It was slow and sometimes completely unresponsive. Certain URLs would redirect to their full URLs, while others wouldn't. The Firefox extension stopped working nicely. Oh yeah, and AppEngine rolled back to a previous revision of the code without me telling it to. That's when everything just died. It didn't take long for me to decide to migrate my project from AppEngine onto my awesome WebFaction hosting.

At this point, I was faced with a small dilemma: keep the code in Python, or port it to PHP. I opted to port it over to PHP, because I didn't want all of the overhead of a full Django instance for a site that needed to be very zippy. And I was unacquainted with other Python options.

By early October 2009, I had managed to turn the project into a PHP beast, running on Apache. It was a lot more responsive than AppEngine ever let 2ze.us be. There were a few bumps along the road, what with the extension and Twitter client relying on various parts of the site. Eventually it got to a point where I could just let it sit and work.

Chromium Extension

Sometime around the end of December, I decided to write another extension for 2ze.us, only for Google Chrome and Chromium this time. This extension isn't quite as feature-packed as its Firefox brother, but it gets the job done.

Clip2Zeus

Shortly after "completing" the Chromium extension, I had what seemed like a pretty original idea. Who knows if it really is, but I still haven't seen another tool quite like the one that I made as a result of this idea. I thought, "Now, why should I need to install an extension in each Web browser I use on each computer I use? Is there a better way?"

The answer came quickly: a standalone, desktop application. Write one program that handles shortening URLs for you. My laziness told me to make a program that monitors your system clipboard for URLs. If a URL is detected, try to shorten it, and update the clipboard contents in place. Boom. Done. All extensions become useless beyond things like the URL preview (which is very useful, imo).

The next question I asked was, "Do I make it platform-dependent? Should I stick it to the majority of computer users and write my tool for Linux only? For OSX only? For, uh... Windows only?" Again, an easy question to answer. Support them all or don't even bother writing the application.

A week's worth of midnight hacking saw the birth of Clip2Zeus 1.0a. It's a cross-platform compatible desktop application that does exactly what I just mentioned. When it's running and detects a URL on your system clipboard, it will try to shorten it and update it in your clipboard. If you copy a block of text, the application will only modify the URLs in that block of text--meaning the block of text will still be in your clipboard, but it will have shorter URLs.

I use the program every day at work (on OSX). It's been very fun for me to see a short URL any time I copy a nasty URL to my clipboard. Imagine that; I'm a big fan of my own work...

Tornado

Lately, I've noticed that the site was getting kind of slow again. Sometimes it would take several seconds for Clip2Zeus to shorten URLs in my clipboard, when it was normally instantaneous. Every once in a while, Clip2Zeus would completely fail to connect to the website.

One of my friends has asked me a lot of questions about the Tornado framework in the past months. I had read a few things about Tornado when it was open-sourced last year, but I didn't really feel the need to dabble with it. These questions prompted me to tinker a little.

Last night I re-ported 2ze.us to Python, using the Tornado framework this time. So far I'm very impressed with its responsiveness. The framework offers a lot of neat little utilities, and it is very fast (as reported by dozens of other reputable sources).

On top of the speed increase that came with the transition to Tornado, my RAM usage on WebFaction has come down by nearly 100MB. Just by turning off the one Apache-backed website. Now I'm nowhere near my RAM cap! Wahoo!!

Enough rambling. Like I said at the beginning of this article, a lot has been happening with this project in the past year. I didn't even think about all of the time I put into projects related to my simple little side project. Looking back, I'm quite satisfied with how things have unfolded.

Statistics

Here are some simple statistics for 2ze.us. Since March 2009...

  • 5,252 URLs have been shortened using 2ze.us
  • 2ze.us links have been clicked 198,267 times
  • 315,951 URL characters have been turned into 11,532 characters

In April 2009...

  • 217 URLs were shortened
  • 2ze.us links were clicked 617 times

In February 2010...

  • 1,182 URLs were shortened
  • 2ze.us links were clicked 32,830 times

Not too shabby for a side project.

Python Tip: Conditional Expressions

I learned something new today about Python 2.5 and newer. I thought it was so nifty that I decided to write a blog article about it. Hopefully someone out there finds it as interestingly useful as I do.

One of the things I find myself doing quite often in my code (be it Python, PHP, Java, or what have you) is a simple conditional assignment like so:

if foo:
    bar = 'baz'
else:
    bar = 'qux'

In a lot of languages these days, you can use a ternary operator as follows to turn these four lines into a one-liner:

<?php
$bar = $foo ? 'baz' : 'qux';
?>

However, this ternary operator does not exist in Python. I have used various means in the past to accomplish the same task without using code like we saw in the first code block above, but they were all very hackish. Today I was happy to learn that there is an official way to do it in Python:

bar = ('baz' if foo else 'qux')

I'm not sure which one is more confusing to newbies: Python's conditional expressions or other languages' ternary operator. Personally, I prefer constructs like these to make my code more concise. I my mind, they also make the code more readable. I have heard some folks argue that ternary operators and the like obfuscate the code more than necessary, so they discourage the use of such tactics and recommend using the classic approach featured in the first example.

For those who are interested, I learned about it and a few other neat things in Python 2.5 and newer at http://www.python.org/doc/2.5/whatsnew/pep-308.html.

Giving OpenSUSE 11.1 An Honest Chance

I've decided that if I ever want to really understand Linux, I'll have to give as many distributions as possible a chance. In the past, I've tried to use OpenSUSE on my HP Pavilion dv8000 laptop, but it never seemed quite as robust or useful as many other distributions that I've tried on the same machine.

With the recent release of OpenSUSE 11.1, I downloaded the final 32-bit DVD ISO as I normally do for newly released distributions (even if I don't plan on using them--it's an addiction). I proceeded to install the GNOME version of it in a virtual machine to see what all the hubbub was about. Evaluating an operating system within a virtual machine is not the most effective way to do things, but everything seemed fairly solid. As such, and since I have always had difficulties keeping any RPM-based distro around for any length of time, I plan on using OpenSUSE 11.1 through March 2008 (perhaps longer if it grows on me). If it hoses my system, I will go back to something better. If it works, I will learn to use and appreciate it better.

The Installation

The first step when the installation program starts is to choose what language to use, after which you choose the type of installation you're going to be doing. Your choices are:

  • New Installation
  • Update
  • Repair Installed System

You also have the option of installing "Add-On Products" from another media. At this step, I chose to do a new installation.

Next, you get to choose your time zone. The interface is very intuitive. You get a map of the world, and you click on the region you want to zoom in on. Once you're zoomed in, you can select a city that is near you to specify your time zone. Alternatively, you can choose your region and time zone from a couple of drop down lists.

After setting your time zone, you get to choose which desktop environment you want to install. Your choices are:

  • GNOME 2.24.1
  • KDE 4.1.3
  • KDE 3.5.10
  • XFCE 4.4
  • Minimal X Window
  • Minimal Server Selection (Text Mode)

I will choose to install GNOME because it seems to be the desktop of the future, especially with the hideous beast that KDE has become in the 4.x series...

Now you get to play with the partitioning. Usually the installer's first guess is pretty good, but I've got a different arrangement for my partitions, so I'm going to customize things a bit.

The next step is to create a regular, unprivileged user account for your day-to-day computing needs. This screen is pretty self-explanatory if you've ever registered for an e-mail address or installed any other operating system.

One thing that seems to have been added to OpenSUSE 11.1 is the option to use your regular user password as the root password. This is probably a nice addition for a lot of people, but I'd rather feel like my computer is a little more secure by having a different password for administrative tasks.

You're also give a few other options, such as being able to receive system mail, logging in automatically, and modifying your authentication settings. Other than the administrative password option, I left everything the same. If you're like me, and choose to have a different administrative password, you will be prompted to enter the new password at the next step.

Finally, you're shown a summary of the installation tasks that will take place. I'm going to customize my software selection just a bit so I don't have to do it manually after the installation is complete. For example, while I do like GNOME to a degree, I prefer to use KDE 3.5.x, so I will choose to install that environment as well just in case I need the comfort of KDE programs. Also, since I like to use the command line interface for a lot of things, I will choose to install the "Console Tools" package, just because it sounds useful. Lastly, I will choose to install a few development packages, such as C/C++, Java, Python, and Tcl/Tk. These changes bumped up my installation size from about 2.8GB to just over 4GB.

After reviewing the remaining tasks, all you need to do is hit the "Install" button. You will be prompted to verify your desire to install OpenSUSE, after which the package installation will begin. While the installation is taking place, you have the option of watching a brain-washing slideshow, viewing the installation details as it progresses, or reading the release notes.

The actual installation took nearly 40 minutes on my laptop. While this isn't necessarily a great improvement over past releases, I'm sure the story would have been much different had I not customized the software I wanted to have installed. The introduction of installation images a few releases ago drastically improved installation times. If you don't customize your package selection, you'll probably notice the speed difference.

When all of the packages have been installed, the installation program begins to configure your newly installed OpenSUSE for your computer, with a "reboot" in between. This is when all of your hardware, such as your network adapters, graphics adapter, sound card, printers, etc are probed and configured. Strangely enough, this step seems to take a lot longer than it does in Windows, which is usually not the case with Linux. What is OpenSUSE up to I wonder?

When all is said and done, the installation program finishes on its own and loads up your desktop.

Annoyances

There are a couple things that really annoyed me right off the bat about OpenSUSE 11.1. The first was that the loading screen and installation program didn't use my laptop's native resolution. My screen is capable of 1680x1050. The installation program chopped off about 1.25 inches of screen real estate on either side of the program. I don't know if this was an intentional occurrence or not. It seems like the artwork in the installation may have been limited to a non-widescreen resolution. If so, that's completely retarded. I'd like to think that more computer users these days have a widescreen monitor than not, at least the ones who would be playing with Linux.

The second annoyance was that the installation program wouldn't use my external USB DVD drive, which I like to think more reliable than my internal DVD drive. I mean, everything would start up fine--I got the boot menu, the installation program loaded fine, and things seemed like they would work. That's up until the package repositories (the DVD) were being built. Then the USB drive just kept spinning and spinning. Once I popped the disc into my internal drive the program proceeded as expected.

Your Desktop

I thought it was interesting that I chose to install GNOME, but since I chose to install KDE 3.5.10 alongside it that's what it booted me into after the installation was completed. No real complaints, though, since I prefer KDE anyway. Nonetheless, I switched back to GNOME to stretch my limits all the more. At least the desktop took up the full resolution that my screen can handle, unlike the installation program and boot screen.

Things seem fairly responsive... nothing like Slackware though. I just received a little popup notification with an excuse for the lag I might be experiencing: the daily indexing has commenced and should be finished soon. Whatever it's up to, it's taking up a consistent 100% of my CPU. How nice. I hope whatever it's indexing ends up being useful.

Sound worked right from the get-go, which is nice. Hardware acceleration for my Radeon Xpress 200M doesn't work, nor does my Broadcom wireless card. These will be fixed soon.

The Wireless

It looks like the most important step in getting my wireless to work was executing these commands as root:

/usr/sbin/install_bcm43xx_firmware
modprobe b43

I did a lot of stuff to try to get my wireless to work before I executed those commands, but nothing did the trick until I tried them. Also, to make the wireless available each time you reboot without requiring the modprobe b43 command, you need to edit your sysconfig.

To do that, open up YaST and find the "/etc/sysconfig Editor" option. Expand the "System" node, and navigate to Kernel > MODULES_LOADED_ON_BOOT. Then put b43 in the value box. Apply the changes. The next time you reboot your computer, the wireless should be available from the get-go.

The Video Card

This section only really applies to folks with ATI graphics adapters.

I found a tutorial on ubuntuforums.org, strangely enough, which described the process for getting ATI drivers to work on OpenSUSE 11.1. The first step is to download the official ATI drivers for Linux. Each of these commands should be executed as root:

wget https://a248.e.akamai.net/f/674/9206/0/www2.ati.com/drivers/\
linux/ati-driver-installer-8-12-x86.x86_64.run

Next, you need to download the kernel source and ensure that you have a few other utilities required for compiling a kernel module:

zypper in kernel-source gcc make patch

Now you should be able to run through the ATI driver installation utility, accepting all of the defaults:

sh ati-driver-installer-8-12-x86.x86_64.run

If you're on 64-bit OpenSUSE, you need to take an extra step to make the driver available:

rm /usr/lib/dri/fglrx_dri.so && ln -s /usr/lib64/dri/fglrx_dri.so \
/usr/lib/dri/fglrx_dri.so

Backup your existing xorg.conf configuration file and configure Xorg to use the new driver:

cp /etc/X11/xorg.conf /etc/X11/xorg.conf.orig
aticonfig --initial -f

Finally, configure Sax2 with the ATI driver:

sax2 -r -m 0=fglrx

Upon rebooting your computer, you should be able to use the hardware-accelerated 3D capabilities of your ATI card. To verify that things are up and running, execute fglrxinfo as a normal user. This command renders the following output on my system:

display: :0.0  screen: 0
OpenGL vendor string: ATI Technologies Inc.
OpenGL renderer string: ATI Radeon Xpress Series
OpenGL version string: 2.1.8304 Release

Other Thoughts

After having played with OpenSUSE 11.1 for a couple hours, I think I might be able to keep it around for a little while. Despite the lack of speed exhibited by other Linux distributions, the "stability" that OpenSUSE seems to offer is attractive to me. It will likely take some time to get used to RPMs over DEBs for package management.

How bad can it be? I mean, it comes with OpenOffice 3.0.0, which is nice. It can handle dual-head mode on my laptop thanks to Xinerama, which no other distro to date has been able to do. This gives me a little more screen real estate to work with, which helps out a lot when I'm developing a Web site or working in an IDE. The package managers are slow, but how often do you really install software anyway?

Again, we'll just have to see how things pan out. Let's hope it turns out to be a positive experience.

Slackware 12.2, Gmail Tasks and SMS

I just had to come up with a blog post about these newly released bits of awesomeness!!

  • Slackware 12.2 was announced this morning
  • Google has launched two new Labs features for Gmail: a simple task list and the ability to send SMS messages within Gmail itself. w00t

I wish I had more time to report other important releases, but I must study!!!

In PHP's Defense

So the other day I wrote up an article that sarcastically compared PHP and Python syntax a little. While I am completely serious when I say that I prefer Python's syntax a heck of a lot more than that of PHP, I thought it might be a good thing for me to demonstrate that the code I posted before could have been more appealing had it been thought out a little more. After a solid year of not really dealing with anything besides Python, I will share a feeble attempt at cleaning up/optimizing the PHP code.

Let's start with this snippet:

<?php
$this->result=(isset($tmp["UMstatus"])?$tmp["UMstatus"]:"Error");
$this->resultcode=(isset($tmp["UMresult"])?$tmp["UMresult"]:"E");
$this->authcode=(isset($tmp["UMauthCode"])?$tmp["UMauthCode"]:"");
$this->refnum=(isset($tmp["UMrefNum"])?$tmp["UMrefNum"]:"");
$this->batch=(isset($tmp["UMbatch"])?$tmp["UMbatch"]:"");
$this->avs_result=(isset($tmp["UMavsResult"])?$tmp["UMavsResult"]:"");
$this->avs_result_code=(isset($tmp["UMavsResultCode"])?$tmp["UMavsResultCode"]:"");
$this->cvv2_result=(isset($tmp["UMcvv2Result"])?$tmp["UMcvv2Result"]:"");
$this->cvv2_result_code=(isset($tmp["UMcvv2ResultCode"])?$tmp["UMcvv2ResultCode"]:"");
$this->vpas_result_code=(isset($tmp["UMvpasResultCode"])?$tmp["UMvpasResultCode"]:"");
$this->convertedamount=(isset($tmp["UMconvertedAmount"])?$tmp["UMconvertedAmount"]:"");
$this->convertedamountcurrency=(isset($tmp["UMconvertedAmountCurrency"])?$tmp["UMconvertedAmountCurrency"]:"");
$this->conversionrate=(isset($tmp["UMconversionRate"])?$tmp["UMconversionRate"]:"");
$this->error=(isset($tmp["UMerror"])?$tmp["UMerror"]:"");
$this->errorcode=(isset($tmp["UMerrorcode"])?$tmp["UMerrorcode"]:"10132");
$this->custnum=(isset($tmp["UMcustnum"])?$tmp["UMcustnum"]:"");

$this->avs=(isset($tmp["UMavsResult"])?$tmp["UMavsResult"]:"");
$this->cvv2=(isset($tmp["UMcvv2Result"])?$tmp["UMcvv2Result"]:"");
?>

We see a lot of duplicate code in this chunk of code. The only thing that really changes much are the variable names and associative array keys. If we had defined a function that looked something like this...

1
2
3
4
5
<?php
function assign($member, $arr, $key, $default='') {
    $this->$member = isset($arr[$key]) ? $arr[$key] : $default;
}
?>

...things might just look a bit better. Let's see what the snippet might look like with this function defined in the same class:

<?php
$this->assign("result", $tmp, "UMstatus", "Error");
$this->assign("resultcode", $tmp, "UMresult", "E");
$this->assign("authcode", $tmp, "UMauthCode");
$this->assign("refnum", $tmp, "UMrefNum");
$this->assign("batch", $tmp, "UMbatch");
$this->assign("avs_result", $tmp, "UMavsResult");
$this->assign("avs_result_code", $tmp, "UMavsResultCode");
$this->assign("cvv2_result", $tmp, "UMcvv2Result");
$this->assign("cvv2_result_code", $tmp, "UMcvv2ResultCode");
$this->assign("vpas_result_code", $tmp, "UMvpasResultCode");
$this->assign("convertedamount", $tmp, "UMconvertedAmount");
$this->assign("convertedamountcurrency", $tmp, "UMconvertedAmountCurrency");
$this->assign("conversionrate", $tmp, "UMconversionRate");
$this->assign("error", $tmp, "UMerror");
$this->assign("errorcode", $tmp, "UMerrorcode", "10132");
$this->assign("custnum", $tmp, "UMcustnum");

$this->assign("avs", $tmp, "UMavsResult");
$this->assign("cvv2", $tmp, "UMcvv2Result");
?>

In my opinion, this still isn't as appealing as the Python solution, but I'd take it over the original code. It's a lot easier to read. This may or may not be the best solution on any level of scrutiny--feel free to comment with any suggestions for ways to further improve things.

The second snippet from my original post could use a lot more help than the first one. I don't know who these guys are who wrote the PHP USA ePay module, but I think they could use a little assistance. No offense if you're reading this article--just some friendly constructive criticism. I would expect no less from anyone else who was examining my code and found ways to improve its efficiency.

Here's the original:

<?php
switch(substr($ccnum,0,1))
{
    case 2: //enRoute - First four digits must be 2014 or 2149. Only valid length is 15 digits
        if((substr($ccnum,0,4) == "2014" || substr($ccnum,0,4) == "2149") && strlen($ccnum) == 15) return 20;
        break;
    case 3: //JCB - Um yuck, read the if statement below, and oh by the way 300 through 309 overlaps with diners club.  bummer.
        if((substr($ccnum,0,4) == "3088" || substr($ccnum,0,4) == "3096" || substr($ccnum,0,4) == "3112" || substr($ccnum,0,4) == "3158" || substr($ccnum,0,4) == "3337" ||
            (substr($ccnum,0,8) >= "35280000" ||substr($ccnum,0,8) <= "358999999")) && strlen($ccnum)==16)
        {
            return 28;
        } else {
            switch(substr($ccnum,1,1))
            {
                case 4:
                case 7: // American Express - First digit must be 3 and second digit 4 or 7. Only Valid length is 15
                    if(strlen($ccnum) == 15) return 3;
                    break;
                    case 0:
                case 6:
                case 8: //Diners Club/Carte Blanche - First digit must be 3 and second digit 0, 6 or 8. Only valid length is 14
                    if(strlen($ccnum) == 14) return 4;
                    break;
            }
        }
        break;
    case 4: // Visa - First digit must be a 4 and length must be either 13 or 16 digits.
        if(strlen($ccnum) == 13 || strlen($ccnum) == 16)
        {
            return 2;
        }
        break;

    case 5: // Mastercard - First digit must be a 5 and second digit must be int the range 1 to 5 inclusive. Only valid length is 16
        if((substr($ccnum,1,1) >=1 && substr($ccnum,1,1) <=5) && strlen($ccnum) == 16)
        {
            return 1;
        }
        break;
case 6: // Discover - First four digits must be 6011. Only valid length is 16 digits.
        if(substr($ccnum,0,4) == "6011" && strlen($ccnum) == 16) return 10;
}
?>

The first, and most obvious, improvement I would make to this code is to cram the substr($ccnum,0,4) junk into its own variable. It's used 8 different times up there. While substring operations might not be the most costly of functions out there, there's no need to repeatedly call the same function to get the same value that many times in the same block of code.

Similar to how I wrote the Python version, I would also throw the things that are repeatedly compared to the substr($ccnum,0,4) into an array and use the in_array function to increase readability. Oh, and consistent indentation (and not just because I like Python--it's good style to align things).

<?php
$four = substr($ccnum, 0, 4);
switch (substr($ccnum, 0, 1)) {
    case 2:
        /* enRoute - First four digits must be 2014 or 2149. Only valid
           length is 15 digits */
        if (in_array($four, array("2014", "2149")) && strlen($ccnum) == 15) return 20;
        break;
    case 3:
        /* JCB - Um yuck, read the if statement below, and oh by the way
           300 through 309 overlaps with diners club.  bummer. */
        if (in_array($four, array("3088", "3096", "3112", "3158", "3337")) ||
            in_array(substr($ccnum, 0, 8), array("35280000", "358999999")) &&
            strlen($ccnum) == 16) {
            return 28;
        } else {
            switch (substr($ccnum, 1, 1)) {
                case 4:
                case 7:
                    /* American Express - First digit must be 3 and second
                       digit 4 or 7. Only Valid length is 15 */
                    if(strlen($ccnum) == 15) return 3;
                    break;
                case 0:
                case 6:
                case 8:
                    /* Diners Club/Carte Blanche - First digit must be 3
                       and second digit 0, 6 or 8. Only valid length is 14 */
                    if(strlen($ccnum) == 14) return 4;
                    break;
            }
        }
        break;
    case 4:
        /* Visa - First digit must be a 4 and length must be either 13 or
           16 digits. */
        if (strlen($ccnum) == 13 || strlen($ccnum) == 16) {
            return 2;
        }
        break;
    case 5:
        /* Mastercard - First digit must be a 5 and second digit must be
           int the range 1 to 5 inclusive. Only valid length is 16 */
        if ($ccnum[1] >= 1 && $ccnum[1] <= 5 && strlen($ccnum) == 16) {
            return 1;
        }
        break;
    case 6:
        /* Discover - First four digits must be 6011. Only valid length
           is 16 digits. */
        if ($four == "6011" && strlen($ccnum) == 16) return 10;
}
?>

That just feels better to me. It should work exactly the same as the original snippet (though I admit I haven't tested it--don't even have PHP installed these days), but it just looks a heck of a lot better to me. Again, it might not be the most efficient way of accomplishing the desired task, but I consider these minor changes to make all the difference when you're required to maintain the code you wrote :)

You might notice that my version of the PHP is 10 lines longer than the original. That's mostly due to the fact that I try to respect the 80-character margin by wrapping lines before reaching that point. I believe this also adds to the pleasing appearance, but I realize that's more of a subjective thing these days.

Flame away folks!

Beautiful, Beautiful Python

Today I was working on a USA ePay payment module for Satchmo. I had access to some work done by another chap, but I wanted to get a better feel for what needed to be happening in the payment module, so I examined some of the stuff that other people had done as well. I studied the "official" PHP version of the payment module for a little while. Things seemed pretty self-explanatory, so I decided I might as well translate the PHP library into Python.

Most of the translation process was quite mundane... removing dollar signs here and there, getting rid of the dirty -> junk, etc. However, toward the end of the translation process, I started to actually enjoy myself. That's because I wasn't just defining variables left and right--I actually started doing some stuff for processing. Here's some of the PHP code:

<?php
$this->result=(isset($tmp["UMstatus"])?$tmp["UMstatus"]:"Error");
$this->resultcode=(isset($tmp["UMresult"])?$tmp["UMresult"]:"E");
$this->authcode=(isset($tmp["UMauthCode"])?$tmp["UMauthCode"]:"");
$this->refnum=(isset($tmp["UMrefNum"])?$tmp["UMrefNum"]:"");
$this->batch=(isset($tmp["UMbatch"])?$tmp["UMbatch"]:"");
$this->avs_result=(isset($tmp["UMavsResult"])?$tmp["UMavsResult"]:"");
$this->avs_result_code=(isset($tmp["UMavsResultCode"])?$tmp["UMavsResultCode"]:"");
$this->cvv2_result=(isset($tmp["UMcvv2Result"])?$tmp["UMcvv2Result"]:"");
$this->cvv2_result_code=(isset($tmp["UMcvv2ResultCode"])?$tmp["UMcvv2ResultCode"]:"");
$this->vpas_result_code=(isset($tmp["UMvpasResultCode"])?$tmp["UMvpasResultCode"]:"");
$this->convertedamount=(isset($tmp["UMconvertedAmount"])?$tmp["UMconvertedAmount"]:"");
$this->convertedamountcurrency=(isset($tmp["UMconvertedAmountCurrency"])?$tmp["UMconvertedAmountCurrency"]:"");
$this->conversionrate=(isset($tmp["UMconversionRate"])?$tmp["UMconversionRate"]:"");
$this->error=(isset($tmp["UMerror"])?$tmp["UMerror"]:"");
$this->errorcode=(isset($tmp["UMerrorcode"])?$tmp["UMerrorcode"]:"10132");
$this->custnum=(isset($tmp["UMcustnum"])?$tmp["UMcustnum"]:"");

$this->avs=(isset($tmp["UMavsResult"])?$tmp["UMavsResult"]:"");
$this->cvv2=(isset($tmp["UMcvv2Result"])?$tmp["UMcvv2Result"]:"");
?>

Seems fairly self-explanatory, right? Heh. Now let's look at the Python version of this:

self.result = res.get('UMstatus', 'Error')
self.resultcode = res.get('UMresult', 'E')
self.authcode = res.get('UMauthCode', '')
self.refnum = res.get('UMrefNum', '')
self.batch = res.get('UMbatch', '')
self.avs_result = res.get('UMavsResult', '')
self.avs_result_code = res.get('UMavsResultCode', '')
self.cvv2_result = res.get('UMcvv2Result', '')
self.cvv2_result_code = res.get('UMcvv2ResultCode', '')
self.vpas_result_code = res.get('UMvpasResultCode', '')
self.convertedamount = res.get('UMconvertedAmount', '')
self.convertedamountcurrency = res.get('UMconvertedAmountCurrency', '')
self.conversionrate = res.get('UMconversionRate', '')
self.error = res.get('UMerror', '')
self.errorcode = res.get('UMerrorcode', '10132')
self.custnum = res.get('UMcustnum', '')

self.avs = res.get('UMavsResult', '')
self.cvv2 = res.get('UMcvv2Result', '')

<sarcasm>Wow, yeah... PHP really does rock! It must be the coolness factor behind the ternary operator they use! The Python code is just horrible compared to the PHP code.

How about the part where you check to see if a credit card number fits the profile of a known credit card? The PHP:

<?php
switch(substr($ccnum,0,1))
{
    case 2: //enRoute - First four digits must be 2014 or 2149. Only valid length is 15 digits
        if((substr($ccnum,0,4) == "2014" || substr($ccnum,0,4) == "2149") && strlen($ccnum) == 15) return 20;
        break;
    case 3: //JCB - Um yuck, read the if statement below, and oh by the way 300 through 309 overlaps with diners club.  bummer.
        if((substr($ccnum,0,4) == "3088" || substr($ccnum,0,4) == "3096" || substr($ccnum,0,4) == "3112" || substr($ccnum,0,4) == "3158" || substr($ccnum,0,4) == "3337" ||
            (substr($ccnum,0,8) >= "35280000" ||substr($ccnum,0,8) <= "358999999")) && strlen($ccnum)==16)
        {
            return 28;
        } else {
            switch(substr($ccnum,1,1))
            {
                case 4:
                case 7: // American Express - First digit must be 3 and second digit 4 or 7. Only Valid length is 15
                    if(strlen($ccnum) == 15) return 3;
                    break;
                    case 0:
                case 6:
                case 8: //Diners Club/Carte Blanche - First digit must be 3 and second digit 0, 6 or 8. Only valid length is 14
                    if(strlen($ccnum) == 14) return 4;
                    break;
            }
        }
        break;
    case 4: // Visa - First digit must be a 4 and length must be either 13 or 16 digits.
        if(strlen($ccnum) == 13 || strlen($ccnum) == 16)
        {
             return 2;
        }
        break;

    case 5: // Mastercard - First digit must be a 5 and second digit must be int the range 1 to 5 inclusive. Only valid length is 16
        if((substr($ccnum,1,1) >=1 && substr($ccnum,1,1) <=5) && strlen($ccnum) == 16)
        {
            return 1;
        }
        break;
case 6: // Discover - First four digits must be 6011. Only valid length is 16 digits.
        if(substr($ccnum,0,4) == "6011" && strlen($ccnum) == 16) return 10;
}
?>

Very eloquent. Let's see how bad the Python looks on this one:

first = ccnum[0]
four = ccnum[0:4]
if first == '2' and len(ccnum) == 15 and four in ['2014', '2149']:
    # enRoute: first four digits must be 2014 or 2149. Only valid length
    # is 15 digits
    return 20
elif first == '3':
    # JCB
    if len(ccnum) == 16 and four in ['3088', '3096', '3112', '3158', '3337'] \
        or ccnum[0:8] in ['35280000', '35899999']:
        return 28
    else:
        if len(ccnum) == 15 and ccnum[1] in ['4', '7']:
            # American Express: first digit must be 3 and second must be
            # 4 or 7.  Only valid length is 15 digits
            return 3
        elif len(ccnum) == 14 and ccnum[1] in ['0', '6', '8']:
            # Diners Club/Carte Blanche: first digit must be 3 and second
            # digit must be 0, 6, or 8.  Only valid length is 14
            return 4
elif first == '4' and len(ccnum) in [13, 16]:
    # Visa: first digit must be 4 and length must be either 13 or 16
    return 2
elif first == '5' and len(ccnum) == 16:
    # Mastercard: first digit must be a 5 and second must be in the range
    # 1 to 5 inclusive.  Only valid length is 16
    if int(ccnum[1]) in range(1, 6):
        return 1
elif first == '6' and len(ccnum) == 16 and four == '6011':
    # Discover: first four digits must be 6011.  Only valid length is 16
    return 10

Eek gads!!! It's so obvious why PHP is the language of choice for so many people out there. Python doesn't even have a switch statement, for crying out loud! Inconceivable!</sarcasm>

I'm so glad I was able to escape the grips of PHP.

Andy Rutledge: Don't Walk; Run

This morning I've been doing some research for work, and I stumbled upon an article that is very helpful for freelancers like me (relatively new to the freelance arena) and people looking for freelancers/design agencies.

Given the recent freelance opportunities that I have been involved with, I found this article to be particularly beneficial. There are many pieces of advice that, taken into consideration, really do distinguish the good freelancers/design agencies from the bad ones.

If the agency has any clue at all, your company will basically have to "interview for the job" in order to be accepted as a client. Agencies that know what they're doing work hard to take on only those projects and those clients that are a good fit for their skill set, even their personality. They'll want to get to know you a bit and they'll want to learn much about your project, its scope, and your specific expectations before they agree to work with you. If you find that they immediately suggest you work with them and start talking about sending you a contract, you're probably talking to the wrong agency. Hang up the phone.

I know that I am guilty of such behavior. In the past, I have quickly jumped at any opportunity to make some extra cash as a freelancer. It usually doesn't take a long time before I begin regretting my hasty acceptance of a job. This is extremely sound advice.

If anyone other than the designer who did the work presents the designs to you, something's amiss and there is likely more yet to go astray. It is an unfortunate habit among some agencies to ever get between the client and the designer. This is a terrible mistake and a clear sign that the agency, large or small, is not well run. As a result, the work produced by the agency is likely to be flawed for the voids in understanding and effectiveness that result. In short, if your project's design phase does not begin and end with direct contact between yourself and the designer, you're getting short shrift. Find another agency.

I'd like to twist this one around a little. I completely agree with what Andy has said here, but I have also been on the receiving end of this particular type of situation. On one recent project, I was receiving instructions from a person who apparently didn't have the same goals as the "real" client. The end result was a product the "real" client didn't really want. Working directly with the client/agency is critical to a successful project.

There are several other important points in Andy's article, and I wholeheartedly recommend that you read it if you find yourself in such a situation. It will only help. In conclusion, I just want to share another quote from Andy's article:

And if you do notice too many of these telltale signs of incompetence in your current agency — don't walk; run.

PyScriptures 0.2a Is Here!

So, for those of you who aren't in my immediate vicinity or who aren't on Google Talk all the time, this might well be your first exposure to my latest project. I'm calling it PyScriptures. Py because I wrote it all in Python. Scriptures because it is a program that provides the entire LDS standard works (as far as the actual scriptures are concerned, anyway).

Some History

(feel free to skip to the good stuff if you don't care about history, or skip to the downloads)

I have been working on this sort of program for a very long time. My first attempt was way back in probably 2001, using PHP. I wanted to have a way to easily read the scriptures on my computer without requiring an Internet connection. To go along with this, I wanted to be able to highlight text, mark verses, and easily navigate the scriptures. Obviously, I never quite got it right--that's why I'm still working on new versions all the time.

After a while, I learned that I would be getting a nifty Sharp Zaurus SL-5500 as a graduation present. It's a Linux-based PDA (one of the first, actually), and it's still pretty powerful considering that it's 6 years old now. Anyway, once I got that little gadget, I wanted to get the scriptures on there, too. I didn't have any of the MarkMyScriptures software that other PDA users enjoyed, because of the different operating system. Not to mention how cheap I am (I hate buying software). I ended up porting my PHP/MySQL version of the scriptures to my PDA and using it that way for a little while, but that proved to be very inefficient. The project went on hold for a while, during my time as a missionary in Romania.

When I returned home from my mission, I picked up the scriptures project again. I think my next stab was a Swing-based Java application. It worked well enough, but it never really got too far beyond, "Oh look! The scriptures!"

It was also during the time I was working on the Java version that I realized that the database I was relying upon for my scriptures was incomplete. I'm not sure what the extent of the missing information was, but I remember specifically looking up Doctrine & Covenants 77 only to find questions with no answers. The database was also not very "normalized" but that's more of a nerdy topic, so I will spare you the details. I attempted to contact the bloke in responsible for maintaining that database to let him know of the problems, but it seems like he died or something. Absolutely no response from him, and no activity on his website for two years.

After discovering the lack of complete scripture in that database, I made a promise to myself that I would make my own version of the database so I wouldn't have to stumble upon more incomplete or inaccurate scriptures. This became a reality early in May, as I wrote a program (in Python) that actually downloaded (I call it "harvesting") all of the scriptures directly from the Church's website. It took quite a bit of time to perfect, but as far as I can tell, it works great now. It puts all of the scriptures in a nice, normalized database. So far I know it works with SQLite and MySQL, but it should work just dandy with others as well.

Once I had that fresh database, I began working on a graphical interface for the scriptures. I had been tinkering with something called wxPython for a little while, but I'd never really built anything useful with it. I could never get used to laying things out after using the amazing GUI builder in NetBeans.

This past weekend I've been hacking nearly non-stop to get a nice, functional interface for my scripture program. I'm very satisfied with it, and I have to admit that it performs far better than any previous iteration of this project. There's still a lot to be done to make it work the way I want it to, but here's a brief list of features in this version 0.2a release:

Features Include:

  1. Cross-Platform Compatible: This program works exactly the same on Windows, Linux, and Mac. I've tested it on Windows XP, Vista, Ubuntu Linux, Slackware Linux, and MacOS X (leopard) and have only found minor differences that don't really matter anyway. The program itself does work though.
  2. Fast: Python does a good job at working quickly, even with my crummy code. It boasts incredible speed when retrieving and rendering the entire canon of scripture.
  3. Simple searching: You can type in a word, part of a word, or a whole phrase, and it will find any and all matches (case-insensitively) in the entire standard works.
  4. Quick Jump: Know the exact reference to the scripture you want? Type it in and you're immediately taken to that verse. I never understood why other programs don't have this feature. My implementation is not perfect, but it sure as heck didn't take much to get it where it is.
  5. Adjustable font sizes: You can easily adjust the size of the scripture text (within reasonable limits). That way you can make it easier to read if you're not sitting right in front of your computer.
  6. Easy navigation: You can quickly and easily jump to the next or previous chapter or book. I realize that this might not be very useful to a lot of people, but I love this sort of functionality.
  7. Random verse: Click one button to jump to some random verse anywhere in the scriptures. This is mostly a database deal, and it seems to prefer the Old Testament in my experience. Maybe that's just because the Old Testament probably has more verses than the rest of the volumes put together?
  8. Good memory: Prefer to have your window maximized? Don't like seeing the toolbar? The program will remember things like that, as well as the size and position of the window on your screen (if it's not maximized) and what verse you had selected immediately before closing down the program.
  9. Keyboard shortcuts: For those of us who hate to use mice, there are keyboard shortcuts to do most things in the program.

There's still more fun stuff to come, but I had to get something out the door. I spent most of today just trying to get the program to behave well on other platforms (mostly Windows), because I develop on Linux. If you're interested in trying out what I have now, feel free to download whatever suits you best:

Downloads

Windows Installer (32-bit) (9.0MB)

Debian Linux (including Ubuntu) (2.9MB)

Launch pyscriptures after installing and it should work.

MacOS X (11.3MB)

Man... Gotta love the size differences.

Requirements

This program requires Python 2.4+, pysqlite2 (or sqlite3 if you have Python 2.5), and wxPython 2.8+. These may be different, but that's what I used to develop with, so I know it works with them. The Windows installer should include everything you need to get started, as should the Mac installer.

Note: The .dmg is very, very shabby right now. I plan on making it prettier as time goes on, but this _is_ an alpha release, after all. You can't expect too much.

I should stop here. Enjoy!