Arduino-Powered Webcam Mount    Posted:


Earlier this month, I completed yet another journey around the biggest star in our galaxy. Some of my beloved family members thought this would be a good occasion to send me some cash, and I also got a gift card for being plain awesome at work. Even though we really do need a bigger car and whatnot, my wife insisted that I only spend this money on myself and whatever I wanted.

Little did she know the can of worms she just opened up.

I took pretty much all of the money and blew it on stuff for my electronics projects. Up to this point, my projects have all been pretty boring simply because nothing ever moved--it was mostly just lights turning on and off or changing colors. Sure, that's fun, but things really start to get interesting when you actually interact with the physical world. With the birthday money, I was finally able to buy a bunch of servos to begin living out my childhood dream of building robots.

My first project since getting all of my new toys was a motorized webcam mount. My parents bought me a Logitech C910 for my birthday because they were tired of trying to see their grandchildren with the crappy webcam that is built into my laptop. It was a perfect opportunity to use SparkFun's tutorial for some facial tracking (thanks to OpenCV) using their Pan/Tilt Servo Bracket.

It took a little while to get everything setup properly, but SparkFun's tutorial explains perfectly how you can get everything setup if you want to repeat this project.

The problem I had with the SparkFun tutorial, though, is that it basically only gives you a standalone program that does the facial tracking and displays your webcam feed. What good is that? I actually wanted to use this rig to chat with people!! That's when I set out to figure out how to do this.

While the Processing sketch ran absolutely perfect on Windows, it didn't want to work on my Arch Linux system due to some missing dependencies that I didn't know how/care to satisfy. As such, I opted to rewrite the sketch using Python so I could do the facial tracking in Linux.

This is still a work in progress, but here's the current facial tracking program which tells the Arduino where the webcam should be pointing, along with the Arduino sketch.

Now that I could track a face and move my webcam in Linux, I still faced the same problem as before: how can I use my face-tracking, webcam-moving program during a chat with my mom? I had no idea how to accomplish this. I figured I would have to either intercept the webcam feed as it was going to Skype or the Google Talk Plugin, or I'd have to somehow consume the webcam feed and proxy it back out as a V4L2 device that the Google Talk Plugin could then use.

Trying to come up with a way of doing that seemed rather impossible (at least in straight Python), but I eventually stumbled upon a couple little gems.

So the GStreamer tutorial walks you step-by-step through different ways of using a gst-launch utility, and I found this information very useful. I learned that you can use tee to split a webcam feed and do two different things with it. I wondered if it would be possible to split one webcam feed and send it to two other V4L2 devices.

Enter v4l2loopback.

I was able to install this module from Arch's AUR, and using it was super easy (you should be root for this):

modprobe v4l2loopback devices=2

This created two new /dev/video* devices on my system, which happened to be /dev/video4 and /dev/video5 (yeah... been playing with a lot of webcams and whatnot). One device, video4, is for consumption by my face-tracking program. The other, video5, is for VLC, Skype, Google+ Hangouts, etc. After creating those devices, I simply ran the following command as a regular user:

gst-launch-0.10 v4l2src device=/dev/video1 ! \
    'video/x-raw-yuv,width=640,height=480,framerate=30/1' ! \
    tee name=t_vid ! queue ! \
    v4l2sink sync=false device=/dev/video4 t_vid. ! \
    queue ! videorate ! 'video/x-raw-yuv,framerate=30/1' ! \
    v4l2sink device=/dev/video5

There's a whole lot of stuff going on in that command that I honestly do not understand. All I know is that it made it so both my face-tracking Python program AND VLC can consume the same video feed via two different V4L2 devices! A co-worker of mine agreed to have a quick Google+ Hangout with me to test this setup under "real" circumstances (thx man). It worked :D Objective reached!

I had really hoped to find a way to handle this stuff inside Python, but I have to admit that this is a pretty slick setup. A lot of things are still hardcoded, but I do plan on making things a little more generic soon enough.

So here's my little rig (why yes, I did mount it on top of an old Kool-Aid powder thingy lid):

And a video of it in action. Please excuse the subject of the webcam video, I'm not sure where that guy came from or why he's playing with my webcam.

Comments

Simple Weighted Sort in Python    Posted:


Last night I found myself in need of a simple weighted sort function in Python. I had a list of integers which represented object IDs in my project. Some of the objects needed to be processed before the others while iterating over the list of integers, and I already knew which object IDs those were. The order the rest of the object IDs were processed didn't matter at all. I just wanted the special object IDs to arrive at the beginning of the list, and the remaining object IDs could be in any order.

I was surprised at how simple it was to produce such a weighted sort. Here's an example of what I did:

import random
object_ids = [random.randint(0, 100) for i in range(20)]
special_ids = [random.choice(object_ids) for i in range(5)]
print 'Object IDs:', object_ids
print 'Special IDs:', special_ids

object_ids.sort(key=special_ids.__contains__, reverse=True)
print 'Object IDs:', object_ids

And some sample output:

Object IDs: [13, 97, 67, 5, 77, 58, 24, 99, 29, 20, 29, 75, 100, 31, 79, 5, 27, 11, 6, 1]
Special IDs: [13, 1, 27, 6, 67]
Object IDs: [13, 67, 27, 6, 1, 97, 5, 77, 58, 24, 99, 29, 20, 29, 75, 100, 31, 79, 5, 11]

Notice that each of the "special" IDs have shifted from their original position in the object_ids list to be at the beginning of the list after the sort.

The Python documentation for sort says that the key argument "specifies a function of one argument that is used to extract a comparison key from each list element." I'm using it to check to see if a given element in the list is in my special_ids list. If the element is present in the special_ids list, it will be shifted to the left because of the way the special_ids.__contains__ works.

In sorting, a value of 1 (or other positive integer) out of a comparison function generally means "this belongs to the right of the other element." A value of -1 (or other negative integer) means "this belongs to the left of the other element." A value of 0 means "these two elements are equal" (for the purposes of sorting). I'm assuming it works similarly with the key argument. Please correct me if I'm wrong!

As lqc states in the comments below, the key argument works differently. It creates a new sequence of values which is then sorted. Before lqc jumped in, I was using key=int(i in special_ids) * -2 + 1 to do the sorting, which is pretty dumb. Using key=special_ids.__contains__ is much more appropriate. Thanks lqc!!

This sort of weighted sort might not be just right for your needs, but hopefully it will give you a place to start to build your customized weighted sort!

Comments

Django-Tracking 0.3.5    Posted:


I've finally gotten around to looking at a bunch of tickets that had been opened for django-tracking in the past year and a half or so. I feel horrible that it's really taken that long for me to get to them! Every time I got a ticket notification, I told myself, "Okay, I'll work on that this weekend." Many have weekends have passed without any work on any of my projects. I'm going to get better about that!

Anyway, several fixes have gone into the latest version of django-tracking. Some have to do with unicode problems (thanks ramusus!). Others have to do with overall performance, while yet others have to do with overall stability.

The first interesting change in this release is that django-tracking no longer relies on the GeoIP Python API. Instead it's now using django.contrib.gis.utils.GeoIP. I had hoped that this would remove the dependency on the GeoIP C API, but it appears that I was mistaken. Oh well.

Perhaps the biggest improvement in this new release is the use of caching. With caching in place, the middleware classes don't slam the database nearly as badly as they used to. There's still more that could be done with caching to improve performance, but I think what I've got now will be a big help.

Another noteworthy change, in my opinion, is the use of logging. I've sprinkled mildly useful logging messages throughout the code so you can learn when something bad happens that is silently handled. I hope that this will help me improve the quality of the code as it will allow anyone who uses the project (and pays attention to the log messages, of course) to tell me when bad things are happening.

Finally, the packaging code has been updated to be much more simple. Version 0.3.5 has been uploaded to PyPI and is available via pip or easy_install. If you prefer to have the latest copy of the code, the official code repositories are (in order of my personal preference):

I can't wait for your feedback!

Comments

Site-Wide Caching in Django    Posted:


My last article about caching RSS feeds in a Django project generated a lot of interest. My original goal was to help other people who have tried to cache QuerySet objects and received a funky error message. Many of my visitors offered helpful advice in the comments, making it clear that I was going about caching my feeds the wrong way.

I knew my solution was wrong before I even produced it, but I couldn't get Django's site-wide caching middleware to work in my production environment. Site-wide caching worked wonderfully in my development environment, and I tried all sorts of things to make it work in my production setup. It wasn't until one "Jacob" offered a beautiful pearl of wisdom that things started to make more sense:

This doesn't pertain to feeds, but one rather large gotcha with the cache middleware is that any javascript you are running that plants a cookie will affect the cache key. Google analytics, for instance, has that effect. A workaround is to use a middleware to strip out the offending cookies from the request object before the cache middleware looks at it.

The minute I read that comment, I realized just how logical it was! If Google Analytics, or any other JavaScript used on my site, was setting a cookie, and it changed that cookie on each request, then the caching engine would effectively have a different page to cache for each request! Thank you so much, Jacob, for helping me get past the frustration of not having site-wide caching in my production environment.

How To Setup Site-Wide Caching

While most of this can be gleaned from the official documentation, I will repeat it here in an effort to provide a complete "HOWTO". For further information, hit up the official caching documentation.

The first step is to choose a caching backend for your project. Built-in options include:

To specify which backend you want to use, define the CACHE_BACKEND variable in your settings.py. The definition for each backend is different, so check out the official documentation for details.

Next, install a couple of middleware classes, and pay attention to where the classes are supposed to appear in the list:

  • django.middleware.cache.UpdateCacheMiddleware - This should be the first middleware class in your MIDDLEWARE_CLASSES tuple in your settings.py.
  • django.middleware.cache.FetchFromCacheMiddleware - This should be the last middleware class in your MIDDLEWARE_CLASSES tuple in your settings.py.

Finally, you must define the following variables in your settings.py file:

  • CACHE_MIDDLEWARE_SECONDS - The number of seconds each page should be cached
  • CACHE_MIDDLEWARE_KEY_PREFIX - If the cache is shared across multiple sites using the same Django installation, set this to the name of the site, or some other string that is unique to this Django instance, to prevent key collisions. Use an empty string if you don't care

If you don't use anything like Google Analytics that sets/changes cookies on each request to your site, you should have site-wide caching enabled now. If you only want pages to be cached for users who are not logged in, you may add CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True to your settings.py file--its meaning should be fairly obvious.

If, however, your site-wide caching doesn't appear to work (as it didn't for me for a long time), you can create a special middleware class to strip those dirty cookies from the request, so the caching middleware can do its work.

import re

class StripCookieMiddleware(object):
    """Ganked from http://2ze.us/Io"""

    STRIP_RE = re.compile(r'\b(_[^=]+=.+?(?:; |$))')

    def process_request(self, request):
        cookie = self.STRIP_RE.sub('', request.META.get('HTTP_COOKIE', ''))
        request.META['HTTP_COOKIE'] = cookie

Edit: Thanks to Tal for regex the suggestion!

Once you do that, you need only install the new middleware class. Be sure to install it somewhere between the UpdateCacheMiddleware and FetchFromCacheMiddleware classes, not first or last in the tuple. When all of that is done, your site-wide caching should really work! That is, of course, unless your offending cookies are not found by that STRIP_RE regular expression.

Thanks again to Jacob and "nf", the original author of the middleware class I used to solve all of my problems! Also, I'd like to thank "JaredKuolt" for the django-staticgenerator on his github account. It made me happy for a while as I was working toward real site-wide caching.

Comments

Tip: easy_install / pip    Posted:


With all of the exciting updates to Mercurial recently, I've been on a rampage, updating various boxes everywhere I go. I'm in the habit of using easy_install and/or pip to install most of my Python-related packages. It's pretty easy to install packages that are in well-known locations (like PyPI or on Google Code, for example). It's also pretty easy to update packages using either utility. Both take a -U parameter, which, to my knowledge, tells it to actually check for updates and install the latest version.

That's all fine and dandy, but what happens when you want to install an "unofficial" version of some package? I mean, what if your favorite project all of the sudden includes some feature that you will die unless you can have access to it and the next official version is weeks or months in the future? There are typically a few avenues you can take to satisfy your needs, but I wanted to bring up something that I think not many people are aware of: easy_install and pip can both understand URLs to installable Python packages.

What do I mean by that, you ask? Well, when you get down to the basics of what both utilities do, they just take care of downloading some Python package and installing it with the setup.py file contained therein. In many cases, these utilities will search various package repositories, such as PyPI, to download whatever package you specify. If the package is found, it will be downloaded and extracted.

In most cases, you can do all of that yourself:

$ wget http://pypi.python.org/someproject/somepackage.tar.gz
$ tar zxf somepackage.tar.gz
$ cd somepackage
$ python setup.py install

Both easy_install and pip obviously do a lot of other magic, but that is perhaps the most basic way to understand what they do. To answer that last question, you can help your utility of choice out by specifying the exact URL to the specific package you want it to install for you:

$ easy_install http://pypi.python.org/someproject/somepackage.tar.gz
$ pip install http://pypi.python.org/someproject/somepackage.tar.gz

For me, this feature comes in very handy with projects that are hosted on BitBucket, for example, because you can always get any revision of the project in a tidy .tar.gz file. So when I'm updating Mercurial installations, I can do this to get the latest stable revision:

$ easy_install http://selenic.com/repo/hg-stable/archive/tip.tar.gz

It's pretty slick. Here's a full example:

[user@web ~]$ hg version
Mercurial Distributed SCM (version 1.2.1)

Copyright (C) 2005-2009 Matt Mackall <mpm@selenic.com> and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[user@web ~]$ easy_install http://selenic.com/repo/hg-stable/archive/tip.tar.gz
Downloading http://selenic.com/repo/hg-stable/archive/tip.tar.gz
Processing tip.tar.gz
Running Mercurial-stable-branch--8bce1e0d2801/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Gnk2c9/Mercurial-stable-branch--8bce1e0d2801/egg-dist-tmp--2VAce
zip_safe flag not set; analyzing archive contents...
mercurial.help: module references __file__
mercurial.templater: module references __file__
mercurial.extensions: module references __file__
mercurial.i18n: module references __file__
mercurial.lsprof: module references __file__
Removing mercurial unknown from easy-install.pth file
Adding mercurial 1.4.1-4-8bce1e0d2801 to easy-install.pth file
Installing hg script to /home/user/bin

Installed /home/user/lib/python2.5/mercurial-1.4.1_4_8bce1e0d2801-py2.5-linux-i686.egg
Processing dependencies for mercurial==1.4.1-4-8bce1e0d2801
Finished processing dependencies for mercurial==1.4.1-4-8bce1e0d2801
[user@web ~]$ hg version
Mercurial Distributed SCM (version 1.4.1+4-8bce1e0d2801)

Copyright (C) 2005-2009 Matt Mackall <mpm@selenic.com> and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Notice the version change from 1.2.1 to 1.4.1+4-8bce1e0d2801. w00t.

Edit: devov pointed out that pip is capable of installing packages directly from its repository. I've never used this functionality, but I'm interested in trying it out sometime! Thanks devov!

Comments

My Fedora 11 Adventures: Part III    Posted:


Alrighty folks. Good night's rest? Check. Need to get work done? Check. Today's adventure will be about getting my computer set up for the regular development tasks that I need to do every day for my work and hobbies.

Getting Work Done

The first thing I noticed this morning when I turned on my computer was that it took exactly 1 minute from the time I hit the power button to the time I hit the enter key to log into my computer. Logging in took an additional 15-20 seconds. That was quite nice.

The next thing I noticed was that I was not connected to my network as I should be. Clicking the system tray menu item as I did last night did the trick, but I'm going to have to investigate how to make it connect automatically at boot.

Automatic Network Connectivity

It looks like I can have my Ethernet be activated automatically by right clicking on the network manager icon in my system tray, selecting "Edit Connections," selecting "System eth0," clicking the "Edit" button, and finally checking the "Connect automatically" option in the subsequent window. We'll see if this truly activates my connection next time I boot.

In an effort to get my wireless working, I poked around a little more in the "Edit Connections" screen, but I didn't see anything that seemed useful. I did find something that seemed a bit more interesting by selecting Applications > Administration > Network Configuration from the KDE menu. This utility suggested that my wireless adapter was actually wlan1 instead of the wlan0 that the tray icon seemed to think it was.

I tweaked a few settings about my wireless adapter, such as marking the "Activate device when computer starts" and "Allow all users to enable and disable the device." In the Hardware Device tab, I selected my actual Broadcom wireless adapter instead of the non-existant wlan0. I also hit the probe button next to the "Bind to MAC address" box.

My network manager tray icon still shows no wireless networks (of which there is no shortage around here), and running iwlist scan as root says "Network is down" next to wlan1. I think I will just mess with it later. Maybe it will "just work" when I reboot next time.

Installing/Configuring The Tools

As I previously mentioned, I prefer to use things that work well without getting in my way. When talking about text editors, VIM is just fine for me, and VIM 7.2.148 is already installed on my Fedora 11. One less thing to install.

Next up comes the installation of all of the goods for Firefox. It turns out that Fedora comes with Firefox 3.5 Beta 4--a bold move. I hope my extensions all work! The extensions I will be installing right now include:

  • AdBlock Plus: get rid of pesky ads that slow down my computer
  • Firebug: an amazing tool when debugging Web pages
  • Web Developer: has some niceties that Firebug doesn't come with
  • Screengrab: fantastic for taking screenshots of full Web pages
  • 2Zeus: my own little extension that allows me to quickly get short URLs a la tinyurl.com and is.gd

When I plugged in my external 1TB Seagate hard drive, I got a delicious Fatal Error message:

/images/fedora/p3/fatal_error.png

All appears to be in order, however, as I have access to all of the partitions on the external drive.

Next I want to install Opera. It appears that the place to look is Applications > System > Software Management in the KDE menu. Let's see what we have. Searching for Opera in the only obvious search box sent my computer into a crazy "let me do something without telling you" cycle. I have no idea what's really going on, but my processor has been maxed out for the past 3 minutes and my network has been working a little here and there. Can it really be that difficult to find a simple package? Oh! It finished! It took 6 minutes and 54 seconds to find nothing. Excellent. Let me look somewhere else.

Awesome. My computer is non-responsive. The hard drive is still working, but my GUI is doing nothing. I love it. Attempts to drop back to a trusty console using Control, Alt, and F1-F6 rendered no results. I wonder if I can SSH in from here... I sure can! Fantastic. Let's see what's happening.

It appears that X is taking up 90% of my processing power, but my computer is still not responding to any of my input. Dang it! Now my SSH session isn't working. Looks like the only option I have now is to do a hard reset. Joy of joys. Thank you for this opportunity, Fedora. Last time I did a hard reset, I was in Windows and it trashed my 1TB external.

So far rebooting seems to be going well. I wonder if my network will be setup properly still... Fantastic! It works! Wireless is still not available though. I can live without that for the time being.

Back in the Software Management utility, searching for Opera again proved to work much more quickly, but I didn't get any results. I suppose I'll just go download it from their site. The download for Opera 10 beta 1 is a mere 7.2MB, and it looks like it will open in the same Software Management utility that I've been dinking around in.

When I downloaded the Opera package, I asked it to open directly in the default program, KPackageKit. That doesn't seem to be working in the least, so I am going to try to just save it to my home directory and install it some other way. Sorry guys and gals, I ended up just dropping back to a terminal to run rpm -Uvh opera-10.00-b1.gcc4-shared-qt3.x86_64.rpm and that seemed to work fine. Opera appeared in my KDE menu, and it runs well now.

Next up is Pidgin. Pidgin 2.5.5 is installed by default, and getting it up and running was as trivial as ever.

Now to test Flash... YouTube, here I come!! Beh, Flash is not installed by default, and it's also not in the Software Management tool. What use is that thing?! Maybe if I apply all of the updates in the "Software Updates" section it will feel more useful... Here it goes.

Cool. System is unresponsive again. Let's see if I can reboot from here. Nope! Thank you, Fedora, for making me hard reset my system more in 2 hours than I have had to in YEARS. Yeah, thanks buddy.

10:50 AM So the software updates continue to not work. It appears that a ypbind package is the culprit which is causing everything to hang... I disabled it and tried to install the software updates again.

10:53 AM GUI is non-responsive again. Yay.

10:56 AM Third hard reset in 3 hours. Maybe I will have to modify my original parameters and try GNOME to see if that makes the computer usable for more than an hour at a time.

11:00 AM That's it! I'm getting rid of KDE 4... sorry folks, GNOME is my only hope of getting work done. Second clean shutdown out of 5 since the installation completed last night.

Comments

Incompetent Secretaries    Posted:


Attention!

This is an opinionated rant. If you're offended, that's your problem. Feel free to leave comments about your hurt feelings.

Attention!

To the secretary/assistant: If you're reading this, I truly apologize for any offense you may take from my article. If you're new to the job, I'll apologize again--it's not always easy to hit the ground running. I'm just trying to understand the world a bit better than I did when I woke up this morning.

Several weeks ago, I received an email from the secretary/assistant of my Computer Information Technology department chairperson. This email contained a request for me, as a student graduating with a degree in CIT, to fill out a survey to gauge how well the program is functioning from my perspective.

Attached to said email was a Microsoft Excel spreadsheet, formatted for Excel 2007. As an avid Linux user and open-source advocate, I am appalled when I encounter simple files that are not formatted in what could be considered the "lowest common denominator" format, especially when they come from my CIT department. So many other formats would have worked just fine: .doc, .xls, .rtf, .txt, and even the infamous PDF with editable form fields. Come on folks...

Anyway, it's not so revolting to me when other departments send out such abominations. But when the files come from a department which teaches and encourages the use of Linux and whatnot, there seems to be a problem. Several full-time CIT professors even refuse to use Windows and instead choose to use a Macintosh. Yeah, sure, our university requires that students have Microsoft Office for a lot of classes, and they still provide computer labs with the software if we're too poor (or opinionated, like me) to buy it ourselves.

After getting over my own self-righteousness about file formats, I decided to download the spreadsheet anyway. I opened it up in my trusty OpenOffice and filled in the appropriate details. When it came time to save the file with my responses, I opted for OpenOffice's default spreadsheet format: ODS. In my mind, I figured that this individual, the secretary/assistant to the department chairperson, must have access to something that can at least read ODS files.

Well, I mailed off the survey last night. This morning I received the following response from the department chairperson himself:

I dont know what program you used but it is not working with my computer.If you could save it in another program and send it to me again as soon as possible that would be great. Thanks so much

There's something to be said about a CIT department chairperson, the official representative of the program, who is unaware of the ODS format. There's no excuse for that. I like the department chairperson, so I won't rag on him much. Instead, I'll rag on his little helper. When I returned home from class today, I found this email in my inbox, from the secretary/assistant:

I have an hp computer and my windows programs are 2007. If you could change something on the file that would make it so i can read it that would be very helpful.

Wow. Seriously? Apparently it makes a difference in this person's mind that their computer is an "hp" (so is mine). And the "windows programs are 2007," eh... What an ambiguous statement. I sure hope this individual is not a CIT major, minor, or in any other way affiliated with technology.

My question to you is this: how often do you encounter individuals who are blatantly incompetent to be fulfilling the duties with which they have been tasked? I mean, shouldn't there be some reasonable expectation that a person working in a position, such as this secretary/assistant, have some (even rudimentary) working knowledge of the industry in which they are employed? Or am I simply expecting too much from my peers? Am I asking too much of this poor individual who is the target of today's rant?

If I am not alone in my frustration, please speak up. Please also let me know if I am just being cruel and unusual.

Oh, and if you're wondering, I am going to send the survey back in a format that is more portable... I'm not that mean, despite what you might think after reading this article.

Comments

AES Encryption in Python Using PyCrypto    Posted:


I spent a little bit of time last night and this morning trying to find some examples for AES encryption using Python and PyCrypto. To my surprise, I had quite a difficult time finding an example of how to do it! I posted a message on Twitter asking for any solid examples, but people mostly just responded with things I had seen before--the libraries that do the encryption, not examples for how to use the libraries.

It wasn't long after that when I just decided to tackle the problem myself. My solution ended up being pretty simple (which is probably why there weren't any solid examples for me to find). However, out of respect for those out there who might still be looking for a solid example, here is my solution:

#!/usr/bin/env python

from Crypto.Cipher import AES
import base64
import os

# the block size for the cipher object; must be 16, 24, or 32 for AES
BLOCK_SIZE = 32

# the character used for padding--with a block cipher such as AES, the value
# you encrypt must be a multiple of BLOCK_SIZE in length.  This character is
# used to ensure that your value is always a multiple of BLOCK_SIZE
PADDING = '{'

# one-liner to sufficiently pad the text to be encrypted
pad = lambda s: s + (BLOCK_SIZE - len(s) % BLOCK_SIZE) * PADDING

# one-liners to encrypt/encode and decrypt/decode a string
# encrypt with AES, encode with base64
EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))
DecodeAES = lambda c, e: c.decrypt(base64.b64decode(e)).rstrip(PADDING)

# generate a random secret key
secret = os.urandom(BLOCK_SIZE)

# create a cipher object using the random secret
cipher = AES.new(secret)

# encode a string
encoded = EncodeAES(cipher, 'password')
print 'Encrypted string:', encoded

# decode the encoded string
decoded = DecodeAES(cipher, encoded)
print 'Decrypted string:', decoded

Edit: thanks to John and Kaso for their suggestions, though John's didn't seem to work for me (?)

If you plan to use this script, you'll need to have PyCrypto installed on your computer. I have had a difficult time finding this for Windows in the past, so I will mirror the installer that I found over here: http://jintoreedwine.wordpress.com/2008/07/20/python-25-and-encryption-pycrypto-under-windows/. I haven't tried it on Mac OS X yet, but it should be fairly simple to install it. Same goes for Linux.

The output of the script should always change with each execution thanks to the random secret key. Here's some sample output:

$ python aes_encryption.py
Encrypted string: aPCQ8v9WzLM/JusrJPS19K8uUA/34Xiu/ZR+arzl1oM=
Decrypted string: password

$ python aes_encryption.py
Encrypted string: F0cp4hMk8RXjcww270leHnigH++yqysIyPy8Em/qEbI=
Decrypted string: password

$ python aes_encryption.py
Encrypted string: 7gH2QCIPOxXVBjTXrMmdgU2l7Iku5Lch5jpG9OScGZw=
Decrypted string: password

$ python aes_encryption.py
Encrypted string: oJUq0/XHdmYgC3ILgFgF6Tpuo8ZhoEHN9wmnuYvV58Y=
Decrypted string: password

If the comments in the script aren't explanatory enough, please comment and ask for clarification. I will offer any that I am capable of, and I invite others to do the same.

Comments

Giving OpenSUSE 11.1 An Honest Chance    Posted:


I've decided that if I ever want to really understand Linux, I'll have to give as many distributions as possible a chance. In the past, I've tried to use OpenSUSE on my HP Pavilion dv8000 laptop, but it never seemed quite as robust or useful as many other distributions that I've tried on the same machine.

With the recent release of OpenSUSE 11.1, I downloaded the final 32-bit DVD ISO as I normally do for newly released distributions (even if I don't plan on using them--it's an addiction). I proceeded to install the GNOME version of it in a virtual machine to see what all the hubbub was about. Evaluating an operating system within a virtual machine is not the most effective way to do things, but everything seemed fairly solid. As such, and since I have always had difficulties keeping any RPM-based distro around for any length of time, I plan on using OpenSUSE 11.1 through March 2008 (perhaps longer if it grows on me). If it hoses my system, I will go back to something better. If it works, I will learn to use and appreciate it better.

The Installation

The first step when the installation program starts is to choose what language to use, after which you choose the type of installation you're going to be doing. Your choices are:

  • New Installation
  • Update
  • Repair Installed System

You also have the option of installing "Add-On Products" from another media. At this step, I chose to do a new installation.

Next, you get to choose your time zone. The interface is very intuitive. You get a map of the world, and you click on the region you want to zoom in on. Once you're zoomed in, you can select a city that is near you to specify your time zone. Alternatively, you can choose your region and time zone from a couple of drop down lists.

After setting your time zone, you get to choose which desktop environment you want to install. Your choices are:

  • GNOME 2.24.1
  • KDE 4.1.3
  • KDE 3.5.10
  • XFCE 4.4
  • Minimal X Window
  • Minimal Server Selection (Text Mode)

I will choose to install GNOME because it seems to be the desktop of the future, especially with the hideous beast that KDE has become in the 4.x series...

Now you get to play with the partitioning. Usually the installer's first guess is pretty good, but I've got a different arrangement for my partitions, so I'm going to customize things a bit.

The next step is to create a regular, unprivileged user account for your day-to-day computing needs. This screen is pretty self-explanatory if you've ever registered for an e-mail address or installed any other operating system.

One thing that seems to have been added to OpenSUSE 11.1 is the option to use your regular user password as the root password. This is probably a nice addition for a lot of people, but I'd rather feel like my computer is a little more secure by having a different password for administrative tasks.

You're also give a few other options, such as being able to receive system mail, logging in automatically, and modifying your authentication settings. Other than the administrative password option, I left everything the same. If you're like me, and choose to have a different administrative password, you will be prompted to enter the new password at the next step.

Finally, you're shown a summary of the installation tasks that will take place. I'm going to customize my software selection just a bit so I don't have to do it manually after the installation is complete. For example, while I do like GNOME to a degree, I prefer to use KDE 3.5.x, so I will choose to install that environment as well just in case I need the comfort of KDE programs. Also, since I like to use the command line interface for a lot of things, I will choose to install the "Console Tools" package, just because it sounds useful. Lastly, I will choose to install a few development packages, such as C/C++, Java, Python, and Tcl/Tk. These changes bumped up my installation size from about 2.8GB to just over 4GB.

After reviewing the remaining tasks, all you need to do is hit the "Install" button. You will be prompted to verify your desire to install OpenSUSE, after which the package installation will begin. While the installation is taking place, you have the option of watching a brain-washing slideshow, viewing the installation details as it progresses, or reading the release notes.

The actual installation took nearly 40 minutes on my laptop. While this isn't necessarily a great improvement over past releases, I'm sure the story would have been much different had I not customized the software I wanted to have installed. The introduction of installation images a few releases ago drastically improved installation times. If you don't customize your package selection, you'll probably notice the speed difference.

When all of the packages have been installed, the installation program begins to configure your newly installed OpenSUSE for your computer, with a "reboot" in between. This is when all of your hardware, such as your network adapters, graphics adapter, sound card, printers, etc are probed and configured. Strangely enough, this step seems to take a lot longer than it does in Windows, which is usually not the case with Linux. What is OpenSUSE up to I wonder?

When all is said and done, the installation program finishes on its own and loads up your desktop.

Annoyances

There are a couple things that really annoyed me right off the bat about OpenSUSE 11.1. The first was that the loading screen and installation program didn't use my laptop's native resolution. My screen is capable of 1680x1050. The installation program chopped off about 1.25 inches of screen real estate on either side of the program. I don't know if this was an intentional occurrence or not. It seems like the artwork in the installation may have been limited to a non-widescreen resolution. If so, that's completely retarded. I'd like to think that more computer users these days have a widescreen monitor than not, at least the ones who would be playing with Linux.

The second annoyance was that the installation program wouldn't use my external USB DVD drive, which I like to think more reliable than my internal DVD drive. I mean, everything would start up fine--I got the boot menu, the installation program loaded fine, and things seemed like they would work. That's up until the package repositories (the DVD) were being built. Then the USB drive just kept spinning and spinning. Once I popped the disc into my internal drive the program proceeded as expected.

Your Desktop

I thought it was interesting that I chose to install GNOME, but since I chose to install KDE 3.5.10 alongside it that's what it booted me into after the installation was completed. No real complaints, though, since I prefer KDE anyway. Nonetheless, I switched back to GNOME to stretch my limits all the more. At least the desktop took up the full resolution that my screen can handle, unlike the installation program and boot screen.

Things seem fairly responsive... nothing like Slackware though. I just received a little popup notification with an excuse for the lag I might be experiencing: the daily indexing has commenced and should be finished soon. Whatever it's up to, it's taking up a consistent 100% of my CPU. How nice. I hope whatever it's indexing ends up being useful.

Sound worked right from the get-go, which is nice. Hardware acceleration for my Radeon Xpress 200M doesn't work, nor does my Broadcom wireless card. These will be fixed soon.

The Wireless

It looks like the most important step in getting my wireless to work was executing these commands as root:

/usr/sbin/install_bcm43xx_firmware
modprobe b43

I did a lot of stuff to try to get my wireless to work before I executed those commands, but nothing did the trick until I tried them. Also, to make the wireless available each time you reboot without requiring the modprobe b43 command, you need to edit your sysconfig.

To do that, open up YaST and find the "/etc/sysconfig Editor" option. Expand the "System" node, and navigate to Kernel > MODULES_LOADED_ON_BOOT. Then put b43 in the value box. Apply the changes. The next time you reboot your computer, the wireless should be available from the get-go.

The Video Card

This section only really applies to folks with ATI graphics adapters.

I found a tutorial on ubuntuforums.org, strangely enough, which described the process for getting ATI drivers to work on OpenSUSE 11.1. The first step is to download the official ATI drivers for Linux. Each of these commands should be executed as root:

wget https://a248.e.akamai.net/f/674/9206/0/www2.ati.com/drivers/\
linux/ati-driver-installer-8-12-x86.x86_64.run

Next, you need to download the kernel source and ensure that you have a few other utilities required for compiling a kernel module:

zypper in kernel-source gcc make patch

Now you should be able to run through the ATI driver installation utility, accepting all of the defaults:

sh ati-driver-installer-8-12-x86.x86_64.run

If you're on 64-bit OpenSUSE, you need to take an extra step to make the driver available:

rm /usr/lib/dri/fglrx_dri.so && ln -s /usr/lib64/dri/fglrx_dri.so \
/usr/lib/dri/fglrx_dri.so

Backup your existing xorg.conf configuration file and configure Xorg to use the new driver:

cp /etc/X11/xorg.conf /etc/X11/xorg.conf.orig
aticonfig --initial -f

Finally, configure Sax2 with the ATI driver:

sax2 -r -m 0=fglrx

Upon rebooting your computer, you should be able to use the hardware-accelerated 3D capabilities of your ATI card. To verify that things are up and running, execute fglrxinfo as a normal user. This command renders the following output on my system:

display: :0.0  screen: 0
OpenGL vendor string: ATI Technologies Inc.
OpenGL renderer string: ATI Radeon Xpress Series
OpenGL version string: 2.1.8304 Release

Other Thoughts

After having played with OpenSUSE 11.1 for a couple hours, I think I might be able to keep it around for a little while. Despite the lack of speed exhibited by other Linux distributions, the "stability" that OpenSUSE seems to offer is attractive to me. It will likely take some time to get used to RPMs over DEBs for package management.

How bad can it be? I mean, it comes with OpenOffice 3.0.0, which is nice. It can handle dual-head mode on my laptop thanks to Xinerama, which no other distro to date has been able to do. This gives me a little more screen real estate to work with, which helps out a lot when I'm developing a Web site or working in an IDE. The package managers are slow, but how often do you really install software anyway?

Again, we'll just have to see how things pan out. Let's hope it turns out to be a positive experience.

Comments

Installing Python 3.0 Alongside an Existing Python    Posted:


With the recent release of Python 3.0 final, I've had a crazy itch that needed to be scratched. That itch, my friends, was to do a write-up of how to install Python 3.0 alongside my existing Python 2.5.2 installation without borking things up. The reason I thought it would be useful is that I'm running a Debian-based distribution of Linux called sidux right now, and neither Python 2.6 nor Python 3.0 are in the package repositories. I assume that Ubuntu and other Debian-based distributions might be the same way, and that there are others like me who would like to tinker with these new releases.

Just before attempting to install Python 3.0 on my computer, I started getting ready to write this article. Before I knew it, Python 3.0 was installed on my system, and I had no notes to share with you! That is how stinkin easy it is to install Python 3.0 without interfering with an existing install. I mentioned to a good friend that I wasn't sure the process was worth writing an article because it was so simple, but he encouraged me to carry on. Thanks bro.

So, without any further ado, here is what I did to install Python 3.0 from source alongside my Python 2.5 installation:

Note: If you're using a Mac or Windows, you should be able to simply install the packages for your platform to accomplish this same feat. Once the packages are in the repositories it will be just as easy for Linux.

$ wget http://python.org/ftp/python/3.0/Python-3.0.tar.bz2
  • Unpack the archive:
$ tar jxf Python-3.0.tar.bz2
  • Descend into the newly extracted Python-3.0 directory:
$ cd Python-3.0
  • Install libreadline-dev. This step is necessary for the arrow keys to work, as pointed out by jazevec below in the comments. If you are on a Debian-based system, you can execute a command such as this:
  $ sudo apt-get install libreadline-dev

Other distributions may have different package names, such as ``readline-dev``.  If neither one of those package names work for your distribution, try searching your package manager for ``readline`` and install the development files.  Alternatively, you should be able to manually install what you need by installing `readline itself <http://directory.fsf.org/project/readline/>`_.
  • Configure Python 3.0 for your computer:
$ ./configure
  • Compile Python 3.0:
  $ make


**UPDATE** (9 Dec): Depending on your setup, you may or may not see a message such as this after executing the ``make`` command (thanks again to ``jazevec`` for pointing out that this can happen)::

  Failed to find the necessary bits to build these modules:
  _dbm               _gdbm              _hashlib
  _sqlite3           _ssl               _tkinter
  bz2                zlib
  To find the necessary bits, look in setup.py in detect_modules() for the module's name.

If you need any of those capabilities, you should install the appropriate development files for the missing module(s).  For example, above, we installed the ``libreadline-dev`` package.  To resolve the missing module problem for each one listed above (except ``_dbm`` because it's apparently borked on Debian right now... possibly other distros too), install these packages:

  * ``tk-dev`` to satisfy ``_tkinter``
  * ``libsqlite3-dev`` to satisfy ``_sqlite3``
  * ``libbz2-dev`` to satisfy ``bz2``
  * ``zlib1g-dev`` to satisfy ``zlib``
  * ``libssl-dev`` to satisfy ``_ssl`` and ``_hashlib``
  * ``libgdbm-dev`` to satisfy ``_gdbm``
  • Install Python 3.0:
  $ sudo make install

or:
# make install
  • Test Python 3.0:
$ python3.0
Python 3.0 (r30:67503, Dec  5 2008, 11:05:45)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'Hi'
File "<stdin>", line 1
    print 'Hi'
            ^
SyntaxError: invalid syntax
>>> print('Hi')
Hi
>>>
  • Make sure that your old version of Python is still around:
$ python
Python 2.5.2 (r252:60911, Sep 29 2008, 21:15:13)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'Hi'
Hi
>>>
  • Rejoice

That's about it folks! Extremely simple and fast, compared to what it could have been.

If you find yourself in a situation where you don't have access to sudo or straight root-level access, you can install Python 3.0 locally by doing something like this:

  • Configure Python 3.0 for your computer:
$ ./configure --prefix=$HOME/local/
  • Compile Python 3.0:
$ make
  • Install Python 3.0:
  $ make install

Not that I omitted the ``sudo`` part of the command here.
  • Symlink to Python 3.0, assuming you have a bin/ directory in your home directory (i.e. /home/[yourusername]/bin), and that said bin directory is on your PATH:
$ ln -s ~/local/bin/python3.0 ~/bin
  • Test your locally installed Python 3.0:
  $ python3.0

or, if your local ``bin`` directory isn't on your ``PATH``:
$ ~/bin/python3.0
  • Do the dance.

Please comment with any problems you find with this process, or any additional advice you can offer to newbies!

Comments