Django Projects

Over the past 6 years, I've built a lot of things with Django. It has treated me very well, and I have very much enjoyed seeing it progress. I got into Django when I helped the company I was working for transition away from a homegrown PHP framework toward something more reliable and flexible. It was very exciting to learn more about Django at a time when the ecosystem was very young.

When I started with Django, there weren't a lot of pluggable apps to fill the void for things like blogs, event calendars, and other useful utilities for the kinds of sites I was building. That has changed quite a bit since then. The ecosystem has evolved and progressed like mad, and it's wonderful. We have so many choices for simple things to very complex things. It's amazing!

Unfortunately, during this whole time period, my development efforts have shifted from creating my own open source projects to share with the world toward more proprietary solutions for my employers. If it's not obvious to you from my blog activity in recent years, I've become very busy with family life and work. I have very little time to give my open source projects the attention they deserve.

For at least 4 years, I've been telling myself that I'd have/make time to revamp all of my projects. To make them usable with what Django is today instead of what it was when I built the projects. Yeah, this time has never showed up. Take a look at the last time I wrote a blog article!

I have decided to disown pretty much all of my open source Django projects. I've basically done this with one or two of the more popular projects already--let someone else take the reigns while I lurk in the background and occasionally comment on an issue here or there. I truly appreciate those who have taken the initiative here. But there are still plenty of projects that people may find useful that need some attention. I'm putting it up to the community to take these projects over if you find them useful so they can get the love and attention they need.

Here is a list of Django projects that anyone is free to assume responsibility for. Most of them are silly and mostly useless now. Some are unpleasant to look at and could use an entire rewrite.

  • django-articles - Blogging engine, which I wrote for my own site many years ago. This one could probably use a full rewrite. [already maintained by John Leith].
  • django-tracking - Visitor tracking to see what page a user is on and their general location based on IP. Update 2015-07-28: Adopted by bashu.
  • django-axes - Track failed login attempts and ban users. This one is already pretty much owned by aclark4life
  • django-reploc - Designed to show representative/retailer locations. I only used it on one site, but it was designed to be generic.
  • django-smileys - To easily replace things like :) and B-) with emoticons or whatever. Pretty lame. Update 2015-07-28: Adopted by bashu.
  • django-pendulum - A very basic time-tracking app. I believe this has already been forked and used for some more interesting commercial projects.
  • django-bibliophile - A way to let your visitors know what you're reading.
  • django-watermark - Some utilities to add watermarks to images. Update 2015-07-28: Adopted by bashu.
  • Any other Django-related project you see at https://github.com/codekoala?tab=repositories or https://bitbucket.org/codekoala/

The fact that I'm giving up these projects does not mean I'm giving up on Django. On the contrary, I'm still using it quite heavily. I'm just doing it in such a way that I can't necessarily post my work for everyone to use. I honestly don't expect much of this disowning effort, since the projects are mostly stale and incompatible with recent versions of Django. But please let me know if you do want to take over one of my projects and care for it.

Another I-Heart-Linux Story

Just had another experience that represents much of what I love about Linux and free software, and I thought I would share.

I just got my first bluray burner. I have it in a USB3-capable enclosure to use with my USB2-capable laptop. I knew burn speeds would be much lower than the drive is rated to handle primarily because of this arrangement.

A good portion of yesterday included me trying to burn my first bluray in Linux. Unfortunately, the nature of bluray standards and Linux still don't jive very well. My usual burning applications didn't have any trouble at all detecting the burner or the media, but none of them seemed to want to actually burn anything. K3b made a coaster out of one disc; Brasero did nothing. I didn't try using the command line burning tricks that I found on the interwebs.

I decided to give ImgBurn a shot. I installed it with WINE, and it ran beautifully. However, I failed to realize that I left K3b's coaster in the drive and couldn't get a disc to burn. In my own defense, the coaster was still recognized by the computer as a blank BD-R. It was at this point that I just rebooted into Windows just to verify that the drive itself worked for burning.

Windows obviously burned the first disc fine (after I realized the coaster was still in the drive and replaced it with a fresh disc). The media I have are only rated for 4x burn speeds, while the drive itself is rated for 14x burning. Windows only managed to achieve ~2x for the duration of the burn (using ImgBurn again). Also, Windows seemed to struggle trying to keep the buffers full. Every 70 seconds or so, ImgBurn would stop burning to allow the buffers to fill up again and for the hard drive to settle down a bit. But it did end up working--the disc worked in my bluray player that's hooked up to the TV!!! Success!

I decided to give ImgBurn in WINE another shot. I popped a fresh BD-R into the drive, specified the files I wanted to burn, and initiated the burn process. Right off the bat, ImgBurn under WINE was burning at 2.4x. I watched the process for 10 minutes or so before heading upstairs to let it finish. Not once was the main buffer depleted. The drive buffer (4MB) was up and down every so often, but it was never empty from the time the burn began until the time when I left my computer.

I realize that some of you out there will just see the moral of this story being something like, "Linux doesn't support burning bluray discs very well, and Windows worked on the first try." If you find yourself in this boat, you're probably not part of the intended audience of this tale. This is probably because you have some bias against any operating system that isn't the one you're currently using. No, this story is intended for people who like to tinker and don't think it's a waste of time to go through exercises like this.

Anyway, the real reason I wrote this story was to illustrate one of my favorite things about Linux: it's fast! Burning a bluray disc using an application that was written for Windows, running with a (free, open source, community-driven) Windows compatibility layer in Linux is still faster than the same program running natively on Windows. Sure, it might not always be the case, but in my ~12 years of hands-on experience with Linux, I find that it usually is the case.

UPDATE

So I've burned another 5 blurays under Linux with ImgBurn. Here's the interesting part of my most recent burn log:

I 00:51:40 Operation Started!
I 00:51:40 Source File: -==/\/[BUILD IMAGE]\/\==-
I 00:51:40 Source File Sectors: 11,978,272 (MODE1/2048)
I 00:51:40 Source File Size: 24,531,501,056 bytes
I 00:51:40 Source File Volume Identifier: Stargate - Disc 6
I 00:51:40 Source File Volume Set Identifier: 4222067200B6C611
I 00:51:40 Source File Application Identifier: IMGBURN V2.5.7.0 - THE ULTIMATE IMAGE BURNER!
I 00:51:40 Source File Implementation Identifier: ImgBurn
I 00:51:40 Source File File System(s): ISO9660, UDF (1.02)
I 00:51:40 Destination Device: [3:0:0] HL-DT-ST BD-RE  WH14NS40 1.00
I 00:51:40 Destination Media Type: BD-R (Disc ID: PHILIP-R04-000)
I 00:51:40 Destination Media Supported Write Speeds: 4x, 6x, 8x
I 00:51:40 Destination Media Sectors: 12,219,392
I 00:51:40 Write Mode: BD
I 00:51:40 Write Type: DAO
I 00:51:40 Write Speed: MAX
I 00:51:41 Hardware Defect Management Active: No
I 00:51:41 BD-R Verify Not Required: Yes
I 00:51:41 Link Size: Auto
I 00:51:41 Lock Volume: Yes
I 00:51:41 Test Mode: No
I 00:51:41 OPC: No
I 00:51:41 BURN-Proof: Enabled
I 00:51:41 Write Speed Successfully Set! - Effective: 35,968 KB/s (8x)
I 00:52:00 Filling Buffer... (80 MB)
I 00:52:02 Writing LeadIn...
I 00:52:03 Writing Session 1 of 1... (1 Track, LBA: 0 - 11978271)
I 00:52:03 Writing Track 1 of 1... (MODE1/2048, LBA: 0 - 11978271)
I 01:18:33 Synchronising Cache...
I 01:18:34 Closing Track...
I 01:18:35 Finalising Disc...
I 01:18:50 Exporting Graph Data...
I 01:18:50 Graph Data File: C:\users\wheaties\Application Data\ImgBurn\Graph Data Files\HL-DT-ST_BD-RE_WH14NS40_1.00_WEDNESDAY-JANUARY-02-2013_12-51_AM_PHILIP-R04-000_MAX.ibg
I 01:18:50 Export Successfully Completed!
I 01:18:50 Operation Successfully Completed! - Duration: 00:27:09
I 01:18:50 Average Write Rate: 15,067 KB/s (3.4x) - Maximum Write Rate: 18,615 KB/s (4.1x)

As you can see, the last line suggests things are working quite nicely in Linux:

I 01:18:50 Average Write Rate: 15,067 KB/s (3.4x) - Maximum Write Rate: 18,615 KB/s (4.1x)

I would like to note also that two of the discs I've burned so far have had some problems being read on the computer afterwards. These discs do, however, work quite nicely in the bluray player for my TV. Might just be circumstance.

Installing Arch Linux On EeePC 701 4G

I know, I know, a lot of you are asking, "Why on earth would you want to do anything with a machine that's 5 years old?!" The hardware is crap now, and it was crap when the EeePC came out. Well, my friends, that's one of the great things about Linux! It will put new life into old hardware. Granted, the EeePC 701 4G came with Linux on it and all that, but I'm talking about putting something modern on it. Regardless, this guide will hopefully be useful for others who are trying to get Linux installed using LVM that uses something like an SD card as part of the primary volume group.

I've had Arch Linux installed on my EeePC for quite a while, but I only recently began to appreciate the benefits of LVM. The key requirement for this installation was to use LVM partitioning instead of traditional partitioning. I wanted the built-in 4GB SSD and the 8GB SD card to appear as one hard drive to the operating system so I would have that much more space to actually use.

Another key requirement for this installation was that I wanted to do it using a PXE server instead of the usual USB/external CD drive method. I've used PXE servers in the past, but I never really did much with them. I wanted to use this as an excuse to learn a bit more about them.

Prerequisites

If you're following along at home, I'm assuming that you have (access to) the following:

  • An EeePC 701 4G
  • An SD card that will be a permanent fixture in your EeePC
  • A router running DD-WRT that serves as the DHCP server for your network (I am running v24-sp2). Alternatively, just use a USB/CD to install.

PXE Setup

If you want to save some time and just use a USB/CD to install Arch, go ahead and skip to the Arch Installation section.

For those who are unfamiliar with PXE servers, PXE stands for "Preboot eXecution Environment." It allows you to boot your computer over the network using images stored on a different machine. Since I didn't want to plug in my USB key or external DVD drive, this was a perfect option.

I used my main laptop as the PXE server. All I really did was install atftp from the AUR, copy ipxe.pxe from the Arch Linux Netboot page (from the "Using an iPXE image" section), and put it in /var/tftpboot. After that, it was a matter of running atftpd:

rc.d start atftpd

The next step required tweaking the dnsmasq configuration on my router. Here's what I did:

  • Login to router

  • Go to the Services tab

  • Find the DNSMasq section (mine is already enabled for other purposes, so I assume you will want to make sure yours is enabled)

  • In the "Additional DNSMasq Options" box, make sure you have something like:

    dhcp-boot=ipxe.pxe,[your PXE server's name],[your PXE server's IP]
    
  • Apply the changes. I rebooted my router just to be sure.

That's all there really is to the PXE side of things.

EeePC Setup

You'll want to make sure that booting from the network is enabled in your BIOS. I'll post a list of my BIOS settings later.

Then you should be able to boot your EeePC from the network. If your EeePC isn't configured to boot from the network first, you can simply hit Escape while the BIOS POST screen is still showing and select it from the menu that appears.

If all goes according to plan, your EeePC should receive an IP address and download the boot images from your PXE server. Once you select the appropriate Arch Install option from the menu, it should go out and download the ISO and boot it up. It's pretty neat to see in action. Maybe one day I'll try to tweak this to use an ISO image that's available on the local network instead of downloading it from the Internet every time.

Arch Installation

Installing Arch Linux should be fairly simple if you follow the installation guide (which is also available during installation these days at /root/install.txt). The key thing to remember, if you want to use the built-in SSD and a permanent SD card as one hard drive, is to use LVM partitioning.

LVM Partitioning

The Arch Linux LVM guide will likely be an invaluable resource if you're new to LVM. Here I will simply give a short list of commands that I think I used to get my EeePC setup (I did this several days ago, so I might miss something along the way...sorry)

It's important to note that, while it's supposedly possible, I've never had any luck keeping my boot partition in an LVM setup. As such, I first partition my SSD and SD cards using fdisk. There are plenty of resources for fdisk and cfdisk out there, so I'll skip over the specifics of how to use each one and just give you an idea of what needs to be done.

  • Wipe all partitions from both /dev/sda (the SSD) and /dev/sdb (the SD card)
  • Create a 50-100MB primary partition at the beginning of /dev/sda. Mark it as bootable. Set the partition type to 83 (Linux).
  • Create another primary partition on /dev/sda that consumes the remaining disk space. Set the partition type to 8e (Linux LVM).
  • Create a primary partition on /dev/sdb that consumes all space on the SD card. Set the partition type to 8e (Linux LVM).

Now that we have some "Linux LVM" partitions to deal with, we can begin using LVM. Each partition we wish to use (both /dev/sda2 and /dev/sdb1) needs to be configured as an LVM "physical volume":

pvcreate /dev/sda2
pvcreate /dev/sdb1

Next, we create an LVM volume group that uses both of our physical volumes:

vgcreate vg0 /dev/sda2
vgextend vg0 /dev/sdb1

Lastly, create the LVM logical volumes. It's up to you how you setup your partitions, but I chose to create a swap partition (since my EeePC only has 512MB RAM) and a root partition. To create a swap partition, you simply do something like this:

lvcreate -C y -l 1G vg0 -n swap

And here's what the arguments mean:

  • -C y: make the partition contiguous
  • -l 1G: number of logical extents to allocate for the new logical drive
  • vg0: the name of the volume group we created earlier
  • -n swap: the name of the new logical volume

I created my root partition to fill the remaining space in the volume group as follows:

lvcreate -l +100%FREE vg0 -n root

Then it's just a matter of formatting your partitions. I'm not up-to-date on the debate regarding which FS is ideal for solid state media, so I just chose ext2:

mkfs.ext2 /dev/sda1
mkfs.ext2 /dev/mapper/vg0-root
mkswap /dev/mapper/vg0-swap

It's probably worth noting that you now have several ways to reference your newly-created LVM volumes. Here are my favorites:

  • /dev/[volume group name]/[volume name]
  • /dev/mapper/[volume group name]-[volume name]

Install Arch

I really don't want to duplicate the Arch Linux installation guide, so I'll just leave you to follow along there. Go ahead and mount your new partitions, pacstrap everything you want, and arch-chroot your heart away. But you might want to come back before you exit the chroot, otherwise your EeePC might not boot and you'll have to waste some time trying to figure out why.

Help The EeePC Boot With LVM

Before you reboot, you'll probably want to modify your /etc/mkinitcpio.conf to let your EeePC understand your LVM setup. The two key lines are MODULES and HOOKS. Here's what I have:

MODULES="usb-storage scsi-mod sd-mod libata usbcore uhci-hcd ehci-hcd"
HOOKS="base udev autodetect pata scsi sata lvm2 filesystems usbinput fsck"

The key here is to ensure that lvm2 appears before filesystems in your HOOKS list. Also, you want to load some modules to allow the SD card to be detected before the LVM hook attempts to find your partitions. I'm not sure if you need all of those modules, but they worked on the first try for me. To make the changes useful, rebuild your boot images:

mkinitcpio -p linux

To go along with detecting your SD card at boot, you want to give it enough time to settle before LVM tries to use it. In my case, 3 seconds is sufficient, so I added this to my /boot/syslinux/syslinux.cfg:

lvmwait=/dev/vg0/root rootdelay=3

You should be able to add that to the APPEND lines in your syslinux.cfg or the kernel line in your GRUB configuration file. Or whatever it is with GRUB these days.

Also, while you're editing your bootloader configuration, be sure to change the root partition to /dev/vg0/root (or whatever you chose when you did your LVM partitioning). The default syslinux.cfg includes root=/dev/sda3, so I just changed that to root=/dev/vg0/root. Most bootloaders allow you to modify the boot line before you actually boot into Linux if you forgot this step.

Tweak Arch

I'll leave you to install and configure the packages that you find useful. I plan on using my EeePC as a jump box into my home network, so it doesn't need a GUI of any sort. Just OpenSSH and some other goodies.

You might also be interested in my previous article about bonding eth0 and wlan0 now that you have Arch on your EeePC though!

Bonding eth0 and wlan0 on Arch Linux

Bonding eth0 and wlan0 on Arch Linux

Yesterday at work, a couple of my coworkers were trying to bond two ethernet ports on a Synology NAS. For whatever reason, this made me want to play with bonding network interfaces on my own computers. So I began trying to come up with a good reason to use valuable time to play with interface bonding.

One thing that has always driven me nuts about laptops is that connections are severed when I have to unplug my ethernet cable and take the laptop elsewhere. It never seemed to matter if I was using NetworkManager or anything else--connections were dropped, and the resulting annoyances ranged from minor to much rage. Having Pidgin drop connections and reconnect was a minor annoyance. Downloading a 4.2GB linux ISO and having the connection cut at 96% caused much rage.

Anyway, I decided to see if I could use bonding to reduce the frustration with such situations. Since my laptop is usually plugged into the ethernet, I wanted to use eth0 as my primary interface. It's faster and more reliable than wireless. When my ethernet connection fails, for whatever reason, I wanted it to fall back to wlan0. Ideally, things like downloads or streaming music/movies would continue uninterrupted.

Furthermore, I wanted my KVM to be able to use the bonded interfaces inside of my virtual machines. This called for a bridge.

To quickly see this in action, check the videos below!

Software

I configured this on a relatively new Arch Linux installation. Here's the software I used:

  • linux: 3.4.8-1
  • initscripts: 2012.08.2-1
  • netcfg: 2.8.9-1
  • netcfg-bonding: 1.5.1-1
  • ifenslave: 1.1.0-7
  • bridge-utils: 1.5-1

Configuration

The first thing you need to do to get this working, after installing all of the above packages, is load the bonding kernel module. In my particular case, as I mentioned earlier, I wanted to use eth0 when available and use wlan0 otherwise. As such, I used the active-backup bonding mode. If you're interested in the other bonding modes, I would suggest reading up on bonding. I'm pretty new to all of this bonding magic, so I can't guarantee that I'll be of any use if you don't follow my instructions :)

Back to configuration. I placed my bonding configuration in /etc/modprobe.d/bonding.conf. The only thing that may not be self-explanatory is the miimon setting. I learned from this bonding article that this option "specifies the MII link monitoring frequency in milliseconds." In other words, this is how often the system will check to see if a failover is necessary.

/etc/modprobe.d/bonding.conf

options bonding mode=active-backup
options bonding miimon=100
options bonding primary=eth0

After creating that file, you simply insert the module into the kernel:

modprobe bonding

To get bonding to work when you reboot your machine, you should either add bonding to your MODULES array in /etc/rc.conf or, if you're using more recent initscripts stuff, place a file in /etc/modules-load.d. I chose the latter solution.

/etc/modules-load.d/bonding.conf

bonding

Next, you must create some network profiles for netcfg. Prior to this bonding project, I had been using wicd or networkmanager to handle most of my networking needs. It seems like a bad idea to try using either of these, or the standard network daemon even, if you plan to use a configuration similar to what follows. Just FYI.

I created a network profile for the bonded interfaces as such:

/etc/network.d/bonded

CONNECTION="bonding"
INTERFACE="bond0"
SLAVES="eth0"
IP="dhcp"
DHCP_TIMEOUT=10
SLAVE_TIMEOUT=5

Notice that wlan0 isn't in the SLAVES list. This is because I also have a network profile for my home wireless connection to handle the authentication and whatnot:

/etc/network.d/home

CONNECTION="wireless"
DESCRIPTION="Home connection"
INTERFACE="wlan0"
SECURITY="wpa"

ESSID="wouldntyouliketoknow"
KEY="derpies"

IP="static"
IFOPTS="0.0.0.0"
PRE_UP="ifenslave bond0 wlan0"
PRE_DOWN="ifenslave -d bond0 wlan0"

I have one more network profile for the bridged interface that allows my KVM guests to live on the same network as everything else. If you're not using KVM or don't want a bridged interface, you don't need this one.

/etc/network.d/br0

INTERFACE="br0"
CONNECTION="bridge"
DESCRIPTION="KVM Bridge Connection"
BRIDGE_INTERFACES="bond0"
IP="dhcp"
POST_UP="brctl setfd br0 0"

Finally, I added these profiles to my /etc/conf.d/netcfg:

NETWORKS=(bonded home br0)

Again, if you don't need a bridged interface, you don't need to include the br0 profile in there.

You may or may not want to test your individual network profiles by doing something like:

netcfg -u home # try to bring the profile up
netcfg -d home # bring the profile down

If you have problems connecting to your wireless access point, I suggest tweaking and testing your profile until it works. If you use MAC address filtering for wireless security on your router, you might want to make sure that the MAC of your ethernet interface is in the list of permitted MAC addresses. When you setup bonding, all of the bonded interfaces seem to share the same MAC.

Try It Out

Pretty much all that's left is starting up the new network configuration! To do this, first make sure your previous networking daemon is turned off. For example, you might run one of the following commands:

rc.d stop network
rc.d stop wicd
rc.d stop networkmanager

Once you've done that, you simply run:

rc.d start net-profiles

If your setup is anything like mine, you might see ifconfig output similar to this (notice that my MAC addresses have been changed, but that they're identical for bond0, br0, eth0, and wlan0):

bond0: flags=5443<UP,BROADCAST,RUNNING,PROMISC,MASTER,MULTICAST>  mtu 1500
        ether 00:24:be:00:24:be  txqueuelen 0  (Ethernet)
        RX packets 7318719  bytes 8578112000 (7.9 GiB)
        RX errors 0  dropped 33241  overruns 0  frame 0
        TX packets 3404477  bytes 3562828808 (3.3 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.128.55  netmask 255.255.255.128  broadcast 192.168.128.127
        inet6 fe80::224:beff:fec7:9895  prefixlen 64  scopeid 0x20<link>
        ether 00:24:be:00:24:be  txqueuelen 0  (Ethernet)
        RX packets 4759641  bytes 4771378631 (4.4 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2119926  bytes 3491247232 (3.2 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 00:24:be:00:24:be  txqueuelen 1000  (Ethernet)
        RX packets 7243938  bytes 8512402737 (7.9 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3388055  bytes 3561344335 (3.3 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 18

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 16436
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 338869  bytes 263959385 (251.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 338869  bytes 263959385 (251.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 36:11:59:de:92:73  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 00:24:be:00:24:be  txqueuelen 1000  (Ethernet)
        RX packets 74781  bytes 65709263 (62.6 MiB)
        RX errors 0  dropped 33241  overruns 0  frame 0
        TX packets 16422  bytes 1484473 (1.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Also, if you haven't been using netcfg already, you will probably want to update your DAEMONS array in /etc/rc.conf. Again, be sure you disable your previous networking daemon (if necessary) by placing a ! before the daemon's name (or just remove it from the array). Then add net-profiles into the array somewhere, probably closer to the beginning of the array.

Hopefully that works for you! Please share your experiences (both good and bad) in the comments! If you run into troubles, I will try to be of use, and hopefully others will chime in too.

Videos

Here are a couple of videos for those of you who are interested. The first is a lower-quality, but maybe easier-to-read-on-small-screens video. The second is just shy of 1080p size-wise.

Long Time No See

Hello again everyone! Soooo much has happened since I last posted on my blog. I figured it was about time to check in and actually be active on my own site again. What follows is just a summary of what has happened in our lives since the beginning of February this year.

Leaving ScienceLogic

First of all, my wife and I decided toward the end of 2011 that it was time for us to move away from Virginia. For reasons that we did not quite understand yet, we both wanted to move to Utah. I applied for my first Utah-based job opportunity just before Christmas 2011. Several of my friends in the Salt Lake City area were kind enough to get me a few interviews here and there, but none of the opportunities were very serious.

Probably about the time I wrote my last blog post, I was contacted by a recruiter in Boise. I would have loved to move back to Idaho, but my wife would have nothing to do with me if I did that. When I shared this information with the recruiter, he said he had a recruiter friend in the SLC area and that he'd pass my information along to him. Within a day, his friend had me set up with a screening problem for a company just outside of SLC.

I was a little hesitant about that particular opportunity, because it was a Ruby on Rails development shop and an advertising company. However, our timeline was getting smaller and smaller--we had to be out of our apartment by the 1st of April--and I didn't see any other serious opportunities on the horizon. So anyway, I completed the programming problem in both Python and Ruby and had a few video chats with some guys with the company. I guess they liked my work, even though I hadn't touched Ruby in several years.

Sometime in the middle of February, the company extended me an offer letter, which my wife and I considered for a few days before accepting. My last day with ScienceLogic was the 30th of February. My first day with the new company was the 12th of March, so we had a couple of weeks to pack everything up and drive across the country. Packing was ridiculously stressful, but the drive was actually quite enjoyable (my wife wouldn't agree). I drove my Mazda 3 with my 2 year old son in the back, and my wife drove the Dodge Grand Caravan with our 7 month old twins.

The New Job

We arrived in Utah on the 10th of March and immediately fell in love with the little town house we're renting and the surrounding community. It's a really nice area. We spent the first couple of days exploring the area and learning our routes to various locations.

My first week on the new job was interesting. They didn't have much for me to do, and we were all scheduled to go to a local tech conference for the last three days of the week. Very appealing way to begin a new job!

As time went on, I did a bit of work here and there, but most of my time on the job was just spent warming a chair in between requests for things to do. Eventually, I just got fed up with the amount of work I was (or, rather, wasn't) doing. By the beginning of May, I was already looking for another job where I could feel useful.

I got in touch with a guy I worked with for a couple of weeks before he quit working for the company that brought us to Utah (I'm intentionally avoiding the use of the company's name). This guy was only able to stand working for that company for about 3 weeks before he quit and went back to his prior company. He referred me for an interview with his managers, and by the middle of May, I had a new job lined up.

The Better New Job

While I was initially hesitant about the job (test automation), I looked at it as a major step up from what I had been doing since March. That and it cut my commute in half. And they provide excellent hardware. Anyway, I started working for StorageCraft Technology Company at the end of May as a Senior Software Engineer in Test.

My task was to build a framework to make the jobs of the manual testers easier. I had no requirements document to refer to, or any specific guidance other than that. I was simply asked to build something that would make lives easier. StorageCraft had recently hired another test automation developer, and the two of us worked together to come up with a design plan for the framework.

We built a lot of neat things into the framework, gave a couple of demos, and it seems like people are really quite pleased with the direction we've gone. I gave a demo of the (Django) UI just the other day, and my supervisors basically gave me the green light to keep building whatever I wanted to. Since the other test automation guy got the boot for being unreliable, I will get to see many of my plans through exactly the way I want! I'm really excited about that.

Enough About Work

Aside from all of the excitement in my career decisions, things are going very well with the family. We live about 3 hours away from my mom, and we've been out there to visit a few times already. It's really fun to see the kids playing with their grandma! The last time we were out for a visit, for my grandmother's 80th birthday, my son and I took my dad's Rhino for a spin. We got stuck, and it was sooo much fun!

Mudding in the Rhino

The twins are growing so well too. They're crawling and getting around very well now. Jane has started to stand up on her own, and she tries to take a step every once in a while. Claire prefers to sit, but she loves to wave, clap, and repeat noises that she hears.

My wife is planning on starting up a new website soon, and she keeps taunting me with the possibility of having me build it for her. Yes, taunting.

Okay, Back to Hobbies

My wife also picked up a Dremel Trio for Dad's Day. To get used to it, I made some little wooden signs with the kids' names on them. Being the quasi-perfectionist that I am, I'm not completely satisfied with how they all turned out. I suppose they'll do for a "first attempt" sort of result though!

First project with the Dremel Trio

I've still got various projects in the works with my Arduino and whatnot. A couple of months ago, I finished a project that helps me see where I'm walking when I go down to my mancave at night. The light switches for the basement are all at the stairs, and my setup is on the opposite side of the basement. I typically prefer to have the lights off when I'm on my computer, and it was annoying and horribly inefficient to turn the lights on when entering the basement, go to my computer, then go back to turn the lights off.

To solve that problem, I re-purposed one of my PIR motion sensors and picked up a LED strip from eBay. I have the motion sensor pointing at the entrance to the basement, and the LED strip strung across the ceiling along the path that I take to get to my desk. When the motion sensor detects movement, it fades the LED strip on, continues to power it for a few seconds, and gradually fades them out when it no longer detects movement. It's all very sexy, if I do say so myself.

Lazy man's light switch

I've tried to capture videos of the setup, but my cameras all have poor light sensors or something, so it's difficult to really show what it's like. The LED strip illuminates the basement perfectly just long enough for me to get to my desk, but the videos just show a faint outline of my body lurking in the dark. :(

One project that is in the works right now is a desk fan that automatically turns on when the ambient temperature reaches a certain level. The fan's speed will vary depending on the temperature, and there will be an LCD screen to allow simple reporting and configuration of thresholds and whatnot. I'm pretty excited about it, but I want to order a few things off of eBay before I go much further with it.

Obviously, much more had happened in the past months, but this post is long enough already. Things are calming down quite a bit now that we're settled in, so I hope to resume activity on my open source projects as well as this blog.

Command Line Progress Bar With Python

Wow, it's been a very long time since I've posted on my blog. It's amazing how much time twins take up! Now they're almost 6 months old, and things are starting to calm down a bit. I will try to make time to keep my blog updated more regularly now.

Earlier today, a good friend asked me how I would handle displaying a progress bar on the command line in a Python program. I suggested a couple of things, including leaving his already working program alone for the most part and just having a separate thread display the progress bar. I'm not sure exactly why, but he decided to not use a separate thread. It didn't seem like it would be very difficult, so I spent a few minutes mocking up a quick little demo.

This is by no means the most perfect of solutions, but I think it might be useful for others who find themselves in similar circumstances. Please feel free to comment with your suggestions! This is literally my first stab at something like this, and I only spent about 5 minutes hacking it up.

Arduino-Powered Webcam Mount

Earlier this month, I completed yet another journey around the biggest star in our galaxy. Some of my beloved family members thought this would be a good occasion to send me some cash, and I also got a gift card for being plain awesome at work. Even though we really do need a bigger car and whatnot, my wife insisted that I only spend this money on myself and whatever I wanted.

Little did she know the can of worms she just opened up.

I took pretty much all of the money and blew it on stuff for my electronics projects. Up to this point, my projects have all been pretty boring simply because nothing ever moved--it was mostly just lights turning on and off or changing colors. Sure, that's fun, but things really start to get interesting when you actually interact with the physical world. With the birthday money, I was finally able to buy a bunch of servos to begin living out my childhood dream of building robots.

My first project since getting all of my new toys was a motorized webcam mount. My parents bought me a Logitech C910 for my birthday because they were tired of trying to see their grandchildren with the crappy webcam that is built into my laptop. It was a perfect opportunity to use SparkFun's tutorial for some facial tracking (thanks to OpenCV) using their Pan/Tilt Servo Bracket.

It took a little while to get everything setup properly, but SparkFun's tutorial explains perfectly how you can get everything setup if you want to repeat this project.

The problem I had with the SparkFun tutorial, though, is that it basically only gives you a standalone program that does the facial tracking and displays your webcam feed. What good is that? I actually wanted to use this rig to chat with people!! That's when I set out to figure out how to do this.

While the Processing sketch ran absolutely perfect on Windows, it didn't want to work on my Arch Linux system due to some missing dependencies that I didn't know how/care to satisfy. As such, I opted to rewrite the sketch using Python so I could do the facial tracking in Linux.

This is still a work in progress, but here's the current facial tracking program which tells the Arduino where the webcam should be pointing, along with the Arduino sketch.

Now that I could track a face and move my webcam in Linux, I still faced the same problem as before: how can I use my face-tracking, webcam-moving program during a chat with my mom? I had no idea how to accomplish this. I figured I would have to either intercept the webcam feed as it was going to Skype or the Google Talk Plugin, or I'd have to somehow consume the webcam feed and proxy it back out as a V4L2 device that the Google Talk Plugin could then use.

Trying to come up with a way of doing that seemed rather impossible (at least in straight Python), but I eventually stumbled upon a couple little gems.

So the GStreamer tutorial walks you step-by-step through different ways of using a gst-launch utility, and I found this information very useful. I learned that you can use tee to split a webcam feed and do two different things with it. I wondered if it would be possible to split one webcam feed and send it to two other V4L2 devices.

Enter v4l2loopback.

I was able to install this module from Arch's AUR, and using it was super easy (you should be root for this):

modprobe v4l2loopback devices=2

This created two new /dev/video* devices on my system, which happened to be /dev/video4 and /dev/video5 (yeah... been playing with a lot of webcams and whatnot). One device, video4, is for consumption by my face-tracking program. The other, video5, is for VLC, Skype, Google+ Hangouts, etc. After creating those devices, I simply ran the following command as a regular user:

gst-launch-0.10 v4l2src device=/dev/video1 ! \
    'video/x-raw-yuv,width=640,height=480,framerate=30/1' ! \
    tee name=t_vid ! queue ! \
    v4l2sink sync=false device=/dev/video4 t_vid. ! \
    queue ! videorate ! 'video/x-raw-yuv,framerate=30/1' ! \
    v4l2sink device=/dev/video5

There's a whole lot of stuff going on in that command that I honestly do not understand. All I know is that it made it so both my face-tracking Python program AND VLC can consume the same video feed via two different V4L2 devices! A co-worker of mine agreed to have a quick Google+ Hangout with me to test this setup under "real" circumstances (thx man). It worked :D Objective reached!

I had really hoped to find a way to handle this stuff inside Python, but I have to admit that this is a pretty slick setup. A lot of things are still hardcoded, but I do plan on making things a little more generic soon enough.

So here's my little rig (why yes, I did mount it on top of an old Kool-Aid powder thingy lid):

And a video of it in action. Please excuse the subject of the webcam video, I'm not sure where that guy came from or why he's playing with my webcam.

Server Downtime

I just wanted to take a minute to apologize to any of my numerous loyal visitors (can you tell I'm being sarcastic?) who were disappointed to find my site down for the past few days. My web server had some problems, and I simply did not have the time to bring the site back up properly until today. That is all :)

Firefox 6.0 Is My Default Browser Again

I've just made Firefox my default browser again (after using Chrome/Chromium as my default for quite some time). FF6 renders things almost as fast as Chrome, even on our ridiculously complicated pages at work. This was my primary reason for not using FF for such a long time.

However, there are other things that I've decided are worth going back to FF at least for the time being. If any of you know of ways to accomplish this stuff with Chrome, please enlighten me!

  • Tab groups. I often have dozens and dozens of tabs open. Some I like to have open for my "night life." Some I like to have open for research. Some I like to have open for work. Tab groups let me organize these, and hide the ones I'm currently not interested in.
  • Vimperator. This was one thing I missed dearly when I decided to ditch FF for Chrome back in the day. Vimium just doesn't work as well. It's close, but still not as robust. For example, I can still use Vimperator shortcuts when viewing a raw file. Not so with Vimium.
  • Permanent exceptions for bad certificates. Normally this wouldn't be a big deal. You visit a site that has a bad certificate, your browser warns you that bad things could be happening and advises you to be careful. That's great! However, for work we all use several VMs for development (I often have 7 VMs running all the time just for my day-to-day routine). They all share the same certificate, which is bad. Chrome doesn't let me tell it to stop bothering me about the bad certificate. Firefox does.
  • Memory usage. Strangely enough, memory usage isn't something that most people say is awesome in FF these days. But I've personally discovered that it is better than Chrome on "poorly-endowed" systems. Both Chrome and FF are consume gobs of RAM on my regular 8GB+ RAM machines, but I have plenty to spare so it doesn't bother me much. I recently inherited my wife's old MacBook, which only has 1GB of RAM. Chrome is absolutely painful to use--it's slow and brings the rest of the system to a crawl. Firefox is noticeably more responsive on that machine.
  • Selenium. I don't do much UI testing these days, but I do still use Selenium to automate some mundane tasks for work. Haven't checked to see if this is available for Chrome for a while...

So far, the only regret I really have with my decision to move back to Firefox (again, for the time being) is the amount of chrome. Chrome does a good job of staying out of the way and letting me see the web page. Firefox just has just a bit more going on at the top and bottom of the window. Definitely not a deal-breaker on my 1920x1080 laptop though ;)

Lots Of Happenings

Yay! First post in nearly 4 months! I feel kinda bad for leaving my April Fools Day joke on the front page for such a long time, but lucky for me my blog isn't very popular! I could have found myself in a world of hurt.

I'd like to give you all a quick update on what's been going on in my life that somewhat justifies a 4-month window of no blog posts. First of all, we had 25% of our backend development team (1 person, leaving 3 developers) get fired earlier in the year, so the workload at my day job got to be a bit heavier.

Second, my wife and I were pregnant with identical twin girls. It was a relatively high-risk pregnancy, so we spent many days in the hospital for checkups and whatnot. The doctors gave her somewhat strict bed rest orders, and I worked from home since the beginning of July (my job is awesome that way) so I could keep and eye on my wife and help with our soon-to-be two year old son.

Third, we had our identical twin girls this past Tuesday. They arrived at 9:25 and 9:27, and their names are Claire and Jane. Claire weighed 5 lb 3 oz, and Jane weighed 4 lb 14 oz. As expected, they lost a bit of weight at the beginning, but they're starting to gain weight again. My wife and the girls are all doing very well.

My son, on the other hand, is starting to realize that his world is changing quite drastically. We're trying to give him as much attention as we can, but it's definitely not the amount that he's used to. He seems to do very well with his new baby sisters, but there have already been several episodes where he just breaks down. It's sad.

I have a few days of leave and vacation that I'll be taking to help get everything settle at home. The girls are still in the "let's sleep all day and night until we're hungry" sort of phase (which is quite awesome), and my wife is up most of the night with them. That means the only person who doesn't sleep a ton during the day is my son, so we get some good father/son time.

Anyway, on to the nerdy stuff. Since I find myself with a couple hours of downtime here and there, I plan to do a bug-smashing ticket-resolving spree. Just as my blog has sat dormant for months, so have many of my side projects. Now is the time to change that!