Step-by-Step: Installing Django

Being the Django and Python zealot that I am, I often find myself trying to convert those around me to this awesome development paradigm. Once I break them, these people often ask me a lot of questions about Django. Over the past few months I've noticed that one of the biggest sticking points for people who are new to Django is actually getting it up and running to begin with. In response, this is the first in a series of articles dedicated to getting Django up and running.

What is Django?

The Django Web site describes Django as "a high-level Python Web framework that encourages rapid development and clean, pragmatic design." Basically, Django is just about the most amazing thing for Web development. I have tinkered with several different Web technologies, but nothing seems to even come close to what Django can do for me.

What is Python?

Python is a programming language used in numerous aspects of computing these days. It has a very simple yet powerful syntax. It's an easy language for beginners to pick up, but it provides adequate levels of power for the more experienced developers out there. If you have never programmed anything before, or you have dabbled with something like BASIC, Python should be fairly straightforward. If you are a programming veteran, but have only worked with languages like C, C++, Java, etc, you might struggle a bit with the syntax of the language. It's not difficult to overcome the differences in a couple hours of hands-on development.

Let's get started.

Installing Python...

Having Python installed is critical--Django does not work without Python. I'm guessing that you're relatively familiar with the procedures for installing software packages on your particular operating system. However, I will share a few notes to point you in the proper direction if you're lost. If nothing else, just head over to the Python download page to download anything you need to install Python. I whole-heartedly recommend using the latest stable version of Python for Django, but you should be able to get by with as early a version as 2.3.

...On Windows

Simply grab the latest version of the Python installer. It is currently version 2.5.2. Once the installer has downloaded successfully, just run through the installation wizard like any other setup program.

...On Mac OS X

Recent Mac OS X computers come with Python pre-installed. To determine whether or not you actually have it, launch the Terminal (Applications > Utilities > Terminal) and type python -c "import sys; print sys.version". If Python is already installed, you will see the version you have installed. If you have a version that is less than 2.3, you should download the newest version. If you don't have Python installed, you will get a "command not found" error. If you're in this boat, just download the latest version of the Python Universal installer and install it.

...On Linux

Most Linux distributions also have Python pre-installed. Just like with Mac OS X, you can check to see by opening up a terminal/konsole session and running the command python -c "import sys; print sys.version". If you have Python installed, you will see its version. If you get an error message when running that command, or you have a version earlier than 2.3, you need to download and install the latest version of Python.

If you're running a Debian-based distribution (like Ubuntu, sidux, MEPIS, KNOPPIX, etc), you can probably use sudo apt-get install python to get Python. If you're running an RPM-based Distribution, you can probably use something like Yum or YaST to install Python.

A sure-fire way to install Python on any Linux system, however, is to install from source. If you need to do this, you simply:

  1. download the source for the latest version of Python
  2. extract it: tar jxf Python-2.5.2.tar.bz2
  3. go into the newly-extracted directory: cd python-2.5.2
  4. configure it: ./configure
  5. compile it: make
  6. install it: make install

(I've only installed Python from source one time, so I might be wrong)

Setting Up Your PYTHONPATH...

Generally speaking, if you didn't have Python installed before starting this tutorial, you will need to setup your PYTHONPATH environment variable. This is a variable that lets Python know where to find useful things (like Django).

...On Windows

  • Open up your System Properties (Win+Break or right click on "My Computer" on your desktop and select Properties)
  • Go to the "Advanced" tab
  • Click the "Environment Variables" button
  • If you have permission to change system variables, click the "New" button in the bottom pane. Otherwise, create the PYTHONPATH variable for your user account using the "New" button in the top (User variables for [username]) pane.
  • Set the variable name to PYTHONPATH
  • Set the variable value to C:\Python25\Lib\site-packages (replace C:\Python25\ with whatever it is on your system if needed)
  • Save it

You may also need to add the python executable to your PATH. If you can successfully run python from a command prompt window, you don't need to worry about it.

If you can't run python from a command prompt, follow the procedure above, but use the PATH variable instead of PYTHONPATH. PATH most likely already exists, so you just need to append/prepend the existing value with something like C:\Python25\ (again, this might need to change depending on where you installed Python)

...On Mac OS X

Your PYTHONPATH should already be setup for you.

...On Linux

Usually you just need to edit your ~/.bash_rc script to setup your PYTHONPATH environment variable. Go ahead and open that up in your preferred text editor and make sure there's something in it like:

export PYTHONPATH=/usr/lib/python2.5/site-packages:$PYTHONPATH

Save any changes necessary and run the following command:

source ~/.bash_rc

This will take care of updating your current session with any changes you made to your ~/.bash_rc.

Installing Django

Once you have Python and have verified that you have version 2.3 or later, you are ready to install Django. Currently, the latest stable release is 0.96.1, but this is grossly out-dated. Django 1.0 will be released on September 2nd 2008, so the "unstable" copy of Django is pretty close to what 1.0 will have to offer. There are some incredibly useful improvements in the unstable version that I don't think I could do without anymore, so that's what I'll talk about installing here.

First, you need to have a subversion client. On Windows, the most popular one is called TortoiseSVN. On Mac OS X, I have played with a few, but I think Versions is a pretty decent one. Linux also has several to choose from, but if you're using Linux, you're probably going to use the command line anyway (right?).

For brevity, I will just use the subversion commands necessary to accomplish this task (instead of discussing all GUI interfaces to subversion).

The exact location that Django should be installed differs from system to system, but here are some guidelines for typical setups:

  • Windows: C:\Python25\Lib\site-packages
  • Linux: /usr/lib/python2.5/site-packages
  • Mac OS X: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages

If you want a definite location, run the following command:

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Once you know that location, go there in your command prompt or terminal session. Then execute this command svn co http://code.djangoproject.com/svn/django/trunk/django django. You will see loads of output, showing all of the files that you are downloading as you install Django.

As soon as that process completes, you should run python -c "import django" to make sure everything worked properly. If the command doesn't display an ImportError, you're good. Otherwise, you need to try again.

Getting Access to Django Scripts...

Once you can successfully import django, you might want to make sure you can run the django-admin.py script that comes with Django.

...On Windows

This process is very similar to what we did with the PYTHONPATH environment variable earlier.

  • Open your System Properties again
  • Go to the Advanced tab
  • Click the Environment Variables button
  • Find your PATH environment variable (either for your user or system-wide)
  • Make sure that the variable value contains something like C:\Python25\Lib\site-packages\django\bin
  • Save any changes
  • Open a fresh command prompt
  • Try to run django-admin.py. If you're successful, you're ready to get started with Django. Otherwise, you need to fix your path to django/bin or just call the django-admin.py script using an absolute path when needed.

...On Mac OS X

You can run a command similar to this:

sudo ln -s /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/bin/django-admin.py /usr/local/bin

...On Linux

If you have "root" privileges on your Linux system, you can execute a command like:

sudo ln -s /usr/lib/python2.5/site-packages/django/bin/django-admin.py /usr/local/bin

If you don't have "root" privileges, you can setup your own /usr/local/bin:

mkdir ~/bin

Make sure your ~/.bash_rc contains something like:

export PATH=$HOME/bin:$PATH

Then update your current session with any changes you made to ~/.bash_rc by running this command:

source ~/.bash_rc

And that should do it! Now you should be ready to get started with Django.

Feel free to leave a comment if you're having problems installing Django. Good luck!

Check out Installing Django on Shared Hosting.

By George, I Think They've Got it!!

For anyone who's been using Django for any reasonable amount of time, you've probably heard of this thing called the "newforms-admin branch." It's an improved way to interact with the content on your Django website via the built-in Django Administration utility. This particular item has been hot for quite some time--a couple years at least.

These past few weeks have been very exciting for Djangonauts out there. The core developers announced that they would be holding several sprints to bring Django to 1.0 status, which required the completion of the newforms-admin branch. All of this was supposed to be ready by September 2nd.

This morning I decided to do some development while my wife was still asleep. I started by updating several things that I rely upon for my Django sites. Among these was Django itself. Then I noticed that my code wouldn't validate anymore. That's when the excitement began...

I checked the Django website and found this article. It's a public service announcement stating that some backward-incompatible changes were going to be merged into trunk yesterday. Then I checked to see if there were still any tickets stopping the newforms-admin branch from being merged into trunk. None.

So, folks, the official Django newforms-admin is among us. Time to start modifying all of our models and code to play well with the next generation of Django. YAY!

New Website!

In case you haven't noticed, this website is a bit different than vcboard.net where all of this content used to be. As stated on the homepage, I am planning on moving all content from vcboard.net over here to codekoala.com, and then putting vcboard.net back to its originally intended purpose. That could take a while.

I am still in the process of moving all data over, but I think I've got most everything that is critical so far. Please be patient with me while I continue transferring data.

Flash & Silverlight - Possible Agenda Behind Them?

Being a web developer, I'm often presented with questions about what I prefer to use for building websites. I tend to build websites by hand using a plain old text editor. Obviously there are a lot of other tools that I use in the process of creating an entire website, but the fact that I use a text editor is sufficient information for the time being. Some of the people who ask me about my preferences often ask why I don't use Flash or similar technologies when they learn of my archaic habits. I have my personal reasons, but I recently read an article that presented many interesting thoughts concerning the potential future of Adobe's Flash and Microsoft's Silverlight that make me feel all the better about my decision to not use either one.

Many website owners do not feel that they can have a truly "professional" website without having at least one thing done with Flash or Silverlight. Developers often feel that technologies such as these save them massive amounts of time and offer increased functionality. I cannot argue with either of those points in a general sense (I actually used Flash a lot, long before Adobe bought Macromedia). Tristan Nitot, founder of Mozilla Europe, discussed the possibility of these two technologies becoming a problem for web developers in the future. As it stands right now, both companies freely release their respective plug-ins for use all over the world. But what happens if the companies decide to start charging for the plug-ins? Developers already have to fork out an arm and a leg just to be able to create Flash and Silverlight content. When will the consumers be forced to pay as well?

Nitot also feels that such technologies are limiting to the open nature of the Internet. He suggests that the next version of HTML will be capable of presenting video and audio files natively, requiring the user to download no plug-ins at all. This idea in and of itself reduces the usefulness of both Flash and Silverlight tremendously, as many websites simply use Flash applications for audio or video playback (think YouTube).

If you're a web monkey like myself, this article is worth a read. It definitely opened my mind to a few of the potential problems in the future of the Internet.

SwiftFox Debian Repositories

Since Debian is a distribution of Linux that tries to stay as "free" as possible, some of the software it is released with is not exactly useful all the time. Case in point: a site I recently visited complained that I didn't have an acceptable web browser. It said that I could use Firefox, but Debian packages "Iceweasel" instead of Firefox. They are basically the same program, only with different names (as far as I'm concerned). However, there is another program called "SwiftFox," which is a custom build of Firefox that is optimized for various CPU's. This browser is still accepted as Firefox, so this article will explain how to set your Debian system up to be capable of automatically installing and updating SwiftFox.

  1. The SwiftFox website does not currently offer a keyring, so you don't have to worry about that right now. Skipping this step will often give you warnings or error messages when you go to update your repositories or install something from the untrusted repository.

  2. Install the repositories using the following line:

    deb http://getswiftfox.com/builds/debian unstable non-free

    This line typically goes in the /etc/apt/sources.list file. Just put it on its own line. Some newer Debian systems prefer that you put this line in /etc/apt/sources.list.d/[repository].list (where [repository] would likely be swiftfox in this case).

  3. Run apt-get update as root to make sure the repositories are working properly. If they are, you should see no errors or warnings pertaining to swiftfox.com. If you see any such messages, try updating again.

  4. Install SwiftFox Since the builds are all optimized for various processors, check this list and this other list to figure out which package you should install. Since I'm running a 32-bit operating system on a Turion 64, I would run the following command as root:

    apt-get install swiftfox-athlon64-32bit

    You'll likely have to explicitly say "yes" when it asks if you really want to install the package.

Once that completes, you should be able to launch SwiftFox without a hitch!

Opera Debian Repositories

Being a web developer, I have an obligation to ensure that the sites I build work on the wides range of popular web browsers as possible. Because of this, I often find myself installing the wonderful Opera web browser on my Linux systems. Now, I could just download it directly from http://www.opera.com/ each time there's an upgrade, but it is much easier to handle when running a Debian-based distribution thanks to the apt-get infrastructure. This article will explain how to set your Debian system up to be capable of automatically updating Opera any time there is a new release.

  1. Install the keyring. A keyring is simply a way to verify the identity of the repository. It kinda makes sure that you are installing from the server you want to be installing from, instead of a different server that has unauthorized packages. To install the keyring, run the following command as root (or with root privileges using sudo):

    wget -O - http://deb.opera.com/archive.key | apt-key add -

    Note: Copying and pasting the above command does not seem to play well. The dash immediately before the http://... is translated improperly on this webpage. It should just be a regular dash.

  2. Install the repositories. This part usually depends upon your particular flavor of Debian. I am running Debian Sid, so I install the Opera repository using the following line:

    deb http://deb.opera.com/opera/ sid non-free

    or if I am feeling brave, I can install the "unstable" testing version of opera with

    deb http://deb.opera.com/opera-beta/ sid non-free

    If you are running Debian “etch”, you would change sid to etch and likewise for other flavors of Debian. This line typically goes in the /etc/apt/sources.list file. Just put it on its own line. Some newer Debian systems prefer that you put this line in /etc/apt/sources.list.d/[repository].list (where [repository] would likely be opera in this case).

  3. Run apt-get update as root to make sure the repositories are working properly. If they are, you should see no errors or warnings pertaining to opera.com. If you see any such messages, try updating again.

  4. Install Opera. Run the following command as root:

    apt-get install opera

Once that completes, you should be able to launch Opera without a hitch!

Rude Awakening

So I woke up yesterday morning with what I thought was plenty of time before I had to get ready for church. I woke up, looking at the alarm clock, thinking that I had at least 4 hours to spare. Mindy was sound asleep, so I just pulled out my laptop and checked my email and surfed the web for a bit.

Before I knew it, my alarm clock said "10:22" and I figured it was about time to get ready for the day. That same moment, I looked at the clock on my laptop. It said "11:24". I had an appointment at noon. I was very confused... I looked at my alarm clock. I looked at the clock on my laptop. I looked at my alarm clock. I looked at the clock on my laptop. I looked outside. I looked at Mindy. I looked at the alarm clock and scratched my head. Then I checked my cell phone. It agreed with my laptop. What the... Did Mindy intentionally set the clock to be an hour behind so she could get more sleep? I was soooo confused.

I called the bloke I had an appointment with. He enlightened me by telling me that daylight savings time had taken effect. What the heck. Isn't that normally in April?

I have a big problem with daylight savings time. Those folks in Arizona have it right. There's absolutely no logical reason in my mind for jumping ahead of falling back an hour. If people are so concerned about having more daylight, why not just jump ahead 30 minutes or fall back 30 minutes and call it good for the rest of eternity??? That way we wouldn't be losing or gaining (as much as I do like that part) an hour just when we get used to the schedule. We'd always have a balance, because we'd be right in the middle. Bunch of crackheads.

Give me back my hour (or at least half hour) of sleep!!!

Why I Like Python

For the past 8 years or so, I've been very much involved with programming using the PHP scripting language. It is a powerful scripting language that suits building websites very well. PHP has a huge set of useful built-in functions, and more recent versions support object-oriented programming. I first started teaching myself PHP when I got tired of having to build each and every web page on my site manually. I hated having to change dozens of web pages just because I added a new link to my navigation. All sort of reasons like this prompted me to investigate PHP. Little did I know then that this language would occupy so much of my time in the future.

I rapidly learned that PHP offered much more than just allowing me to update one part of my website to change all pages. I started tinkering with all aspects of what PHP offered, and I'm still learning about it. After many years of searching, I finally found a programming language that was easy, fast, and efficient for my needs.

Through the years, I continued to develop various applications using PHP. I attempted to write my own forum/bulletin board software while I was still in high school. If I may say so myself, the forum really had some awesome concepts behind it. But my problem was that I lost interest too fast. I also built a very large application that reduced a 1.2GB MS Access database down to less than 15MB using PHP and MySQL. The new application offered many enhancements over the previous system. For one thing, it was much faster. Second, it allowed multiple simultaneous users to modify the database. Three, so far it has lasted more than 3 years, compared to the 1 year maximum that the MS Access solution always seemed to hit before it crashed.

Using PHP, I helped revolutionize the way one of the companies I work for developed websites. I built a simple in-house web framework that supposedly reduced development time by allowing us to forget about the mundane details involved in virtually every website and just get to the developing. In a matter of two weeks (with a full class load and another job), I managed to write an e-commerce solution for the same company using PHP.

Basically, PHP has treated me well over the years. But this post is not supposed to be about PHP. If that's the case, why have I rambled about PHP this whole time, you ask? Well, it's mostly to demonstrate that I have a lot of experience with the language. I have a pretty good feel for what it's capable of and how I can accomplish most anything I need.

With all of that in mind, I've encountered my frustrations with PHP. They may seem petty and moot to most people, but they have turned out to be the determining factor in what scripting language I prefer. Here is a short list of things I now despise about PHP:

  • dollar signs ($) to signify variables -- while this is a useful feature, it becomes quite bothersome when you're programming all day long (at least it does for me). I'll get to why later.
  • using an actual arrow (->) to access attributes -- most other modern programming languages simply use a period (.) for this functionality. I'll comment more on this and why it frustrates me later as well.
  • lack of true object-oriented constructs -- in other object-oriented languages, like Java, if you have a string and you want to determine its length, you call the length() method of that string. In PHP, you call a function such as strlen($var). This sort of behavior plagues the language.
  • too many unnecessary keystrokes -- as I mentioned before, all mutable variables are preceded by a dollar sign ($). That is 2 keystrokes (shift and 4) every time you want to refer to a variable, wheres most languages nowadays have none). Likewise, accessing attributes of objects in PHP uses an arrow (->), which is three keystrokes (minus, shift, and .). Most other object-oriented languages only require a period (one keystroke) for such functionality. The main reason I make such a big deal out of the number of keystrokes is simple. The more keystrokes a program requires, the more likely you are to have bugs. The fewer keystrokes a program requires, the less likely it is that your program will be broken. It boils down to maintainability. Also associated with the number of keystrokes is the pure laziness within me and most other programmers.

These frustrations have been bothering me for several years now. I continued using PHP mostly because it's so widely supported, but also because I could not find a suitable replacement for it. I investigated a few others, but they apparently didn't have a great influence on me right now because I don't remember any names.

When the whole Ruby on Rails bandwagon was rolling through town, I decided to hop on to see what all of the hubbub was about. I started studying the Ruby script language, and I found that it had some really neat things about it. It uses a more solid approach to object-oriented programming, which I really liked. I also noticed that it employs some intriguing structures for accomplishing things in ways I've never seen before. Despite these things, Ruby still didn't seem like a viable replacement for my PHP. It didn't come up to snuff in performance in many cases, so I essentially abandoned it.

For at least a year now, I've been interested in learning Python. I've heard a lot about it over the years, but I just never seemed to make the time to actually sit down and study it. That is, not until about the beginning of August of 2007. After I made my decision that Ruby and Ruby on Rails weren't quite up to par for my needs, I stumbled upon the Django Project, which is a web framework similar to Ruby on Rails, only built using Python.

I decided this was my chance to actually sit down and learn a little about this "Python" so I could see what it had to offer. I mostly used Django as my portal to Python. As I started learning Django, I became more and more familiar with the way Python works and how I work with Python.

At some point in time, I decided that I actually liked Python, and my wife let me buy some really cool books to help me learn it. By the beginning of October 2007, I had convinced my supervisor at work to let me start building websites using Django instead of our home-grown PHP framework.

And here comes a story. This is the main reason I blabbered about my experience with PHP so much at the start of this article. Again, after all these years, I feel very confident that I can do just about anything I want efficiently and elegantly with PHP.

Back in October of 2006 (after using PHP for some 7 years), I was asked to write a PHP script to parse some log files and output various bits of information in a certain format. After maybe a week, I had a script that did the job fairly well. Most of the time it worked, but there were occasions when it didn't and I had to fix it. The script turned out to be 365 lines of code with very few comments scattered throughout. It's also a maintenance nightmare, even for me.

In October of 2007, I rewrote that same script in Python. After only a couple days, the script seemed to be perfect. It did its job, and it did it well. With comments for just about every single line of code, the Python version of the script took up a mere 118 lines of code. Take out the comments and it is 56 lines of code. The script is several times more understandable and maintainable than its PHP counterpart. I also believe that it is much more efficient at doing its task. Keep in mind that I had only been using Python for about 2 months at this point in time.

It's been through various experiences like the log parser that I have decided I prefer Python over PHP. Obviously, I'm not quite as comfortable with it as I am with PHP, but I don't feel too far behind. Now, less than 6 months after deciding that we'd use Django at work, I don't think my supervisor could be happier. Building a typical website with our PHP framework takes between 1 week and a couple months. Thanks to Python and Django, most of our websites can be "ready" within just a few hours. That time assumes that the website's design itself is ready for content to be put into it and also that the client does not require custom-designed applications.

Python and Django have helped revolutionize the way we do things at work, and I can hardly stop thinking about it. Python fixes nearly all of the frustrations I had with PHP. The frustrations it doesn't take care of are worth the sacrifice. Python is capable of object-oriented programming. It uses a period (.) to access object attributes. Variables are not preceded by some arbitrary symbol.

Also, the fact that Python code can be compiled to bytecode (like Java) is enormously beneficial. Each and every time a PHP script is executed, the PHP interpreter must parse the code. With Python, the first time a script is executed after an edit, the program is compiled to bytecode and subsequent executions are faster. That is because the bytecode is processed directly by the Python Virtual Machine (as opposed to being compiled to bytecode _each_ time and then executed). Python also offers a vast amount of standard library functions that I would really appreciate having in PHP. But from now on (at least for the foreseeable future), I will try to do all of my scripting in Python and leave PHP for the special cases.

Disable Page Navigation with Horizontal Scroll

The following information was snagged from http://gentoo-wiki.com/HARDWARE_Synaptics_Touchpad#Horizontal_Scroll_Issues_with_Firefox

I was looking for this information for a long time and couldn't find it. I finally found reference to this in a cached version of this (gentoo-wiki).

So that firefox will not misinterpret the horizontal scroll as 'back' and 'forward' For many like me this is irritating because you are reading a webpage and by moving the mouse you accidently go to another page.

Some forums suggest disabling horizontal scroll (editing xorg.conf to: Option "HorizScrollDelta" "0"), but instead the correct way is to configure firefox so that it doesn't misinterpret the horizontal scroll. In firefox type in URL (address bar):

about:config

and double-click on the line:

mousewheel.horizscroll.withnokey.action

to set it to 0. And then also set

mousewheel.horizscroll.withnokey.sysnumlines

to true.

Linux Basics

Filesystem

  • /bin - This is where basic Linux commands reside (ls, du, dd, cp, etc).
  • /boot - Your boot images are stored here.
  • /dev - Links to access your machine's devices.
  • /etc - Configuration files and boot scripts.
  • /home - User directories, equivalent to "Documents and Settings" in Windows XP.
  • /lib - System libraries, codecs, etc.. similar to Windows/System and Windows/System32.
  • /mnt, /media - Mount points. A mount point is a directory that the contents of your hard drives, cd/dvd drives, floppy drives, or jump drives will be accessible.
  • /opt - Optional packages and programs. Could be thought of as a "Program Files" directory.
  • /proc - Special dynamic information about your system.
  • /root - System administrator home. Could be thought of as a "Documents and Settings/Administrator" directory.
  • /sbin - Super-user binaries. These programs need super-user (root) privileges to execute.
  • /tmp - Temporary files. Every user usually has read, write, and execute permissions here.
  • /usr - The main place for programs to be installed. Most like "Program Files" in Windows.
  • /var - System logs, mail spools, default web server directory, databases, etc...

Basic Commands

  • cd - Change Directory: moves to a different directory.

    Usage: cd directory, cd .., cd /directory

  • cp - CoPy: Copy a file or directory. If you wish to copy recursively and retain all attributes associated with the file or directory, use the -a option.

    Usage: cp original original.backup, cp -a /home/user/directory /home/user/backup

  • df - Disk usage on Filesystem: Display an overall summary of disk usage on mounted mountpoints. If you want human-readable sizes, use the -h option.

    Usage: df, df -h, df /mnt/mountpoint

  • du - Disk Usage: Display the disk usage of each file (recursively, by default) in the current directory. If you want human-readable sizes (1024 bytes = 1Kb, 1024Kb = 1Mb, etc), use the -h option. If you want a summary of the total disk usage by a directory and everything inside, use the -s option.

    Usage: du, du -s, du -h, du -sh, du -s /directory

  • ln - LiNk: Create a link, or shortcut, to a file or directory. I prefer to do symlinks by using the -s option.

    Usage: ln original link, ln original /directory, ln original /directory/link, ln -s original /directory/link

  • ls - LiSt: lists the contents of a directory.

    Usage: ls, ls .., ls /directory/subdirectory

  • man - View the MANual page for a program or other file. Probably the most useful program ever.

    Usage: man program, man xorg.conf

  • mkdir - MaKe DIRectory: create a new directory/folder.

    Usage: mkdir dirname, mkdir /directory/newdirname

  • mv - MoVe: Move a file or directory to a new location, or rename a file or directory.

    Usage: mv file /directory/newhome, mv file newfilename

  • pwd - Print Working Directory: returns the full path of the directory in which you are working.

    Usage: pwd

  • rm - ReMove: Remove a file or directory. If you want to get rid of a directory and all of its contents, use rm -R or rm -Rf for recursive deletion.

    Usage: rm filename, rm /directory/filename, rm -Rf /directory/dirname

  • rmdir - ReMove DIRectory: remove a directory. The directory must be empty.

    Usage: rmdir dirname, rmdir /directory/dirname

  • whereis - Determine where a certain file exists (if it's in your path)

    Usage: whereis filename

  • whoami - Detemine which user you are currently logged in as

    Usage: whoami

Linux Permissions

Linux has a great permission scheme. Since its inception, three basic levels of security have existed: user, group, and everyone. A simple way to change the permissions on a file or directory is to use the chmod, or CHange MODe, command. Changes to the permissions can be either a symbolic representation or an octal number representing the bit pattern for the new permissions. I prefer the symbolic method, myself, but many others prefer to see the octal pattern.

When working with permissions in Linux, always remember the following orders: User, Group, All; Read, Write, Execute. Those are the orders you will put the permissions in. Let's say that we want to make a file readable and writable only to the owner, while no one else will even be able to read the file. Here are some examples:

NOTE: Commands that begin with $ are executed as a regular user. Commands that begin with # are executed by a superuser (root). These two symbols (when they are the very first character in the command) are not entered by the user.

Symbolic:

$ echo "Hi" >> testing
$ chmod a-rwx,u+rw testing

Octal:

$ echo "Hi" >> testing
$ chmod 600 testing

Let's now examine the commands individually.

$ echo "Hi" >> testing

This command will append "Hi" (without the quotes) to the end of the file called testing. The file will be created if it does not already exist, assuming that the user has write permissions in the current directory. If you didn't want to append, you could overwrite anything that may be in the file by using a single > rather than >>.

$ chmod a-rwx,u+rw testing

This command removes (the - in a-rwx) read (the r in a-rwx), write (the w in a-rwx), and execute (the x in a-rwx) permissions from all (the a in a-rwx) users on the file called testing. Next we add (the + in u+rw) permissions for the owner (the u in u+rw) of the file: read (the r in u+rw) and write (the w in u+rw) on the file called testing.

$ chmod 600 testing

This command sets the permissions for everyone in one shot. I think of the digits in binary:

  • 1 = execute only;
  • 2 = write only;
  • 3 = write and execute, but no read;
  • 4 = read, but no write or execute;
  • 5 = read and execute, but no write;
  • 6 = read and write, but no execute;
  • 7 = read, write, and execute.

A digit is required for each level of permissions (user, group, and all). It is also possible to put another digit before the 3 levels of permissions, but to be honest, I don't know what significance it has. A little bit of testing has shown that it puts either an S or T in place of the execute permissions (depending on the digit).

A couple more things about chmod: Directories must also be executable in order to list the contents. chmod is very powerful. Finally, you can recursively apply permissions to directories and everything underneath with the -R option.

$ chmod -R a+rx /home/user/share

A couple of commands closely associated with chmod are chgrp (CHange GRouP) and chown (CHange OWNer).

chown will change the user ownership of files or directories. This can be done recursively with the -R option. It also has the capability to change the group ownership built into it. The syntax is: chown [options] user[:group] file1 [file...]

chgrp will change the group ownership of files or directories. You can do this recursively with the -R option. The syntax is: chgrp [options] groupname file1 [file...]

Cronjobs

Cronjobs are similar to scheduled tasks in the Windows world. Schedule tasks or cronjobs are simply programs that you want to run regularly, without having to type in the command every time you want it to run. Most distributions come with a cron daemon of some sort installed by default. Generally speaking, you can edit your cronjobs by typing crontab -e. This will bring up an editor like vi (it usually is vi by default) in which you edit your cronjob file. Each user can have their own cronjobs (unless it's been disabled by the administrator, I would assume). Here is an example of a cronjob entry:

47 * * * * /usr/bin/run-parts /etc/cron.hourly 1> /dev/null

You'll notice the 47 with four *'s after it. This is how the daemon knows when to execute a job. This is what each of the stars represent in their order:

  1. Minute: 0-59
  2. Hour: 0-23 (0 = midnight)
  3. Day of month: 1-31
  4. Month: 1-12
  5. Day of week: 0-6 (0 = Sunday)

So the example above will run after 47 minutes every hour of every day of the month of every month. You can also do some fancy things like having a job run every 5 minutes, after 15 and 45 minutes, etc. Let's say that we want to grab our mail every 5 minutes. The cronjob entry would look something like:

*/5 * * * * /usr/bin/fetchmail

If we wanted to grab our mail every 2 hours but only on Mondays, we would use the following:

0 */2 * * 1 /usr/bin/fetchmail

To have a job run after 15 and 45 minutes, we could do this:

15,45 * * * * /usr/bin/fetchmail

Pretty nifty, eh?

Make

This fancy utility is usually the means used for compiling programs from source. The usual sequence of commands for compiling and installing a program from the source in Linux is as follows:

$ ./configure
$ make
# make install

Most packages will follow this convention, but some require special procedures. Sometimes you can even get away with skipping the make and jump straight from ./configure to make install. It is always a good idea to read the README and INSTALL files included in source packages. They will generally tell you about anything out of the ordinary when compiling the source. Obviously, there is a lot more to this utility, but I'm not the person to explain it to you.

Package Management

There are several different types of package managers. The most popular these days are .rpm (RedHat Package Manager) and .deb (Debian). There are some other kinds of packages, but they aren't as popular as RPM and DEB. Slackware uses a straight .tgz (gzipped tarball) as its package system. Frugalware uses .fpm, which are bzipped tarballs. In the end, packages are almost always gzipped or bzipped tarballs.

Each package system has its ups and downs. I've personally found RPM-based distributions to be overly slow, especially with the package management. DEB-based distributions seem to be a lot more speedy when put up against RPM-based distros. However, I have found Slackware's TGZ-based system the most efficient and the fastest. Both RPM's and DEB's have dependency checking. In other words, the package manager will attempt to locate all entities upon which a program may depend in order to function properly before installing or upgrading that program.

A lot of people claim that .tgz packages are inferior to RPM and DEB because of the lack of dependency checking. By default, Slackware does not have dependency checking, but if you know what you're doing, you can get your dependencies a lot easier than you can with RPM or DEB (in my opinion).

RPM packages usually seem quite large compared to other package systems like DEB and TGZ. As far as I have seen, TGZ packages are smaller than both RPM and DEB packages. Here are a few options to help you use the RPM and TGZ package managers. I am not sure on Debian packages, so I won't attempt that one:

  • RPM:
    • rpm -q or rpm --query: look for a package on your system
    • rpm -i or rpm --install: install a new package on your system
    • rpm -U or rpm --upgrade: upgrade a package which is already installed on your system
    • rpm -e or rpm --erase: remove a package from your system
  • TGZ:
    • pkgtool: a text-based package manager
    • installpkg: install a package onto your system
    • upgradepkg: upgrade a package which is currently installed on your system
    • removepkg: remove a package from your system

You can also get some other programs that are VERY useful for package management. I think the latest craze for RPM's is YUM. I have not had great luck with this utility, but a lot of people really like it. Debian packages have used apt-get for ages now. My favorite add-on for Slackware packages is called swaret. Other distributions use the pacman utility, which is very efficient. Each one of these applications has several options and operation procedures.

Secure Shell and Secure Copy

One of my favorite aspects of Linux and other UNIX-derived systems is their secure shell capability (which is usually installed by default). Secure shell, or SSH, is a way for users to log into a remote computer and work on the remote computer as though it were right in front of them. Granted, it's all text-based unless you have X11 forwarding setup properly on both machines. But the command line interface (CLI) is extremely powerful--you should not be afraid to learn and use it. If you need to SSH from a Windows machine, you can use PuTTY.

In order to ssh into another computer, you simply type:

$ ssh hostname

or use the computer's IP address:

$ ssh xxx.xxx.xxx.xxx

By default, ssh on Linux machines will use the username of the account that you are running ssh from. Sometimes you need to login as a different user than the one you're currently using. To do that, you use the -l (lowercase L) or make it look like an e-mail address:

$ ssh hostname -l differentuser
$ ssh differentuser@hostname

Once your ssh session begins with the remote host, you will be asked to enter the password associated with the account with which you are attempting to login. If you do a lot of ssh'ing between machines, typing in your password several times is not only annoying but it could also pose a security risk--some wandering eyes might be watching you each time you enter your password. A great way to get around this is to generate a public and private key for your account. Once you do this, you can use the private key file on the machine you're ssh'ing out of and the public key on the remote machine.

To generate a public/private key, you can use ssh-keygen:

$ ssh-keygen -t rsa

You will be asked to enter and verify a passphrase for your private key. If you were aiming to get around not typing in your password, just hit enter twice for this part. It's still not secure, but it is a lot less hassle if you're only working on machines that no one else has "access" to. Usually your keys will be stored in ~/.ssh/ (or your /home/username/.ssh/ directory: ~ refers to your /home/yourusername).

The next step is to create your identification:

$ cd ~/.ssh
$ echo "IdKey private_key_file" > identification

Now you have to copy your public key (usually the one that ends in .pub) to the remote host:

$ scp public_key_file.pub username@xxx.xxx.xxx.xxx:/home/username/.ssh

And finally you should add your public key to the list of authorized users on the remote host by adding a line like the following to the ~/.ssh/authorization file:

Key public_key_file.pub

At this point you should be able to log into your remote host without your password (assuming you skipped the passphrase part of the key generation above).

As for the secure copy utility, you can get an idea of how to use it from the scp command above. This program uses the SSH system to securely copy files between two computers. This is how I use the scp command:

$ scp [-r] user@remote:/path/to/remote/file /local/destination/path/
$ scp [-r] /path/to/local/file user@remote:/remote/destination/path/

If you have setup public key authorization, you will not have to enter your password each time you use scp. Otherwise, you are asked for a password each time you run scp.

Archiving and Backup

There are many different kinds of compression and archiving tools in Linux. The most common types are tarballs, gzipped, and bzipped files. Below is a list of purposes for each of the three and some of the options:

  • tar - multiple files, little or no compression
    • c, --create - create a tarball
    • f, --file - specify the tarball's filename
    • x, --extract, --get - extract the contents of a tarball
    • j, --bzip2 - use bzip2 compression/decompression
    • v, --verbose - show verbose output
    • z, --gzip, --ungzip - use gzip compression/decompression
    • to create a tarball called filename.tar which contains all of the files in /dir/to/archive: $ tar cf filename.tar /dir/to/archive
    • to create a tarball called filename.tar.gz which contains all of the files in /dir/to/archive and gzip it: $ tar zcf filename.tar.gz /dir/to/archive
    • to create a tarball called filename.tar.bz2 which contains all of the files in /dir/to/archive and bzip it: $ tar jcf filename.tar.bz2 /dir/to/archive
    • to extract the contents of a tarball called filename.tar.gz to the current directory: $ tar zxf filename.tar.gz
    • to extract the contents of a tarball called filename.tar.bz2 to the current directory: $ tar jxf filename.tar.bz2
  • gzip - single file compression
    • To gzip a file called filename to make it filename.gz: $ gzip filename
    • To gunzip a file called filename.gz to make it filename: $ gunzip filename.gz
  • bzip2 - single file compression
    • To bzip a file called filename to make it filename.bz2: $ bzip2 filename
    • To bunzip a file called filename.bz2 to make it filename: $ bunzip2 filename.bz2