Using Django to Design Your Database Schema

Josh VanderLinden

Comments

Last night I had a buddy of mine ask me how I would approach a particular database design problem. I get similar questions quite often from my peers--suggests there is something important lacking from the database classes out there. Instead of answering him directly, I decided to come up with this tutorial for using Django to design your database schema.

For those of you new to Django, this article might seem a bit advanced. In time I will have more introductory-level Django tutorials, but I hope this one is easy enough.

Create a Django Project

The first step is to create a Django project. If you already have a project that you can play with, you can skip this step. To create a project, go to a place where you want to keep your code (like C:\projects or /home/me/projects) in a command prompt/terminal and run the following command:

django-admin.py startproject myproject

This will create a new directory in your current location called myproject (you can replace myproject with whatever you'd like so long as you're consistent). This new directory will contain a few files:

  • __init__.py
  • manage.py
  • settings.py
  • urls.py

If you get an error message when running the above command, you might not have Django installed properly. See Step-by-Step: Installing Django for details on installing Django.

Create An Application

Once you have a Django project setup, you should create a new application.

Note: If you're using Windows, you will probably need to omit the ./ on the ./manage.py commands. I will include them here for everyone else who's using Linux or a Mac.

cd myproject
./manage.py startapp specialapp

This will create a new directory in your myproject directory. This new directory will contain three files: __init__.py, models.py, and views.py. We are only concerned with the models.py file in this article.

Create Your Models

Models are usually a direct representation of what your database will be. Django makes creating these models extremely easy, and Python's syntax makes them quite readable. The Django framework asks for models to be defined in the models.py file that was created in the last step. Here's an example (for my buddy who prompted the creation of this article):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
 
from django.db import models

class Component(models.Model):
    item_number = models.CharField(max_length=20)
    name = models.CharField(max_length=50)
    size = models.CharField(max_length=10)
    quantity = models.IntegerField(default=1)
    price = models.DecimalField(max_digits=8, decimal_places=2)

class Project(models.Model):
    name = models.CharField(max_length=50)
    components = models.ManyToManyField(Component)
    instructions = models.TextField()

(for more information about models, see the Django Model API Reference)

I don't know about you, but that code seems pretty straightforward to me. I'll spare you all the details about what's going on (that can be a future article).

Install Your New Application

Once you have your models setup, we need to add our specialapp to our list of INSTALLED_APPS in order for Django to register these models. To do that, open up settings.py in your myproject directory, go to the bottom of the file, until you see something like

1
2
3
4
5
6
 
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
)

When you find that, add your specialapp to the list

1
2
3
4
5
6
7
 
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'specialapp'
)

Setup Your Database

Now you need to let Django know what kind of database you're using. Django currently supports MySQL, SQLite3, PostgreSQL, and Oracle natively, but you can get third-party tools that allow you to use other database (like SQL Server).

Still in your settings.py, go to the top until you see DATABASE_ENGINE and DATABASE_NAME. Set that to whatever type of database you are using:

1
2
 
DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = 'myproject.db'

Save your settings.py and go back to your command prompt/terminal.

Get Django's Opinion For Your Schema

Make sure you're in your myproject directory and run the following command:

./manage.py sqlall specialapp

This command will examine the models that we created previously and will generate the appropriate SQL to create the tables for your particular database. For SQLite, we get something like this for output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
 
BEGIN;
CREATE TABLE "specialapp_component" (
      "id" integer NOT NULL PRIMARY KEY,
      "item_number" varchar(20) NOT NULL,
      "name" varchar(50) NOT NULL,
      "size" varchar(10) NOT NULL,
      "quantity" integer NOT NULL,
      "price" decimal NOT NULL
)
;
CREATE TABLE "specialapp_project" (
      "id" integer NOT NULL PRIMARY KEY,
      "name" varchar(50) NOT NULL,
      "instructions" text NOT NULL
)
;
CREATE TABLE "specialapp_project_components" (
      "id" integer NOT NULL PRIMARY KEY,
      "project_id" integer NOT NULL REFERENCES "specialapp_project" ("id"),
      "component_id" integer NOT NULL REFERENCES "specialapp_component" ("id"),
      UNIQUE ("project_id", "component_id")
)
;
COMMIT;

Notice how Django does all sorts of nifty things, like wrapping the table creation queries in a transaction, setting up indexes, unique keys, and defining relationships between tables. The output also offers a solution to the original problem my buddy had: an intermediate table that just keeps track of relationships between projects and components (the specialapp_project_components table).

Notice that the SQL above may not work with database servers other than SQLite.

Enhancing The Intermediate Table

After my buddy reviewed this article, he asked a very interesting and valid question: What if a project needs 3 of one component? In response, I offer the following models (this requires a modern version of Django--it doesn't work on Django 0.96.1 or earlier):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
 
from django.db import models

class Component(models.Model):
    item_number = models.CharField(max_length=20)
    name = models.CharField(max_length=50)
    size = models.CharField(max_length=10)
    quantity = models.IntegerField(default=1)
    price = models.DecimalField(max_digits=8, decimal_places=2)

class Project(models.Model):
    name = models.CharField(max_length=50)
    components = models.ManyToManyField(Component, through='ProjectComponent')
    instructions = models.TextField()

class ProjectComponent(models.Model):
    project = models.ForeignKey(Project)
    component = models.ForeignKey(Component)
    quantity = models.PositiveIntegerField()

    class Meta:
        unique_together = ['project', 'component']

Running ./manage.py sqlall specialapp now generates the following SQL:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
 
BEGIN;
CREATE TABLE "specialapp_component" (
    "id" integer NOT NULL PRIMARY KEY,
    "item_number" varchar(20) NOT NULL,
    "name" varchar(50) NOT NULL,
    "size" varchar(10) NOT NULL,
    "quantity" integer NOT NULL,
    "price" decimal NOT NULL
)
;
CREATE TABLE "specialapp_project" (
    "id" integer NOT NULL PRIMARY KEY,
    "name" varchar(50) NOT NULL,
    "instructions" text NOT NULL
)
;
CREATE TABLE "specialapp_projectcomponent" (
    "id" integer NOT NULL PRIMARY KEY,
    "project_id" integer NOT NULL REFERENCES "specialapp_project" ("id"),
    "component_id" integer NOT NULL REFERENCES "specialapp_component" ("id"),
    "quantity" integer unsigned NOT NULL,
    UNIQUE ("project_id", "component_id")
)
;
CREATE INDEX "specialapp_projectcomponent_project_id" ON "specialapp_projectcomponent" ("project_id");
CREATE INDEX "specialapp_projectcomponent_component_id" ON "specialapp_projectcomponent" ("component_id");
COMMIT;

As you can see, most of the SQL is the same. The main difference is that the specialapp_project_components table has become specialapp_projectcomponent and it now has a quantity column. This can be used to keep track of the quantity of each component that a project requires. You can add however many fields you want to this new intermediate table's model.

Using This SQL

There are several ways you can use the SQL generated by Django. If you want to make your life really easy, you can have Django create the tables for you directly. Assuming that you have specified all of the appropriate database information in your settings.py file, you can simply run the following command:

./manage.py syncdb

This will execute the queries generated earlier directly on your database, creating the tables (if they don't already exist). Please note that this command currently will not update your schema if the table exists but is missing a column or two. You must either do that manually or drop the table in question and then execute the syncdb command.

Another option, if you want to keep your DDL(Data Definition Language) in a separate script (maybe if you want to keep it in some sort of version control) is something like:

./manage.py sqlall specialapp > specialapp-ddl-080813.sql

This just puts the output of the sqlall command into a file called specialapp-ddl-080813.sql for later use.

Benefits of Using Django To Create Your Schema

  • Simple: I personally find the syntax of Django models to be very simple and direct. There is a comprehensive API that explains and demonstrates what Django models are capable of.
  • Fast: Being that the syntax is so simple, I find that it makes designing and defining your schema much faster than trying to do it with raw SQL or using a database administration GUI.
  • Understandable: Looking at the model code in Django is not nearly as intimidating as similar solutions in other frameworks (think about Java Persistence API models).
  • Intelligent: Using the same model code, Django can generate proper Data Definition Language SQL for several popular database servers. It handles indexes, keys, relationships, transactions, etc. and can tell the difference between server types.

Downfalls of Using Django To Create Your Schema

  • The Table Prefix: Notice how all of the tables in the SQL above were prefixed with specialapp_. That's Django's safe way of making sure models from different applications in the same Django project do not interfere with each other. However, if you don't plan on using Django for your end project, the prefix could be a major annoyance. There are a couple solutions:
    • A simple "search and replace" before executing the SQL in your database
    • Define the db_table option in your models
  • Another Technology: Django (or even Python) may or may not be in your organization's current development stack. If it's not, using the methods described in this article would just become one more thing to support.

Other Thoughts

I first thought about doing the things mentioned in this application when I was working on a personal Java application. I like to use JPA when developing database-backed applications in Java because it abstracts away a lot of the database operations. However, I don't like coming up with the model classes directly, so I usually reverse engineer them from existing database tables.

Before thinking about the things discussed in this article, I created the tables by hand, making several modifications to the schema before I was satisfied with my JPA models. This proved to be quite bothersome and time-consuming.

After using Django to develop my tables, the JPA models turned out to be a lot more reliable, and they were usually designed properly from the get-go. I haven't created tables manually ever since.

If you find yourself designing database schemas often, and you find that you have to make several changes to your tables before you/the project requirements are satisfied, you might consider using Django to do the grunt work. It's worked for me, and I'm sure it will work for you too.

Good luck!

Step-by-Step: Installing Django

Josh VanderLinden

Comments

Being the Django and Python zealot that I am, I often find myself trying to convert those around me to this awesome development paradigm. Once I break them, these people often ask me a lot of questions about Django. Over the past few months I've noticed that one of the biggest sticking points for people who are new to Django is actually getting it up and running to begin with. In response, this is the first in a series of articles dedicated to getting Django up and running.

What is Django?

The Django Web site describes Django as "a high-level Python Web framework that encourages rapid development and clean, pragmatic design." Basically, Django is just about the most amazing thing for Web development. I have tinkered with several different Web technologies, but nothing seems to even come close to what Django can do for me.

What is Python?

Python is a programming language used in numerous aspects of computing these days. It has a very simple yet powerful syntax. It's an easy language for beginners to pick up, but it provides adequate levels of power for the more experienced developers out there. If you have never programmed anything before, or you have dabbled with something like BASIC, Python should be fairly straightforward. If you are a programming veteran, but have only worked with languages like C, C++, Java, etc, you might struggle a bit with the syntax of the language. It's not difficult to overcome the differences in a couple hours of hands-on development.

Let's get started.

Installing Python...

Having Python installed is critical--Django does not work without Python. I'm guessing that you're relatively familiar with the procedures for installing software packages on your particular operating system. However, I will share a few notes to point you in the proper direction if you're lost. If nothing else, just head over to the Python download page to download anything you need to install Python. I whole-heartedly recommend using the latest stable version of Python for Django, but you should be able to get by with as early a version as 2.3.

...On Windows

Simply grab the latest version of the Python installer. It is currently version 2.5.2. Once the installer has downloaded successfully, just run through the installation wizard like any other setup program.

...On Mac OS X

Recent Mac OS X computers come with Python pre-installed. To determine whether or not you actually have it, launch the Terminal (Applications > Utilities > Terminal) and type python -c "import sys; print sys.version". If Python is already installed, you will see the version you have installed. If you have a version that is less than 2.3, you should download the newest version. If you don't have Python installed, you will get a "command not found" error. If you're in this boat, just download the latest version of the Python Universal installer and install it.

...On Linux

Most Linux distributions also have Python pre-installed. Just like with Mac OS X, you can check to see by opening up a terminal/konsole session and running the command python -c "import sys; print sys.version". If you have Python installed, you will see its version. If you get an error message when running that command, or you have a version earlier than 2.3, you need to download and install the latest version of Python.

If you're running a Debian-based distribution (like Ubuntu, sidux, MEPIS, KNOPPIX, etc), you can probably use sudo apt-get install python to get Python. If you're running an RPM-based Distribution, you can probably use something like Yum or YaST to install Python.

A sure-fire way to install Python on any Linux system, however, is to install from source. If you need to do this, you simply:

  1. download the source for the latest version of Python
  2. extract it: tar jxf Python-2.5.2.tar.bz2
  3. go into the newly-extracted directory: cd python-2.5.2
  4. configure it: ./configure
  5. compile it: make
  6. install it: make install

(I've only installed Python from source one time, so I might be wrong)

Setting Up Your PYTHONPATH...

Generally speaking, if you didn't have Python installed before starting this tutorial, you will need to setup your PYTHONPATH environment variable. This is a variable that lets Python know where to find useful things (like Django).

...On Windows

  • Open up your System Properties (Win+Break or right click on "My Computer" on your desktop and select Properties)
  • Go to the "Advanced" tab
  • Click the "Environment Variables" button
  • If you have permission to change system variables, click the "New" button in the bottom pane. Otherwise, create the PYTHONPATH variable for your user account using the "New" button in the top (User variables for [username]) pane.
  • Set the variable name to PYTHONPATH
  • Set the variable value to C:\Python25\Lib\site-packages (replace C:\Python25\ with whatever it is on your system if needed)
  • Save it

You may also need to add the python executable to your PATH. If you can successfully run python from a command prompt window, you don't need to worry about it.

If you can't run python from a command prompt, follow the procedure above, but use the PATH variable instead of PYTHONPATH. PATH most likely already exists, so you just need to append/prepend the existing value with something like C:\Python25\ (again, this might need to change depending on where you installed Python)

...On Mac OS X

Your PYTHONPATH should already be setup for you.

...On Linux

Usually you just need to edit your ~/.bash_rc script to setup your PYTHONPATH environment variable. Go ahead and open that up in your preferred text editor and make sure there's something in it like:

export PYTHONPATH=/usr/lib/python2.5/site-packages:$PYTHONPATH

Save any changes necessary and run the following command:

source ~/.bash_rc

This will take care of updating your current session with any changes you made to your ~/.bash_rc.

Installing Django

Once you have Python and have verified that you have version 2.3 or later, you are ready to install Django. Currently, the latest stable release is 0.96.1, but this is grossly out-dated. Django 1.0 will be released on September 2nd 2008, so the "unstable" copy of Django is pretty close to what 1.0 will have to offer. There are some incredibly useful improvements in the unstable version that I don't think I could do without anymore, so that's what I'll talk about installing here.

First, you need to have a subversion client. On Windows, the most popular one is called TortoiseSVN. On Mac OS X, I have played with a few, but I think Versions is a pretty decent one. Linux also has several to choose from, but if you're using Linux, you're probably going to use the command line anyway (right?).

For brevity, I will just use the subversion commands necessary to accomplish this task (instead of discussing all GUI interfaces to subversion).

The exact location that Django should be installed differs from system to system, but here are some guidelines for typical setups:

  • Windows: C:\Python25\Lib\site-packages
  • Linux: /usr/lib/python2.5/site-packages
  • Mac OS X: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages

If you want a definite location, run the following command:

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Once you know that location, go there in your command prompt or terminal session. Then execute this command svn co http://code.djangoproject.com/svn/django/trunk/django django. You will see loads of output, showing all of the files that you are downloading as you install Django.

As soon as that process completes, you should run python -c "import django" to make sure everything worked properly. If the command doesn't display an ImportError, you're good. Otherwise, you need to try again.

Getting Access to Django Scripts...

Once you can successfully import django, you might want to make sure you can run the django-admin.py script that comes with Django.

...On Windows

This process is very similar to what we did with the PYTHONPATH environment variable earlier.

  • Open your System Properties again
  • Go to the Advanced tab
  • Click the Environment Variables button
  • Find your PATH environment variable (either for your user or system-wide)
  • Make sure that the variable value contains something like C:\Python25\Lib\site-packages\django\bin
  • Save any changes
  • Open a fresh command prompt
  • Try to run django-admin.py. If you're successful, you're ready to get started with Django. Otherwise, you need to fix your path to django/bin or just call the django-admin.py script using an absolute path when needed.

...On Mac OS X

You can run a command similar to this:

sudo ln -s /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/bin/django-admin.py /usr/local/bin

...On Linux

If you have "root" privileges on your Linux system, you can execute a command like:

sudo ln -s /usr/lib/python2.5/site-packages/django/bin/django-admin.py /usr/local/bin

If you don't have "root" privileges, you can setup your own /usr/local/bin:

mkdir ~/bin

Make sure your ~/.bash_rc contains something like:

export PATH=$HOME/bin:$PATH

Then update your current session with any changes you made to ~/.bash_rc by running this command:

source ~/.bash_rc

And that should do it! Now you should be ready to get started with Django.

Feel free to leave a comment if you're having problems installing Django. Good luck!

Check out Installing Django on Shared Hosting.

How To Compile and Install a 2.6.x Series Linux Kernel

Josh VanderLinden

Comments

The Linux kernel is the core component in any Linux distribution. Without a kernel, your computer would be essentially useless. It is the piece of software which allows interaction between you, your computer's applications, and your computer's hardware. With such a powerful role in your computing experience, it is important to keep your kernel up-to-date. Each new release provides more hardware support and many performance enhancements. It is also important to keep your kernel up-to-date for security purposes.

Let's upgrade our Linux kernels together. I will walk you through each of the steps I take, from beginning to end, to upgrade my kernel. Just as a warning, I prefer to do the whole process on the command line, so you might want to pull up a terminal, konsole, xterm or whatever you prefer to use for your command line operations.

First you need to download the kernel source code. Many Linux distributions provide specialized editions of the Linux kernel. Typically, you don't want to manually compile and install a custom kernel for these distributions. This does not mean that you can't, it simply means that you might be better off using the "official" kernels for your distribution, which can usually be obtained through your distribution's package manager. You can get the official, 100% free, and complete Linux kernel source code from http://www.kernel.org/. Look for "The latest stable version of the Linux kernel is:" and click the link on the F on the same line. Currently, the latest stable version is 2.6.20, and that's what I'll be using for this tutorial. Please note that commands which begin with a dollar sign ($) are executed as a regular user and commands beginning with a pound sign (#) are executed as a superuser.

$ cd /home/user/download
$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.20.tar.bz2

Now login as the superuser, and navigate to the /usr/src directory. Then extract the kernel source into that directory.

$ su -
# cd /usr/src
# tar jxf /home/user/download/linux-2.6.20.tar.bz2

You probably already have a symlink or shortcut called linux which points to your most recent kernel. If you do, delete the link and create another link to the new source tree. Then go into your kernel source tree.

# rm /usr/src/linux
# ln -s /usr/src/linux-2.6.20 /usr/src/linux
# cd /usr/src/linux

I like to identify each compile of my kernel uniquely, to make sure that I'm using the right one. To do that, you have to modify your Makefile

# vi Makefile

You will see the following lines, or something similar, at the very top of the file:

VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 20
EXTRAVERSION =
NAME = Homicidal Dwarf Hamster

Change the EXTRAVERSION property to something you want to use to identify this kernel. I will use -jcv1

EXTRAVERSION = -jcv1

The rest of the Makefile should be fine. In fact, I discourage editing Makefiles unless you know what you're doing. This next step is totally optional, but I like to do it to save some time. You can copy your existing kernel's configuration file in order to have a very similar kernel configuration. My previous kernel version was 2.6.19.1, so this is the command I use:

# cp /usr/src/linux-2.6.19.1/.config /usr/src/linux/

Then I run make oldconfig or make silentoldconfig to update my older kernel configuration file to be able to handle newer features. If you use oldconfig you are required to specify whether or not you want the new features included in your kernel, whereas silentoldconfig will use the defaults determined by kernel developers (they usually know best), asking for minimal input. Let's update our configuration file and then customize it by running make menuconfig (there are several options here, such as make xconfig and make gconfig, but I prefer the text-based menuconfig; there is another you can run by using make config, which runs through each and every option available--it's scary).

# make silentoldconfig
# make menuconfig

menuconfig is a graphical command line application which lets you navigate the features offered by the kernel. Each computer is considerably different from the next, so it really does no good to provide a list of things that I tweak. However, it is important to note what some of the symbols are in the menuconfig utility:

  • M = Module. Modules are loaded when they are required and can contribute to the speed of your system
  • * = built into the kernel. These are typically things which are necessary for your machine to function properly, such as support for your root file system.
  • X = exclusively selected. You'll see this when you select what type of processor you have, for example.

One thing to note before we go further is MAKE SURE YOU KERNEL HAS BUILT-IN SUPPORT FOR YOUR ROOT FILE SYSTEM!!!! My root file system is reiserfs. In my configuration, I made sure that reiserfs was marked with a star. If you don't do this, your kernel won't boot and you will be very frustrated. Trust me.

Your computer is probably quite different than mine, so you might want to just poke around and see if you recognize things that deal with your computer's hardware. Once you are done tweaking your kernel configuration, exit the configuration utility and make sure the configuration is stored in /usr/src/linux/.config

Next we get to build and install the kernel. After that, we have to add an entry to our boot manager so that we can try out our new kernel. The compilation part usually takes just about a half hour on my 2.2Ghz Turion64 processor with 1.25GB of RAM. It takes about 6 hours on my 300Mhz Pentium 2 with 32MB of RAM. Let's find out how long it takes for you to compile your kernel!

# time make
...
real    27m29.663s
user    23m34.476s
sys     2m56.575s

Now let's install the modules and install the appropriate files in the boot area:

# make modules_install
# make install

This is the part that always used to mess me up. I use Slackware Linux, which is more UNIX-ish than most distributions. It's actually the oldest surviving Linux distribution to date, but that's another story. For some reason, the make install command doesn't always work with Slackware. There is a process I use to setup my boot directory when I compile a new kernel. I wrote a simple shell script called fixkernelinstall to take care of it for me:

#!/bin/bash
# Configure my computer for a new kernel
# Author: Josh VanderLinden
# Assisted By: Dan Purcell

# if the user didn't supply a kernel number, ask for it
if [ $# -eq 0 ]; then
    echo -n "Kernel: "
    read kernel
else
    kernel=$1
fi

# determine root partition
echo "Determining root partition..."
rootpart=`mount -l | grep ' / ' | cut -f 1 -d\ `
echo "Root partition is $rootpart"

# copy kernel configuration file
cp /usr/src/linux/.config ./config-$kernel

# now rename everything
echo "Renaming files..."
mv System.map System.map-$kernel
mv vmlinuz vmlinuz-$kernel

# if the config file exists and it's a symlink, remove it
if [ -f 'config' -a `stat config | grep -c 'symbolic link'` = '1' ]; then
    echo "Removing link to configuration file"
    rm config
else
    # otherwise it might be important
    echo "Renaming configuration file"
    mv config config.bak
fi

# Link files
echo "Creating symlinks..."
ln -s System.map-$kernel System.map
ln -s config-$kernel config
ln -s vmlinuz-$kernel vmlinuz

# Update lilo
echo "Adding entry to /etc/lilo.conf for $kernel"
echo "image = /boot/vmlinuz-$kernel" >> /etc/lilo.conf
echo "  root = $rootpart" >> /etc/lilo.conf
echo "  label = $kernel" >> /etc/lilo.conf
echo "  read-only" >> /etc/lilo.conf
echo "Linux kernel $kernel has been configured."
echo "Please check your lilo configuration and run lilo before rebooting"

I'm not an expert on shell scripts, so please feel free to offer suggestions for doing things better if you know how. This script uses the kernel version (given by the user) to setup by /boot directory properly. In my case, I run the script as such

# cd /boot
# fixkernelinstall 2.6.20-jcv1

And the output is something like:

Determining root partition...
Root partition is /dev/hda5
Renaming files...
Renaming configuration file
Creating symlinks...
Adding entry to /etc/lilo.conf for 2.6.20-jcv1
Linux kernel 2.6.20-jcv1 has been configured.
Please check your lilo configuration and run lilo before rebooting

As you can see from the script, I use LILO instead of the arguably more popular GRUB. Either one works for me, but LILO is sufficient for my needs. If you want to use the same kind of script for a GRUB installation, just change the LILO part at the end to something like:

echo 'Adding entry to /boot/grub/menu.lst for $kernel'
echo '  title Linux on ($rootpart)' >> /boot/grub/menu.lst
echo '  root (hd0,4)' >> /boot/grub/menu.lst
echo '  kernel /boot/vmlinuz-$kernel root=$rootpart ro vga=normal' >> /boot/grub/menu.lst

Make sure you change the line with root (hd0,4) to fit your setup. With GRUB, you don't have to worry about applying changes to see the menu entry at boot. It's automatically there. With LILO, however, you have to actually apply changes each time you make them. You do this by running the lilo command as the superuser:

# lilo
Added Windows
Added Linux
Added 2.6.20-jcv1 *

The star (*) signifies the default kernel to boot. Make sure that your root partition is correctly specified in your boot loader configuration. My root partition is on /dev/hda5, but yours may be (and probably is) on a different partition. If you fail to specify the correct root partition, your system will not boot that kernel until the configuration is fixed. GRUB makes this a lot easier than LILO.

And this is the point when you start to cross your figures and hope that your computer doesn't blow up... We get to reboot our computer and hope that our configuration file plays well with our computer. So, let's do that! See you in a few minutes (hopefully).

# shutdown -r now

So here I am, back on Linux on my freshly-rolled kernel. I hope you are as successful as I have been this time around. Keep in mind that you have to reinstall custom kernel modules if you installed others while you were on your other kernel. For example, I use ndiswrapper to access wireless Internet. I have to recompile and reinstall the ndiswrapper module and device drivers before I can use wireless. Likewise, I have VMWare Server on my laptop, which installed special modules. I have to run vmware-config.pl to reconfigure VMWare Server for my new kernel before I can run any virtual machines.

To summarize, here are the commands that I used in this tutorial. Remember that lines beginning with a dollar sign ($) are executed as a non-privileged user, while lines beginning with the pound sign (#) are executed as the superuser (root).

$ cd /home/user/download
$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.20.tar.bz2
$ su -
# cd /usr/src
# tar jxf /home/user/download/linux-2.6.20.tar.bz2
# rm /usr/src/linux
# ln -s /usr/src/linux-2.6.20 /usr/src/linux
# cd /usr/src/linux
# make clean
# vi Makefile (to change EXTRAVERSION to -jcv1)
# cp ../linux-2.6.19.1/.config .
# make silentoldconfig
# make menuconfig (just to ensure settings were good)
# time make
# make modules_install
# make install
# cd /boot
# fixkernelinstall 2.6.20-jcv1
# vi /etc/lilo.conf (to make sure things were good)
# lilo
# shutdown -r now

I hope that you are able to use this tutorial to successfully install or upgrade your kernel. Good luck! Any comments or suggestions are welcome!

Use Your Linux System With A Broken Bootloader

Josh VanderLinden

Comments

Here are some simple steps that should allow you to make some very simple changes that could end up saving you a lot of time and possibly a reinstall of your Linux system.

  1. Acquire a Linux liveCD of your choice. I prefer SLAX, as it is very small and quick. There are countless others such as KNOPPIX, ZenWalk, Ubuntu, Fedora Live, and DSL. Each of these would easily be up to the task.
  2. Boot the live CD and get to a command line somehow. If you are in a GUI environment, this means starting a Terminal or Konsole session. If you are booted into a classic Linux login prompt, you're good to go.
  3. Make sure you're the root user (you can do this by typing whoami). If you're not root, type su - and enter the root password when prompted.
  4. Mount your linux root partition. These steps may vary depending on preference, but they should be fairly similar:
    1. cd /mnt
    2. mkdir myroot
    3. mount /dev/hda5 /mnt/myroot
    4. cd /mnt/myroot
  5. Depending on the complexity of the tasks you wish to perform, you may need to mount your device list according to what your live CD detected:
    1. mount /dev /mnt/myroot/dev
    2. mount -t proc /proc /mnt/myroot/proc
  6. Finally, switch into your installed system, specifying where your root partition is mounted and they shell you wish to use:
    1. chroot /mnt/myroot /bin/bash

At this point you should be able to do several useful things, such as reinstall your bootloader into the MBR. You can also edit hosed configuration files this way.

If you need to add Windows into your boot configuration, and you use LILO, you could try adding something like this to your /etc/lilo.conf:

other = /dev/hda1
    label = Windows
    table = /dev/hda

(be sure to run lilo again to save the change)

When you are all done with your chroot environment, simply type exit or logout or what have you. Then be sure to unmount anything you may have mounted in order to use the system.

Wireless Networking With SuSE Linux Enterprise Desktop 10

Josh VanderLinden

Comments

Note: This tutorial is a continuation of yesterday's tutorial about installing SuSE Linux Enterprise Desktop 10 on my HP Pavilion dv8000. I may or may not refer to steps that I took during installation, so if you are confused, you might want to check out the previous article.

The process of installing and enabling a wireless adapter will vary greatly from machine to machine. Some lucky folks have wireless adapters that come with official Linux drivers. For the rest of us, we usually have a Broadcom-compatible adapter. In order to use a Broadcom device, I use a program called ndiswrapper, which basically takes the drivers for the devices to function with Windows and wraps them in such a manner that Linux can use. Since I have the 64-bit version of SuSE Linux Enterprise Desktop (SLED) 10, I need to get a 64-bit driver in order for my wireless to function properly. These 64-bit drivers took me a while to get ahold of the first time I got my wireless working (on SuSE Linux 10.1), but I still have them in my archives, so I should be fully prepared to get my wireless working. In this article I assume that you are going to use ndiswrapper to install drivers for a Broadcom device. So let's get started.

Install Ndiswrapper

First, make sure that you have ndiswrapper installed on your system. You can install it by entering YaST. In KDE, click the K menu (the little green chameleon in the bottom left), go to System > YaST (Administrator Settings). You will be asked to enter the root password, which you set during installation. Once you've done that, you will see the YaST Control Center, which is a very powerful set of tools and utilities that greatly ease the configuration and management of SLED. Click on the Software category on the left to show a list of software management options (if it's not already displayed). Click on the Software Management module.

Once loaded, you will see an interface which is very similar to what you would see during the expert package selection while installing SLED. Make sure your Filter (in the top left) is set to Search, and enter ndiswrapper in the search box. The search will return a few different results for ndiswrapper. The first result, ndiswrapper by itself, should be sufficient for most of us. When you check the box by ndiswrapper, you will see a warning informing you that ndiswrapper-based network are not officially supported by Novell. Just click OK to dismiss this warning.

Now you should be ready to install ndiswrapper. Click the Accept button in the bottom right. You will be asked to confirm the installation of ndiswrapper; click Continue. If your installation media is not still inserted, YaST will request the disc which contains the ndiswrapper packages. Insert the disc and click OK. In my case, two packages were installed. It may or may not differ for you.

As soon as the packages are done installing, your configuration settings are saved once again, and you will be asked if you want to install or remove more packages. Click No. At this point, ndiswrapper should be installed on your system, and you may dismiss the YaST Control Center.

Determine Your Wireless Adapter Make/Model

This step is absolutely necessary because if you install the wrong drivers, there is a chance (small as it may be) that your wireless adapter will be damaged. So let's ask Linux how our wireless adapter identifies itself. To do this, log into your SLED and open a Terminal or Konsole. On KDE, you can use the third button (a monitor with a black screen and > on it) on the menu panel at the bottom of the screen, or you can also click the "K" menu (same place as a regular start menu in Windows), go to System > Terminal > Konsole (Terminal Program). I am not exactly sure where this item is located with GNOME, but it might be under the System menu.

Once you have opened a terminal window of some sort, you must switch to a root user environment:

$ su -

You will then be asked for the root password, which you set during installation. Enter that password and type

# lspci

This command lists all of your PCI devices, according to the man pages, but you will see most if not all of your devices, PCI or otherwise, listed here. You'll notice that there is probably quite a list of devices. You may be interested in what your computer has in it, but since you're looking specifically for your wireless adapter, try one of the following commands

lspci | grep Broadcom
lspci | grep Wireless

The | after lspci will pipe the output of lspci to a useful and powerful program called grep. In this case, grep just looks for any lines that contain either the word Broadcom or Wireless. If you don't get any results from either of the two commands above, try to think of other keywords that might be used to identify a wireless adapter. My laptop returns the following:

# lspci | grep Broadcom
06:02.0 Network controller: Broadcom Corporation Dell Wireless 1470 DualBand WLAN (rev 02)

When you find the wireless adapter, pay attention to the numbers in front of it (06:02.0 on my laptop). With those numbers, you can get the information you need to find the right drivers for your particular wireless adapter. Enter the following command, substituting my device numbers with yours:

# lspci -n | grep 06:02.0
06:02.0 Class 0280: 14e4:4319 (rev 02)

This command gives you the wireless adapter's numeric ID; mine is 14e4:4319.

Download Your Device Drivers

Now that you know your device's numeric ID, you can go to the ndiswrapper wiki, which has a list of numeric IDs and the drivers that are known to work with that device. Look for your wireless adapter on the list of devices. I would recommend using your browser's search or find on page function to locate your device by the numeric ID that you just found.

I'll leave the retrieval of your device drivers up to you.

Install The Wireless Drivers

Most device drivers will come in an archive of some sort. Mine came in a RAR file. Extract your drivers to the directory of your choice--maybe something like ~/wireless. You can use the archive utility provided by SLED to extract your files. It functions very similar to WinZip, WinRAR, and other popular archive clients. By the way, the ~ in a directory listing refers to the current user's home directory (/home/user, for example).

Now, go back to the root terminal that you used to determine what kind of adapter you have. Navigate to the directory where you extracted your drivers and list the contents of the directory, looking for any *.inf files:

# cd ~/wireless
# ls

Ndiswrapper will use an INF file to know how it is supposed to install the driver. My INF file is called bcmwl5.inf. Now for the actual installation of the drivers:

# ndiswrapper -i bcmwl5.inf
Installing bcmwl5
Forcing parameter IBSSGMode|0 to IBSSGMode|2

Now check to make sure that the driver is there and that it recognizes your hardware:

# ndiswrapper -l
Installed drivers:
nbcmwl5          driver installed

Ooops!!! It doesn't recognize that my hardware is actually there. If you see 'driver installed, hardware present' then you should be good to go. You may proceed to the next step. However, if you have the same problem as me, you either have the wrong drivers or ndiswrapper installed the drivers improperly. This problem took forever to track down when I was first trying to get my wireless to work. Remember the numeric ID that you found earlier? Check this out:

# cd /etc/ndiswrapper/bcmwl5
# ls
14E4:4318.5.conf  bcmwl5.inf  bcmwl564.sys

Wait a second! Remember how my numeric ID was 14E4:4319? Why is there a listing for 14E4:4318.5? To solve this problem, I am just going to make a symlink (a shortcut) to 14E4:4318.5.conf and call it 14E4:4319.5.conf:

# ln -s 14E4:4318.5.conf 14E4:4319.5.conf

Now when I run the command to see if my hardware is recognized, I get this:

# ndiswrapper -l
Installed drivers:
bcmwl5          driver installed, hardware present

Hurray!! It says 'hardware present' in there!!! That means that the drivers are working and that my device can be used!

Enable Your Wireless Device

With ndiswrapper recognizing your wireless adapter, you can now enable it and start wirelessing your life away:

# modprobe ndiswrapper

There have been times when this particular step will lock up my machine and I have to do a hard reset, but most times it will work fine.

Connect to a Wireless Network

This part also gave me issues for a long time when I first installed my wireless drivers on SuSE Linux 10.1. I was able to connect to the wireless access points provided by my apartment complex, but I could not for the life of me connect to my own wireless router. Hopefully you don't encounter the same problem.

To see what access points you have available to you, check out the KNetworkManager applet in your system tray (next to the clock). I have 7 possible access points listed in the menu, including my encrypted router. When I clicked on my network, it asked me for my passphrase and connected immediately. Nice! That's definitely one plus for SLED over SuSE Linux 10.1!!

I am actually amazed at how easy it was to get my wireless working the second time around. Hopefully your wireless adapter installation was as painless as mine with the help of this guide.