Django for ISchoolers

This is a tutorial for how to build a question-and-answer type site using Django and various other technologies, plus some lessons in general web dev concepts and best practices along the way.

Chunk one

Prerequisites

Assumptions

This tutorial assumes you know how to access the command line/terminal on your computer and how to move around (cd) and view files (ls) in it. It also assumes you're reasonably familiar with Python, the Best Programming Language Ever. (Okay, editorializing slightly.)

This tutorial also tends to be oriented towards people with some familiarity with HTML, CSS, and JS (since most of you guys are IOLabbers).

Stuff to install

TODO: Actually write my own dang installation instructions.

Prior to starting this tutorial, please install Python, git, pip, virtualenv, Django, and South on your computer. There's instructions for all of those at http://pystar.org/setup_machine.html.

Setting up yer env

Once you've got pip and virtualenv installed, in your terminal, go into the folder you want to keep this tutorial code in and type (one line at a time):

$ virtualenv --no-site-packages django-tutorial-env

$ source django-tutorial-env/bin/activate

(django-tutorial-env)$ pip install django

(django-tutorial-env)$ pip install south

This will make and activate a virtual python environment for just this tutorial app. While this is a bit overkill if you only have one Django app at a time on your system, giving each Python app you work on its own environment is a good best practices just in case e.g. you're working on multiple apps, which use different versions of Django or other libraries, and you don't want them to conflict with each other.

To turn off the virtualenv (you can see it's active because the env name will appear in parentheses before your terminal prompt), type 'deactivate'. To reactivate the env, type the line that begins with 'source' again.

Spec'ing yer application

Your client, LulzTech, wants you to build them a Q & A website, called qandabear. ('Q and A bear', rhymes with panda bear.) For mysterious reasons, they don't want to use any of the open source Q&A website packages that are out there already; they want you to build one from scratch. Go figure.

There are whole classes on how to research and decide what to build, and how to state those requirements in a useful, clear way. (Bob's ISSD class, for one.) So we won't go into that. For our initial prototype for our client, then, our requirements are as follows:

• On the website, users can see the questions that have been asked and when they were posted
• Under each question, users can read the answers that have been given so far (if any) and when they were posted
• Users can submit a new question
• Users can submit a new answer to a question
• Users can edit an answer to a question
• Users can delete an answer to a question

Meeting these requirements will take us until the 'Bonus points' section. Once we get there, we can add more requirements.

Our prototype will have three pages: an index page of questions (and a form to ask a new question), a question page with the question and its answers (with a form to add a new answer), and a page to edit an answer to a question.

To support these pages, we will need two abstractions (objects, or models): Questions, and Answers.

Starting yer project

Activate your environment in your terminal

If you haven't already, run $ source django-tutorial-env/bin/activate in your terminal window.

Set up your git repository

If you're following this tutorial in class, in your terminal, run $ git clone git://github.com/aldeka/django-for-ischoolers.git. This won't download hardly anything interesting, but it will make it easy for you to sync up with the class later.

We'll be using the master branch (the default name for the main branch of code, which you're in right now) for syncing up throughout the class in case you get lost. You'll need to make your own branches for playing with the code on your computer, so that it doesn't interfere w. re-syncing later.

Think of branches in git as alternate timelines--you "branch" off of the main timeline of changes to the code into a separate timeline. You can try out a new feature or code setup in a branch, without overwriting or getting in the way of yourself or anybody else working on the code as-is. Sometimes a new idea doesn't work out, and you delete the branch or just let it sit around. Sometimes the new feature works, though, and if so you can merge it into the "real" timeline. You can make as many branches as you want, and branches of branches. You can even share branches with others (though we won't be doing that today). Git's branching system (and its non-centralized architecture generally) make it easy to try out new ideas in code without having to ask permission of everyone else on the project first.

To make your first branch, type $ git branch my-chunk-1 to make your new branch, and $ git checkout my-chunk-1 to switch into that branch. Type $ git branch to confirm that you're in your new branch (and that you only have two branches so far).

If you're not following this tutorial in class, just run $ git init.

Create your Django project

Let's create your first Django project, called "mysite".

$ django-admin.py startproject mysite

You'll see we made a folder called "mysite", with some files in it. Let's check them out!

$ cd mysite
$ ls
__init__.py
manage.py
settings.py
urls.py

These files are:

• __init__.py: An empty file that tells Python that this directory should be considered a Python module. Because of the __init__.py file, you can use import to import myproject.
• manage.py: A command-line utility that lets you interact with this Django project in various ways. You can read all the details about manage.py in django-admin.py and manage.py.
• settings.py: Settings/configuration for this Django project.
• urls.py: The URL declarations for this Django project; a “table of contents” of your Django-powered site.

Start your local development server

• Django, handily, comes with some local server software (warning: never use Django's built-in dev server for actually hosting a site in production! It's not meant for that.). This local dev server makes it easy to see the changes you make to your web app as you build it. Let's make sure our project works by verifying that the dev server will start.

• • Run the command:

python manage.py runserver

• • Review the output in your terminal. It should look similar to:

Validating models...
0 errors found.


Django version 1.2, using settings 'myproject.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

• Now that the server’s running, visit http://127.0.0.1:8000/ with your Web browser. You’ll see a “Welcome to Django” page, in pleasant, light-blue pastel. It worked!

• Observe the logging that happens in the terminal where your server is running:

[24/Mar/2011 11:50:18] "GET / HTTP/1.1" 200 2057

which has the format:

DATE METHOD URL PROTOCOL RESPONSE_CODE CONTENTSIZE

• Navigate to http://127.0.0.1:8000/some/url/. What changes in the terminal log?

• Exit the server by returning to the terminal instance where the development server is running and pressing CONTROL-C on your keyboard.

Using version control

Before we do anything else, let’s commit our work. We'll be doing this with a piece of version control software called git.

What's this version control nonsense? Well, you probably know the value of saving your work. When writing an important term paper, you want to "Save early, save often" to make sure you don't lose your work if your computer crashes or something else bad happens! But most paper-writing software isn't very good at saving your history--if you decide to delete a section of your paper, save it, and then later decide that you want that section back, most document writing software doesn't have any way of helping you do that. There's only one saved version of the paper (unless you save multiple copies over time, yourself, by hand).

Programmers often have to go back into the history of a program to change things--whether it's to diagnose a bug or redesign how a feature works. Programmers also have to have an easy way of being able to edit the same application at the same time, and to share their edits with each other. Thus, besides saving their work the normal way, programmers commit their code using version control software.

Each commit is a checkpoint in time, containing the diff--the "difference", all of the changes you made to your code base -- between that commit and the commit before it. Different branches in git share the commits made prior to the branching-off point, then each have their own commit history.

• To make a commit, first type git status into your terminal. This will let you know what changes git has noticed in your code and which of those changes, if any, are staged and ready to be committed.

$ git status


1. On branch my-chunk-1
3. Initial commit
5. Untracked files:
   
6. (use "git add <file>..." to include in what will be committed)
8. __init__.py
   
9. manage.py
   
10. settings.py
    
11. urls.py
    

nothing added to commit but untracked files present (use "git add" to track)

"Untracked files" means that git has noticed some new files inside its folder, but you haven't told git explicitly that you want it to "listen" for, and track, changes in those files.

• Add one file: git add manage.py. What does git status say now?
• Add all the files to the repo, in the local directory:

$ git add *.py # all .py files, using a wildcard match.

What does git status say now?

• git commit to commit those files:

# -m -> what is the 'message' for the commit
git commit -m "Initial commit of Django project from the IOLab Django workshop"

• Look at your changes with git log to see your history. Is your commit message there?

Huzzah! We committed our changes so far. Now let's make some more changes!

Set up settings

Now that we have a the scaffolding for our project in place, we can get to work! First, it needs to be configured.

Add yourself as an admin!

• Open settings.py in your editor. settings.py is a Python script that only contains variable definitions. Django looks at the values of these variables when it runs your web app.
• Find ADMINS and replace Your Name and your_email@example.com with your name and your email address.

Remove the pound mark from the front of the line to uncomment it out.

• git add and commit it:

# even though git knows to "listen" to settings.py now, it still won't add it to the 'staging area' for your commit unless you tell it to explicitly with git add
git add settings.py
git commit -m "made myself an admin"

Set up the Database

Keep looking at settings.py: The DATABASES variable is a dictionary (note the ‘{}’ characters) with one key: default.

DATABASES = {
'default': {
'ENGINE': 'django.db.backends', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
'NAME': , # Or path to database file if using sqlite3.
'USER': , # Not used with sqlite3.
'PASSWORD': , # Not used with sqlite3.
'HOST': , # Set to empty string for 127.0.0.1. Not used with sqlite3.
'PORT': , # Set to empty string for default. Not used with sqlite3.
}
}

Notice that the value of default is itself another dictionary with information about the site’s default database.

• Set your app to use a sqlite database, in the ENGINE attribute. Sqlite is great for development because is stores its data in one normal file on your system and therefore is really simple to move around with your app. It's not sturdy enough to use for a website in production, though.

'ENGINE': 'django.db.backends.sqlite3',

• Set your app to use a file called 'database.db' to store information for this project.

'NAME': 'database.db',

Does database.db exist right now? (No, but that's okay. It'll get created automatically when it's needed.)

Notice the INSTALLED_APPS setting towards the bottom of the settings.py. That variable (a tuple... note the ‘()’ symbols) holds the names of all Django applications that are activated in this Django instance. Apps can be used in multiple projects, and you can package and distribute them for use by others in their projects. What do you think these various apps do? Why does it make sense for them to come in a standard configuration?

• Add South to our list of installed apps. (We'll need it later.)

INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages',

1. Uncomment the next line to enable the admin:
2. 'django.contrib.admin',
3. Uncomment the next line to enable admin documentation:
4. 'django.contrib.admindocs',

'south', )

• Each of these applications makes use of at least one database table, so we need to create the tables in the database before we can use them. To do that, run the following command:

python manage.py syncdb

The syncdb command looks at the INSTALLED_APPS setting and creates any necessary database tables according to the database settings in your settings.py file. You’ll see a message for each database table it creates.

• When prompted, you’ll get a prompt asking you if you’d like to create a superuser account for the authentication system. Say yes! Use ‘super’ as your password.

Does database.db exist right now? (Yes! Calling syncdb made Django realize it needed a sqlite database file, so it made one for us.)

• Save and commit your work

Projects v. apps -- what's the difference?

We’ve talked a little about Django apps and projects. You might be wondering what that nonsense is all about. Here are the things to know:

• An app is component of a website that does something. South is a Django app. So is qandabear (or will be, anyway). An app is:
  • single purpose - login, passwords, polls, forum, etc.
  • orthogonal to / independent of other apps - qandabear shouldn’t have to know the inside details of authentication, for example.

• A project corresponds to a ‘website’: it contains a settings.py file, and it may have corresponding databases or other data stores that the apps interact with.

Most projects--those for more complex websites--should use multiple apps, one for each core piece of functionality.

Ideally, you should reuse existing free, open source Django apps that other people have written for things like user accounts, image galleries, and other common use cases. [Django Packages http://www.djangopackages.com] is a good resource for finding such apps. Django adheres to the principle of DRY -- Don't Repeat Yourself -- and in your real world coding adventures, you should strive to avoid repeating other, wiser developers as well. :)

Starting yer app

In this tutorial, we’ll create our poll app in the myproject directory for simplicity. In the future, when you decide that the world needs to be able to use your poll app and plug it into their own projects, and after you determine that your app plays nicely with other apps, you can publish that directory separately!

• Open your terminal and navigate into the mysite folder
• Make scaffolding for the app:

$ python manage.py startapp qandabear

That’ll create a directory qandabear to house our Q & A application.

• Verify what is new.

$ git status


1. should show 'qandabear/' in 'untracked'

• Examine the layout of qandabear (we will do more of this in following sections).

# remember not to type the '$', it just means the prompt'.
$ ls qandabear
qandabear/
__init__.py
models.py
tests.py
views.py


• Add and commit qandabear/*py.
• Install the Q&A app into the project. Edit the settings.py file again, and change the INSTALLED_APPS setting to include the string ‘qandabear’ as the last entry.
• Save and commit the settings.py file.

Chunk 2

At the end of each 'chunk', I'm going to push our progress so far to the tutorial git repository. If you're doing this tutorial in class, please switch back to the master branch, then pull my changes.

$ git checkout master
$ git pull origin master

This will make it so that everyone starts out each chunk with the same working code, so even if you get stuck or confused in one session, you can still keep up. (Since I don't have any TAs to help debug or answer questions, and we don't have that much time.)

Remember that *you* shouldn't be putting your modifications inside the master branch, though! (Otherwise, you'd have to overwrite them at the next chunk checkpoint, and that would be sad.) Instead, make a new branch for chunk 2 based off the master code you just downloaded, and let's get going!

$ git branch my-chunk-2
$ git checkout my-chunk-2

M-V-C separation

Model: how the data is organized. Controller: how the logic of the app works. View: How the app looks to the user.

(chart explaining model-view-controller separation in Django)

urls.py

The first step to writing your application is to design your URL structure. You do this by creating a Python module, called a URLconf. URLconfs are how Django associates a given URL with given Python code.

How URLconfs work (or, the joy of regular expressions!)

When a user requests a Django-powered page, the system looks at the ROOT_URLCONF setting, which contains a string in Python dotted syntax.

Django loads that module and looks for a module-level variable called urlpatterns, which is a sequence of Python tuples in the following format: (regular expression, Python callback function [, optional dictionary])

Django starts at the first regular expression and makes its way down the list, comparing the requested URL against each regular expression until it finds one that matches.

You might ask, “What’s a regular expression?” Regular expressions, or "regexes", are patterns for matching text. In this case, we’re matching the URLs people go to, and using regular expressions to match whole ‘groups’ of them at once. (If you’d like to learn more about regular expressions, read the [Dive into Python guide to regular expressions http://diveintopython.org/regular_expressions/index.html] sometime. Or you can look at this [xkcd http://xkcd.com/208/].)

In addition to matching text, regular expressions can capture text. Capturing means to remember that part of the string, for later use. Regexes use parentheses () to wrap the parts they’re capturing.

For Django, when a regular expression matches the URL that a web surfer requests, Django extracts the captured values (if any) and passes them to a function of your choosing. This is the role of the callback function above. When a regular expression matches the url, Django calls the associated callback function with any captured parts as parameters. This will much clearer after the next section.

Adding URLs to our urls.py

When we ran django-admin.py startproject mysite to create the project, Django created a default URLconf file called `urls.py`.

• Write our URL mapping. Edit the file mysite/urls.py so it looks like this:

urlpatterns = patterns(,
(r'^questions/$', 'qandabear.views.index'),
(r'^questions/(\d+)/$', 'qandabear.views.question'),
(r'^questions/(\d+)/answers/(\d+)/edit/$', 'qandabear.views.edit_answer'),


1. Examples:
   
2. url(r'^$', 'myproject.views.home', name='home'),
   
3. url(r'^myproject/', include('myproject.foo.urls')),
   

1. Uncomment the admin/doc line below to enable admin documentation:
   
2. url(r'^admin/doc/', include('django.contrib.admindocs.urls')),
   

1. Uncomment the next line to enable the admin:
   
2. url(r'^admin/', include(admin.site.urls)),
   

)

Suppose a visitor goes to http://127.0.0.1:8000/questions/23/.

1. which regex pattern is tripped?
2. what function is then called?
3. what arguments is that function called with?

• Save urls.py.

• Start the dev server and try that url out! What happens?

• Save and commit.

Review: When somebody requests a page from your Web site – say, “/questions/23/”, Django will load the urls.py Python module, because it’s pointed to by the ROOT_URLCONF setting. It finds the variable named urlpatterns and traverses the regular expressions in order. When it finds a regular expression that matches – r'^polls/(\d+)/$' – it loads the function question() from qandabear/views.py. Finally, it calls that module’s detail() function like so:

detail(request=<HttpRequest object>, '23')

The ‘23’ part comes from (\d+). Using parentheses around a pattern “captures” the text matched by that pattern and sends it as an argument to the view function; the \d+ is a regular expression to match a sequence of digits (i.e., a number).

The idea that a URL doesn’t have to map onto a file or a folder, or some other sort of static resource, is quite powerful. The URL is just a way of giving instructions to some server, somewhere.

(Rant: In Django, as in most modern frameworks, you have total control over the way your URLs look. People on the web won’t see cruft like .py or .php or even .html at the end of your URLs. There is no excuse for that kind of stuff in the modern era! (Though, putting .asp on the end of all your Django URLs, while pointless, is kind of hilarious and super easy.))

views.py

• Start the development server: python manage.py runserver
• Fetch “http://127.0.0.1:8000/questions/” in your browser. You should get a pleasantly-colored error page with the following message:

ViewDoesNotExist at /polls/

Tried index in module qandabear.views. Error was: 'module' object has no attribute 'index'

Recall this line: (r'^questions/$', 'qandabear.views.index').

• Explore this using your django-shell: python manage.py shell

>>> import qandabear # imports fine!
>>> import qandabear.views # imports fine also!
>>> dir(qandabear.views) # what is in there!
>>> 'index' in dir(qandabear.views)
False
>>> import inspect
>>> inspect.getsourcefile(qandabear.views)


1. something like
   

'/Users/adalovelace/gits/mysite/questions/views.py'

So, a mystery? Where is the view!? It’s nowhere! The URL parsing is going fine, but there is no one listening at the other end of the phone! This ViewDoesNotExist error happened because you haven’t written a function index() in the module polls/views.py.

Well, I guess we should do that!

• Write some views. Open qandabear/views.py and put the following Python code in it:

from django.http import HttpResponse

def index(request):

   return HttpResponse("Hello, world. You're at the questions index.")

This is a very simple view.

• Save the views.py file, then go to http://127.0.0.1:8000/questions/ in your browser, and you should see that text.

• Add a few more views by adding to the views.py file. These views are slightly different, because they take an argument or two (which, remember, is passed in from whatever was captured by the regular expression in the URLconf):

def question(request, question_id):

   return HttpResponse("You're looking at question %s." % (question_id,))

def edit_answer(request, question_id, answer_id):

   return HttpResponse("You're looking at the edit page for answer %s." % (answer_id,))

• Save views.py.

• Navigate to http://127.0.0.1:8000/questions/34/. It’ll run the question() method and display whatever ID you provide in the URL.
• Add a little html to the ‘question’ view. Wrap the question_id in tags and verify that the view is indeed bold!
• Add and commit your code. Remember to write a good commit message that mentioned what changed (in English) and more details below.

Bonus points

Test-driven development, part two

(walk through how to write tests for a new feature)