Django for ISchoolers

This is a tutorial for how to build a question-and-answer type site using Django and various other technologies, plus some lessons in general web dev concepts and best practices along the way.

= Chunk one =

Assumptions
This tutorial assumes you know how to access the command line/terminal on your computer and how to move around (cd) and view files (ls) in it. It also assumes you're reasonably familiar with Python, the Best Programming Language Ever. (Okay, editorializing slightly.)

This tutorial also tends to be oriented towards people with some familiarity with HTML, CSS, and JS (since most of you guys are IOLabbers).

Stuff to install
TODO: Actually write my own dang installation instructions.

Prior to starting this tutorial, please install Python, git, pip, virtualenv, Django, and South on your computer. There's instructions for all of those at http://pystar.org/setup_machine.html.

Setting up yer env
Once you've got pip and virtualenv installed, in your terminal, go into the folder you want to keep this tutorial code in and type (one line at a time):

This will make and activate a virtual python environment for just this tutorial app. While this is a bit overkill if you only have one Django app at a time on your system, giving each Python app you work on its own environment is a good best practices just in case e.g. you're working on multiple apps, which use different versions of Django or other libraries, and you don't want them to conflict with each other.

To turn off the virtualenv (you can see it's active because the env name will appear in parentheses before your terminal prompt), type 'deactivate'. To reactivate the env, type the line that begins with 'source' again.

Spec'ing yer application
Your client, LulzTech, wants you to build them a Q & A website, called qandabear. ('Q and A bear', rhymes with panda bear.) For mysterious reasons, they don't want to use any of the open source Q&A website packages that are out there already; they want you to build one from scratch. Go figure.

There are whole classes on how to research and decide what to build, and how to state those requirements in a useful, clear way. (Bob's ISSD class, for one.) So we won't go into that. For our initial prototype for our client, then, our requirements are as follows:


 * On the website, users can see the questions that have been asked and when they were posted
 * Under each question, users can read the answers that have been given so far (if any) and when they were posted
 * Users can submit a new question
 * Users can submit a new answer to a question
 * Users can edit an answer to a question
 * Users can delete an answer to a question

Meeting these requirements will take us until the 'Bonus points' section. Once we get there, we can add more requirements.

Our prototype will have three pages: an index page of questions (and a form to ask a new question), a question page with the question and its answers (with a form to add a new answer), and a page to edit an answer to a question.

To support these pages, we will need two abstractions (objects, or models): Questions, and Answers.

Activate your environment in your terminal
If you haven't already, run  in your terminal window.

Set up your git repository
If you're following this tutorial in class, in your terminal, run. This won't download hardly anything interesting, but it will make it easy for you to sync up with the class later.

We'll be using the master branch (the default name for the main branch of code, which you're in right now) for syncing up throughout the class in case you get lost. You'll need to make your own branches for playing with the code on your computer, so that it doesn't interfere w. re-syncing later.

Think of branches in git as alternate timelines--you "branch" off of the main timeline of changes to the code into a separate timeline. You can try out a new feature or code setup in a branch, without overwriting or getting in the way of yourself or anybody else working on the code as-is. Sometimes a new idea doesn't work out, and you delete the branch or just let it sit around. Sometimes the new feature works, though, and if so you can merge it into the "real" timeline. You can make as many branches as you want, and branches of branches. You can even share branches with others (though we won't be doing that today). Git's branching system (and its non-centralized architecture generally) make it easy to try out new ideas in code without having to ask permission of everyone else on the project first.

To make your first branch, type  to make your new branch, and   to switch into that branch. Type  to confirm that you're in your new branch (and that you only have two branches so far).

If you're not following this tutorial in class, just run.

Create your Django project
Let's create your first Django project, called "mysite".

You'll see we made a folder called "mysite", with some files in it. Let's check them out!

These files are:


 * __init__.py: An empty file that tells Python that this directory should be considered a Python module. Because of the __init__.py file, you can use import to import myproject.
 * manage.py: A command-line utility that lets you interact with this Django project in various ways. You can read all the details about manage.py in django-admin.py and manage.py.
 * settings.py: Settings/configuration for this Django project.
 * urls.py: The URL declarations for this Django project; a “table of contents” of your Django-powered site.

Start your local development server

 * Django, handily, comes with some local server software (warning: never use Django's built-in dev server for actually hosting a site in production! It's not meant for that.). This local dev server makes it easy to see the changes you make to your web app as you build it. Let's make sure our project works by verifying that the dev server will start.


 * Run the command:


 * Review the output in your terminal. It should look similar to:


 * Now that the server’s running, visit http://127.0.0.1:8000/ with your Web browser. You’ll see a “Welcome to Django” page, in pleasant, light-blue pastel. It worked!


 * Observe the logging that happens in the terminal where your server is running:

which has the format:


 * Navigate to http://127.0.0.1:8000/some/url/. What changes in the terminal log?


 * Exit the server by returning to the terminal instance where the development server is running and pressing CONTROL-C on your keyboard.

Using version control
Before we do anything else, let’s commit our work. We'll be doing this with a piece of version control software called git.

What's this version control nonsense? Well, you probably know the value of saving your work. When writing an important term paper, you want to "Save early, save often" to make sure you don't lose your work if your computer crashes or something else bad happens! But most paper-writing software isn't very good at saving your history--if you decide to delete a section of your paper, save it, and then later decide that you want that section back, most document writing software doesn't have any way of helping you do that. There's only one saved version of the paper (unless you save multiple copies over time, yourself, by hand).

Programmers often have to go back into the history of a program to change things--whether it's to diagnose a bug or redesign how a feature works. Programmers also have to have an easy way of being able to edit the same application at the same time, and to share their edits with each other. Thus, besides saving their work the normal way, programmers commit their code using version control software.

Each commit is a checkpoint in time, containing the diff--the "difference", all of the changes you made to your code base -- between that commit and the commit before it. Different branches in git share the commits made prior to the branching-off point, then each have their own commit history.


 * To make a commit, first type  into your terminal. This will let you know what changes git has noticed in your code and which of those changes, if any, are staged and ready to be committed.

"Untracked files" means that git has noticed some new files inside its folder, but you haven't told git explicitly that you want it to "listen" for, and track, changes in those files.


 * Add one file: . What does git status say now?
 * Add all the files to the repo, in the local directory:

What does git status say now?


 * git commit to commit those files:


 * Look at your changes with  to see your history. Is your commit message there?

Huzzah! We committed our changes so far. Now let's make some more changes!

Set up settings
Now that we have a the scaffolding for our project in place, we can get to work! First, it needs to be configured.

Add yourself as an admin!
Remove the pound mark from the front of the line to uncomment it out.
 * Open settings.py in your editor. settings.py is a Python script that only contains variable definitions. Django looks at the values of these variables when it runs your web app.
 * Find ADMINS and replace Your Name and your_email@example.com with your name and your email address.
 * git add and commit it:

Set up the Database
Keep looking at settings.py: The DATABASES variable is a dictionary (note the ‘{}’ characters) with one key: default.

Notice that the value of default is itself another dictionary with information about the site’s default database.


 * Set your app to use a sqlite database, in the ENGINE attribute. Sqlite is great for development because is stores its data in one normal file on your system and therefore is really simple to move around with your app. It's not sturdy enough to use for a website in production, though.


 * Set your app to use a file called 'database.db' to store information for this project.

Does database.db exist right now? (No, but that's okay. It'll get created automatically when it's needed.)

Notice the INSTALLED_APPS setting towards the bottom of the settings.py. That variable (a tuple... note the ‘’ symbols) holds the names of all Django applications that are activated in this Django instance. Apps can be used in multiple projects, and you can package and distribute them for use by others in their projects. What do you think these various apps do? Why does it make sense for them to come in a standard configuration?


 * Add South to our list of installed apps. (We'll need it later.)


 * Each of these applications makes use of at least one database table, so we need to create the tables in the database before we can use them. To do that, run the following command:

The syncdb command looks at the INSTALLED_APPS setting and creates any necessary database tables according to the database settings in your settings.py file. You’ll see a message for each database table it creates.


 * When prompted, you’ll get a prompt asking you if you’d like to create a superuser account for the authentication system. Say yes! Use ‘super’ as your password.

Does database.db exist right now? (Yes! Calling syncdb made Django realize it needed a sqlite database file, so it made one for us.)


 * Save and commit your work

Projects v. apps -- what's the difference?
We’ve talked a little about Django apps and projects. You might be wondering what that nonsense is all about. Here are the things to know:


 * An app is component of a website that does something. South is a Django app. So is qandabear (or will be, anyway). An app is:
 * single purpose - login, passwords, polls, forum, etc.
 * orthogonal to / independent of other apps - qandabear shouldn’t have to know the inside details of authentication, for example.


 * A project corresponds to a ‘website’: it contains a settings.py file, and it may have corresponding databases or other data stores that the apps interact with.

Most projects--those for more complex websites--should use multiple apps, one for each core piece of functionality.

Ideally, you should reuse existing free, open source Django apps that other people have written for things like user accounts, image galleries, and other common use cases. [Django Packages http://www.djangopackages.com] is a good resource for finding such apps. Django adheres to the principle of DRY -- Don't Repeat Yourself -- and in your real world coding adventures, you should strive to avoid repeating other, wiser developers as well. :)

Starting yer app
In this tutorial, we’ll create our Q&A app in the myproject directory for simplicity. In the future, when you decide that the world needs to be able to use your Q&A app and plug it into their own projects, and after you determine that your app plays nicely with other apps, you can publish that directory separately!


 * Open your terminal and navigate into the mysite folder
 * Make scaffolding for the app:

That’ll create a directory qandabear to house our Q & A application.


 * Verify what is new.


 * Examine the layout of qandabear (we will do more of this in following sections).


 * Add and commit.
 * Install the Q&A app into the project. Edit the settings.py file again, and change the INSTALLED_APPS setting to include the string ‘qandabear’ as the last entry.
 * Save and commit the settings.py file.

= Chunk 2 =

At the end of each 'chunk', I'm going to push our progress so far to the tutorial git repository. If you're doing this tutorial in class, please switch back to the master branch, then pull my changes.

This will make it so that everyone starts out each chunk with the same working code, so even if you get stuck or confused in one session, you can still keep up. (Since I don't have any TAs to help debug or answer questions, and we don't have that much time.)

Remember that *you* shouldn't be putting your modifications inside the master branch, though! (Otherwise, you'd have to overwrite them at the next chunk checkpoint, and that would be sad.) Instead, make a new branch for chunk 2 based off the master code you just downloaded, and let's get going!

M-V-C separation
Model: how the data is organized. Controller: how the logic of the app works. View: How the app looks to the user.

(chart explaining model-view-controller separation in Django)

urls.py
The first step to writing your application is to design your URL structure. You do this by creating a Python module, called a URLconf. URLconfs are how Django associates a given URL with given Python code.

How URLconfs work (or, the joy of regular expressions!)
When a user requests a Django-powered page, the system looks at the ROOT_URLCONF setting, which contains a string in Python dotted syntax.

Django loads that module and looks for a module-level variable called urlpatterns, which is a sequence of Python tuples in the following format: (regular expression, Python callback function [, optional dictionary])

Django starts at the first regular expression and makes its way down the list, comparing the requested URL against each regular expression until it finds one that matches.

You might ask, “What’s a regular expression?” Regular expressions, or "regexes", are patterns for matching text. In this case, we’re matching the URLs people go to, and using regular expressions to match whole ‘groups’ of them at once. (If you’d like to learn more about regular expressions, read the [Dive into Python guide to regular expressions http://diveintopython.org/regular_expressions/index.html] sometime. Or you can look at this [xkcd http://xkcd.com/208/].)

In addition to matching text, regular expressions can capture text. Capturing means to remember that part of the string, for later use. Regexes use parentheses to wrap the parts they’re capturing.

For Django, when a regular expression matches the URL that a web surfer requests, Django extracts the captured values (if any) and passes them to a function of your choosing. This is the role of the callback function above. When a regular expression matches the url, Django calls the associated callback function with any captured parts as parameters. This will much clearer after the next section.

Adding URLs to our urls.py
When we ran  to create the project, Django created a default URLconf file called `urls.py`.


 * Write our URL mapping. Edit the file mysite/urls.py so it looks like this:

Suppose a visitor goes to http://127.0.0.1:8000/questions/23/.
 * 1) which regex pattern is tripped?
 * 2) what function is then called?
 * 3) what arguments is that function called with?


 * Save urls.py.


 * Start the dev server and try that url out! What happens?


 * Save and commit.

Review: When somebody requests a page from your Web site – say, “/questions/23/”, Django will load the urls.py Python module, because it’s pointed to by the ROOT_URLCONF setting. It finds the variable named urlpatterns and traverses the regular expressions in order. When it finds a regular expression that matches – r'^questions/(\d+)/$' – it loads the function question from qandabear/views.py. Finally, it calls that module’s detail function like so:

The ‘23’ part comes from (\d+). Using parentheses around a pattern “captures” the text matched by that pattern and sends it as an argument to the view function; the \d+ is a regular expression to match a sequence of digits (i.e., a number).

The idea that a URL doesn’t have to map onto a file or a folder, or some other sort of static resource, is quite powerful. The URL is just a way of giving instructions to some server, somewhere.

(Rant: In Django, as in most modern frameworks, you have total control over the way your URLs look. People on the web won’t see cruft like .py or .php or even .html at the end of your URLs. There is no excuse for that kind of stuff in the modern era! (Though, putting .asp on the end of all your Django URLs, while pointless, is kind of hilarious and super easy.))

views.py

 * Start the development server:
 * Fetch “http://127.0.0.1:8000/questions/” in your browser. You should get a pleasantly-colored error page with the following message:

Recall this line:.


 * Explore this using your django-shell:

So, a mystery? Where is the view!? It’s nowhere! The URL parsing is going fine, but there is no one listening at the other end of the phone! This ViewDoesNotExist error happened because you haven’t written a function index in the module questions/views.py.

Well, I guess we should do that!


 * Write some views. Open qandabear/views.py and put the following Python code in it:

This is a very simple view.


 * Save the views.py file, then go to http://127.0.0.1:8000/questions/ in your browser, and you should see that text.


 * Add a few more views by adding to the views.py file. These views are slightly different, because they take an argument or two (which, remember, is passed in from whatever was captured by the regular expression in the URLconf):


 * Save views.py.


 * Navigate to http://127.0.0.1:8000/questions/34/. It’ll run the question method and display whatever ID you provide in the URL.
 * Add a little html to the ‘question’ view. Wrap the question_id in tags and verify that the view is indeed bold!
 * Add and commit your code. Remember to write a good commit message that mentioned what changed (in English) and more details below.

Databases and the ORM (or: finally something that HTML/CSS/JS couldn't do for you on its own)
Okay, being able to design your URLs however you want, without them having to correspond to your actual file structure, is pretty neat. But besides that, we haven't done anything yet that you couldn't do with just HTML, CSS, and JavaScript (or just HTML, as it happens!).

Time to fix that.

ORM = Object Relational Mapper/Mapping. Translating Python classes -- real object-oriented programming stuff -- into representations in a database on your server for you.

(models.py --> ORM --> DB chart)

models.py
Let's make it so we can store some real live data in our application!

In our simple Q&A app, we’ll create two models: Questions and Answers. As per our spec from the customer:


 * A question has:
 * The text of the question
 * A publication date.
 * An answer has:
 * The text of the answer
 * A publication date.

Each Answer is associated with a Question and each Question has associated Answers. We will respesent these concepts with python classes derived from django.db.models.

Edit the qandabear/models.py file so it looks like this:


 * Save the models.py file.

All models in Django code are represented by a class that subclasses django.db.models.Model. Each model has a number of class variables, each of which represents a database field in the model. (cf: http://docs.djangoproject.com/en/dev/topics/db/models/)

Each field is represented by an instance of a Field class – e.g., CharField for character fields and DateTimeField for datetimes. This tells Django what type of data each field holds.

The name of each Field instance (e.g. answer or pub_date) is the field’s name, in machine-friendly format. You’ll use this value in your Python code, and your database will use it as the column name.

Some Field classes have required elements. CharField, for example, requires that you give it a max_length. That’s used not only in the database schema, but in validation as well.

Most Field classes also have optional elements that you can set. For example, setting auto_now_add to True on a datetime field means that by default, that field will be set to the date and time that the object is first created.

Finally, note a relationship is defined, using ForeignKey. That tells Django each Answer is related to a single Question. Django supports all the common database relationships: many-to-ones, many-to-manys and one-to-ones.

Database migrations and South, part one
When you create your models, you might not always know exactly what fields your models will need in advance. Maybe someday your qandabear app will have users, and you’ll want to keep track of the author of each question! Then you would want to add another field to the model to store that information. Maybe someday you'll decide that tracking all the pub_dates is silly, and want to delete that field.

Unfortunately, Django (and most database-using software) can’t figure out how to handle model changes very well on its own. Fortunately, a Django app called `south` can handle these changes–called ‘migrations’–for us.

Now that we’ve made our first version of our models file, let’s set up our qandabear app to work with South so that we can make migrations with it in the future!


 * On the command line, write:

As you can see, that’s created a migrations directory for us, and made a new migration inside it.


 * All we need to do now is apply our new migration:

Great! Now our database file knows about qandabear and its new models, and if we need to change our models, South is set up to handle those changes. We’ll come back to South later.

IMPORTANT: You can't migrate an app if it's already been synced in the database using. But you do need to run syncdb at least once before you use south (since south itself uses syncdb to give itself space in your database). That's why it's super important that when you run syncdb, south should be listed under INSTALLED_APPS, but none of your own apps should be, and after you add your app to INSTALLED_APPS, you must not run syncdb again until after you've already set up migrations with that app.


 * Add and commit all your work, including the migrations folder that South generated for you.

= Chunk 3 =

You know the drill. Switch back to the master branch, pull down all the new code, and create a new branch called my-chunk-3. (Or something else, if you like. Your branch names are up to you!)

Let's add some data (via the Django shell)!
Now, let’s hop into the interactive Python shell and play around with the free API ("Application programming interface" -- APIs aren't just data doodads that hip web startups provide for you, they're an important concept in software architecture.) that Django gives you. To invoke the Python shell, use this command:

We’re using this instead of simply typing “python”, because manage.py sets up the project’s environment for you. “Setting up the environment” involves two things:


 * 1) Making sure qandabear (and any other apps you might have) are on the right path to be imported.
 * 2) Setting the DJANGO_SETTINGS_MODULE environment variable, which gives Django the path to your settings.py file.

Once you’re in the shell, explore the database API:


 * Import the model classes we just wrote:


 * List all the current Questions:

How many questions is this?

Zen koan: Can there be an Answer for a Question that doesn’t yet exist?

Add our first Question

 * Save the Question instance into the database. You have to call save explicitly.


 * Get the id of the Question instance. Because it’s been saved, it has an ID in the database.


 * Access the database columns (Fields, in Django parlance) as Python attributes:


 * Send the Question back in time:


 * Ask Django to show a list of all the Question objects available:

Fix The Hideous Default Representation
Wait a minute!  is an utterly unhelpful, truly wretched, beyond contemptable representation of this object. Let’s fix that by editing the Question model. Use your text editor to open the qandabear/models.py file and adding a __unicode__ method to both Question and Answer:

It’s important to add __unicode__ methods to your models, not only for your own sanity when dealing with the interactive prompt, but also because objects’ representations are used throughout Django’s automatically-generated admin.

(If you’re using to Python programming from a time in the past, you might have seen __str__. Django prefers you use __unicode__ instead.)

Adding custom methods to models
Enough of these normal python methods! Let's build in some useful functionality to our models.

Note the addition of import datetime to reference Python’s standard datetime module. This allows us to use the datetime library module in models.py by calling it with datetime.


 * Save these changes to the models.py file.

Test that all those changes did something
Start a new Python interactive shell by running python manage.py shell:

>>> from polls.models import Poll, Choice Verify our __unicode__ addition worked: >>> Poll.objects.all [] Search your database using the filter method on the objects attribute of Poll. >>> polls = Poll.objects.filter(question="What is the Weirdest Cookbook Ever?") >>> polls [] >>> polls[0].id # remember python lists start with element 0. 1 If you try to search for a poll that does not exist, filter will give you the empty list. The get method will always return one hit, or raise an exception. >>> Poll.objects.filter(question="What is the Weirdest Cookbook Ever?") []

>>> Poll.objects.get(id=1)  >>> Poll.objects.get(id=2) Traceback (most recent call last): ... DoesNotExist: Poll matching query does not exist. Add Choices

Observe, there is a Poll in the database, but it has no Choices. >>> p = Poll.objects.get(id=1) >>> p.choice_set.all [] Create three choices: >>> p.choice_set.create(choice='To Serve Man', votes=0)  >>> p.choice_set.create(choice='The Original Road Kill Cookbook', votes=0)  >>> c = p.choice_set.create(choice='Mini-Mart A La Carte', votes=0) >>> c  Go in reverse! Find the poll a particular choice belongs to: >>> c.poll  Because a Poll can have more than one Choice, Django creates the choice_set attribute on each Poll. You can use that to look at the list of available Choices, or to create them. >>> p.choice_set.all [, , ] >>> p.choice_set.count 3 No really. Can one be a Choice for a Poll that doesn’t yet exist?: >>> koan = choice("Is this even a choice") >>> koan.poll_id >>> koan.poll

Views with actual data
= Chunk 4 =

Oh, CRUD! (plus, ModelForms)
= Bonus points =

Test-driven development, part two
(walk through how to write tests for a new feature)