Django for Designers/Adding models

Part 3: Models and our database
Time: 50 minutes

Remember to make a new branch for section 3 based off the official branch!

====Introduction to databases and the ORM, or: finally, something we couldn't've done with plain HTML/CSS/JS!====

Okay, being able to design your URLs however you want, without them having to correspond to your actual file structure, is pretty neat. But besides that, we haven't done anything yet that you couldn't do with just HTML, CSS, and JavaScript. Time to fix that.

One key thing that separates web apps from web pages is that apps typically store data somewhere. The code that runs the app chooses what data seems most important to show at the moment. When writing an app using Django, we configure the storage through Django models.

Every Django app comes with a models.py in which you list each kind of data you want to store. These are configured through Python classes (to name the kind of data) with a sequence of attributes (which control the pieces of data that make up the model). If you're familiar with, or interested in, SQL, these correspond to tables and columns respectively.

The models that you define here are built on top of Django's object-relational mapper, or ORM. Because you define your models as Python objects, you can effectively write queries against a relational database by instead coding in Python with reference to your objects. Django then maps your Python code into relational database queries in SQL.

By configuring your data access in Python, Django makes it easy to take advantage of your data layout in other places. For example, you'll see later how the ORM makes it easy to automatically generate forms that ask for exactly the information you need. It is also generally convenient to write your entire app in one language, rather than being required to switch to SQL to do queries. The ORM also handles "escaping," which makes your SQL queries handle Unicode and other strange characters properly. These conveniences let you avoid mojibake-style data corruption and SQL injection attacks effortlessly.

Creating a basic model
Let's make it so we can store some real live data in our application! In our simple bookmarking app, we’ll create two models: Bookmarks and Tags. What information do we need to store about these objects?

A bookmark has:
 * A URL
 * A title (optionally)
 * A timestamp for when the bookmark was made

A tag has:
 * The tag slug/name

A tag also needs to know which bookmarks it applies to.

How do we represent this information in a Django model?

Open up bookmarks/models.py in your editor.

What's going on here?
 * We've made a Bookmark class that inherits from Django's model class.
 * We gave it a url, which is a URLField (a field that expects a string that parses as a valid URL).
 * We gave it a timestamp, which is a DateTimeField (a field that expects a Python datetime object). auto_now_add means that the field will automatically set itself to the time when the model is created.
 * Finally, we gave it a title, which is a CharField (a field that expects a string of some length or less, in this case less than 200 characters). "blank=True" means that Django will permit this field to be blank. We also set a default of the empty string, so that if someone fails to specify a title we permit that.

Now let's add a tag model! In the same file, below the Bookmark model:

The tag slug is stored in another CharField, which we've seen before. But what's this ManyToManyField? Well, it's one type of field for denoting a relationship between two models.

In relational databases, there's two basic types of relationships. In a ForeignKey relationship, one model relates to one and only one other model. You could imagine this like a Car model and a Wheel model. Each Wheel instance belongs to one and only one Car, so it would have a ForeignKey field for its car. The car, of course, can have multiple wheels.

With a ManyToManyField, on the other hand, the relationship isn't exclusive for either of the models involved. For instance, you could imagine having a Pizza model and a Topping model. Each Pizza can have multiple Toppings, and each Topping can be on multiple Pizzas.

(There are other variants of these fields, but this is the basic concept. See https://docs.djangoproject.com/en/dev/ref/models/fields/#module-django.db.models.fields.related for more details for how these relationship fields work in Django.)

In this case, each Bookmark can have multiple Tags, and each Tag can apply to multiple Bookmarks. So we use a ManyToManyField to store that information.

Creating database tables for your app (carefully, with South)
When you first create your models, you might not always know exactly what fields your models will need in advance. Maybe someday your bookmarks app will have preview thumbnail images! Then you would want to add another field to the model to store that information. Maybe someday you'll decide that tracking all the timestamps is silly, and want to delete that field.

Unfortunately, Django (and most database-using software) can’t figure out how to handle model changes very well on its own. Fortunately, a Django app called South that we installed earlier can handle these database changes--called ‘migrations’--for us.

Now that we’ve made our first version of our models file, let’s set up our bookmarks app to work with South so that we can make migrations with it in the future!

On the command line, write:

As you can see, that’s created a migrations directory for us, and automatically made a new migration file inside it.

All we need to do now is apply our new migration:

Great! Now our database file knows about bookmarks and its new models, and if we need to change our models, South is set up to handle those changes. We’ll come back to South later.

IMPORTANT: You can't migrate an app if it's already been synced in the database using

. But you do need to run syncdb at least once before you use south (since south itself uses syncdb to give itself space in your database). That's why it's super important that when you run syncdb, south should be listed under INSTALLED_APPS, but none of your own apps should be, and after you add your app to INSTALLED_APPS, you must not run syncdb again until after you've already set up migrations with that app.

Add and commit all your work, including the migrations folder that South generated for you!

Add some bookmarks via the command line
Now, let’s hop into the interactive Python shell and play around with the free API ("Application programming interface" -- APIs aren't just data doodads that hip web startups provide for you, they're an important concept in software architecture.) that Django gives you. To invoke the Python shell, use this command:

We’re using this instead of simply typing “python”, because manage.py's shell sets up the project’s environment for you. “Setting up the environment” involves two things:
 * Making sure bookmarks (and any other apps you might have) are on the right path to be imported.
 * Setting the DJANGO_SETTINGS_MODULE environment variable, which gives Django the path to your settings.py file.

Once you’re in the shell, let's explore the database API. Let's import the model classes we just wrote:

List all the current Bookmarks:

How many bookmarks is this? It's an empty list, so zero!

Let's add a bookmark:

We could specify the value for timestamp as well here, but we don't have to since when we wrote its model definition, we specified that timestamp has a default value (of when the bookmark is created). We did specify a title for the bookmark, even though we didn't have to since it's an optional field.

Try getting the ID number of our new bookmark 'b' by typing:

Python has given you back your prompt because the value of b.id is None. That is because we have not yet saved the object; to the database, it does not exist.

Save your bookmark to the database. In Django, you have to call save explicitly.

Now, try again to get the id of the Bookmark instance. Because it’s been saved, it has an ID in the database now! Even though we didn't specify one in our models.py, every saved model instance automatically has an id field.

Access the database columns (Fields, in Django parlance) as Python attributes:

We can change the bookmark title by changing its title attribute, then calling save.

If we now ask Django to show a list of all the Bookmark objects available, we can see it's no longer an empty list!

Fix the hideous default model representation
Wait a minute!  is an utterly unhelpful representation of this object. Let’s fix that by editing the Bookmark model. Use your text editor to open the bookmarks/models.py file and adding a __unicode__ method to both Bookmark and Tag:

Let's see our new shiny __unicode__ methods in action. Start a new Python interactive shell by running:

Save and commit your changes.

Adding more data via the shell
We've created a bunch of bookmarks, but no tags! Let's change that.

Slicing and dicing Django data
There are many methods for searching across your Django models.

We can filter our bookmarks, for instance for ones with a particular URL:

Or for bookmarks which have titles:

If you try to use filter to search for a bookmark that does not exist, filter will give you the empty list.

The get method, on the other hand, returns exactly one hit. If it finds zero matches, or more than one match, it will raise an exception.

More information on making queries with Django's ORM can be found in the Django docs at https://docs.djangoproject.com/en/dev/topics/db/queries/#retrieving-objects.

Go take a break!!!

Part 3.5: Changing our mind and adding users
D'oh! You know what every social bookmarking app has, that ours doesn't have? Users!

I don't mean like the number of people using it--I mean a way to store different users' accounts and keep track of who owns which bookmarks. So let's change our app to include this feature.

Lucky for us, Django comes with an app for user accounts and authentication from the get-go! In fact, it's already installed. If you look back at your settings.py file, you'll see that in INSTALLED_APPS, there is an entry for 'django.contrib.auth'.

That's our built-in authentication app!

If it isn't already open, let's open the Django shell and play with this app a bit.

Whaaaaat?? There's already a User here. How can that be?

You might recall making a 'superuser' account when you first set up your Django project. That superuser was, in fact, created using Django's built-in auth app.

What is our user account's id number?

Neato!

Add user field to bookmark
Now we need to create a relationship between the built-in User model and our Bookmark model.

First, in our models.py file, we need to import django.contrib.auth's built-in User model so that we can refer to it in our models.

Now we need to think--what kind of relationship do users and bookmarks have?

Well, a user can have multiple bookmarks. But (right now, anyway) a bookmark should only have one user. So that means that we should use a ForeignKey field to add the user to our Bookmark model.

While we're at it, let's update the __unicode__ method too to let us know to whom a bookmark belongs to.

Make a migration in South
Now that we've added a field to our model, we are going to need to create a database migration. South can help us do this!

To create our migration, run

Note that we're now using --auto instead of --initial (which we used back when we first wrote our models and set up our app to use South).

Eep! Before it will make our migration file, South wants some information from us:

We could in theory modify our models to specify a default value for author, or make it optional. But neither of those sound like good options. We *want* future bookmarks to be forced to have an author! So we'll choose 2, to set up a default value for our new author field just for the purposes of this migration.

We then need to come up with a default value. Well, right now there's only one User in our system who the sample bookmarks we'd entered so far could belong to--our superuser account, which (if you don't remember) had an ID number of 1.

So let's enter 1 for our default.

Remember, the first step creates the migration, but doesn't run it. So let's do what South says and run a command to apply our migration!

Save and commit your work to add users to your bookmarks app!

Templates and links for login/logout/etc
Django's auth app comes with built-in views, which we can use to handle login and logout functionality for our users. Once we point some URLs at them, that is.

First, let's edit urls.py:

Now if anyone goes to localhost:8000/login/, the built-in login view will get triggered.

Run your dev server and try that. What error do you see?

While there is a built-in login view, there is no built-in login template. We need to build one for it. While we could put it anywhere and tell login explicitly where to look, by default the login view expects the template to reside at templates/registration/login.html. So we may as well put it there.

Create a new file within the new registration directory called login.html, and put this inside:

  {% endblock %}

Note the hidden input with the name "next". This input tells the login view what URL to send the user to after they successfully log in. We have it set to '/', so it'll just take them back to the home page.

We need to add a logout view too! Let's do that. Back in urls.py:

The dictionary after the logout URL sends some extra arguments to the logout view. Specifically it tells the logout view where to send the user after they log out. We could make a special goodbye splash page or something, but nah, let's just send them back to the home page again.

Now we need to add a login/logout link to our site so people can actually use these views! We want this link to be at the top of every page, so we'll edit our base template:

Our template checks to see if there's a logged-in user, and if so, shows a hello message and a logout link. If the user isn't logged in, it shows a login link instead.

Check http://localhost:8000 and try logging in with the superadmin username and password you created before! It should work. :)

You might be wondering--where did the 'user' variable in the template come from? If you look at views.py, you'll notice we never added a user variable to our context dictionary. So how did this happen?

The answer is in the function we are using to render our templates, render. render automatically uses the request we sent it to create a Django RequestContext, which contains a bunch of extra context variables that get sent along to every view that uses a RequestContext. The Django auth app adds the current user to the RequestContext automatically. This is handy, since you don't want to have to look up the current user to every single view you ever write separately, just so the nav section on your website will work everywhere!

There are other functions that return an HTMLResponse, like render, but don't include a RequestContext. render_to_response is one common shortcut function that doesn't include it by default; another was the HttpResponse function we used earlier! Just something to remember--if you're trying to send a piece of data to almost every page in your web app, 1.) you probably want to use render in your views, and 2.) you want to find a way to add your data to your app's RequestContext. (https://docs.djangoproject.com/en/dev/ref/templates/api/#django.template.RequestContext says more about this!)

Save and commit your changes in adding login/logout functionality.

Part 3.75: Modify views and templates to use model data
Let's edit our views and templates so they use real bookmark data from our database!

Add more data
First, let's add a bit more bookmark and tag data to our database. We'll use a Python loop to add a lot of bookmarks at once.

That will bring us to the Django management shell. Within that special Python prompt, type the following:

(Remember, any time you see ">>>", you don't have to type those characters.)

This tells Python we'll be accessing the user model, and then grabs the first user, storing it in a variable called me.

Keeping that Python prompt open, run the following commands:

This creates a number of bookmarks, tagged appropriately! You can exit the shell by typing:

Get views.py to talk to our models
Then, we'll need to have our views.py file import the bookmark model and send data to the index view.

We'll add an import statement:

And we'll edit index(request):

Wait! We don't actually want to load every bookmark in our database when we go to the front page. If we have lots of bookmarks, that will get slow and unwieldy quickly.

Instead, let's show the 10 most recent bookmarks:

We also want our tag view to show real bookmarks. We only want to show bookmarks that have been tagged with the given tag. So we'll edit the import statement we just added:

And alter the tag definition as well:

Change templates to handle bookmark data
Now, let's modify our templates to use this data.

In index.html, update just block content per the following. Make sure to keep the extends directive and the block subheader directive.

Instead of writing out each bookmark list element individually ahead of time, we are using a Django template language for loop to create a list element for each bookmark. Thus, we only have to specify the HTML formatting of our list elements once!

Start your development server and look at http://localhost:8000 now. You'll see that instead of the fake HTML we had before, each link's text is the unicode representation of each bookmark in your database. Cool, huh?

That's okay, but the programmer-friendly unicode representation isn't really what we want to go there. Let's fix that:

If the bookmark has a title, we'll make the title the link. Otherwise we'll show the URL. We also fill in the URL on the  tag, to make the link work.

We also want to fill in the other metadata, like tags and author info. To do that:

We check if there are any tags for the bookmark, and if so loop over the tags to put each of them in.

Additionally, we use Django's {% url %} tag to generate the URL for our tag's link. The URL tag takes first an argument which is the path of a particular view function (bookmarks.views.tag), then additional arguments for any input variables that the function expects to glean from the URL. (As you may recall, the tag view takes an argument tag_name, which is the name of the tag in question).

Why use {% url %} instead of just writing "/tags/{&#8288;{ tag.slug }}"? Django principle of DRY--Don't Repeat Yourself--means that we want to avoid duplicating work as much as possible. If later down the line we decided to change our URL structure so that tag pages would appear at "/bookmarks/by_tag/" instead, we'd have to go in and fix all these hard-coded URL patterns by hand. Using the {% url %} tag makes Django generate our URL for us, based on our urls.py file, so any changes we make automatically get propagated outward!

Run

and confirm that your tag links working and our other changes are visible. It's easy to typo or make a mistake with {% url %} tags, so we want to make sure everything is working.

The default timestamp formatting that Django gave us is pretty neat, but in our original mockup we just used the date--not this long timestamp. Fortunately, Django's built-in filters make it easy to format a date or time any way we want! Since we just want to show the date, we'll use the date filter:

We tell Django that we're using a filter via the |filter_name syntax. The arguments that come after the colon are a standard Python code for describing different ways of formatting dates. You can read more about the date filter and all the different formatting codes at https://docs.djangoproject.com/en/dev/ref/templates/builtins/#date. For our purposes, Y outputs the full four-digit year, while m and d outputs the month and the day as two digit numbers. The hyphens we put between them are included in the formatting, too -- if we wanted the date to use slashes instead, we'd simply write |date:"Y/m/d".

Spin up your dev server, if you haven't got it running already, and check out your changes! Then save and commit your precious work.

Updating the tag template, and dealing with errors
Let's update our tag.html template to use the same formatting as index.html. Since we're using the same formatting for the list of bookmarks, and there aren't any other content block differences between index.html and tag.html, we can simply modify our tag template to inherit from index.html. Edit tag.html to have only the following contents. (That means you can remove the mockup "block content" section in tags.html. Yay!)

Now we can go to http://localhost:8000/tags/funny and see a nicely styled list of bookmarks tagged with that tag!

What happens if we go to http://localhost:8000/tags/asdfjkl ?

Eek, a DoesNotExist error! That's not so great. Our application is erroring on one of our view lines:

Fortunately, Django has a shortcut function that can help us -- get_object_or_404. This function will attempt to get a Django model based on the parameters you give it, and if it fails, automatically throw a standard HTTP "not found" error, number 404.

Now http://localhost:8000/tags/asdfjkl will throw a nicer 404 error. We could even make a pretty 404.html template to handle such errors! There are also shortcuts in Django for 500 and 403 (Forbidden) errors, if you want to handle and style those as well.

Save and commit your error-catching work.

Let me show off
So far, we've all only been able to access our own Django-powered sites. In this section, you will see how to access my (the instructor's) super cool bookmarks site! (We'll talk later about how you can share your code the same way. It might require firewall configuration changes on your computer, so we save that complexity for later.)

Assuming that you and the instructor are on the same wifi network, you can visit a temporary website on the instructor's computer. Visit this link:

http://192.168.1.1:8000/

(If you are wondering exactly who can connect to the instructor's app, it is typically only people on the same wifi/wired network as her, rather than the whole Internet. This is due to the a technique in wide use called "network address translation", and it is not considered perfect security.)

When you visit the instructor's app, you are interacting with the database stored on her laptop. So all the changes made by other people in the room are reflected in what you see! So try not to be too silly.

Take a break!!