Django for Designers/Whats next

Updating and deleting bookmarks (the last two parts of CRUD)
Let's make it possible to delete a bookmark. The changes we will make to support that are to add a delete button next to each bookmark, add a a view that handles that delete button, and hook all that up to urls.py.

To make the delete button appear, edit bookmark.html so that it has the following text, top to bottom. The only change we are making is the addition of a new div toward the bottom.

We added a very dense block there. Here is an explanation, piece-by-piece:


 * It only renders if there is a user logged-in.
 * It looks for a URL corresponding to a view (that we have not yet written) called bookmarks.delete.
 * It passes the bookmark ID to that view as a URL parameter.
 * It provides a CSRF token, and a submit button labeled Delete.

To make that work, the next step to to take is to write our view. Open up bookmarks/views.py and add the following to the end of the file:

Namely, if the request is a POST, we try to find the corresponding bookmark and delete it. No matter what, we redirect back to the start.

Finally, we need to adjust urls.py to be aware of this view. To do that, open up urls.py in your editor, and add this line anywhere within the urlpatterns sequence (for example, underneath the final current URL pattern):

(The (\d+) means, "any sequence of digits, 1 or longer, captured." Capturing it permits the variable to be passed to the Python view function.)

Save all of that, and now try loading up the home page. If you see a bookmark you want to delete, click the delete button. What should appear to happen is that the page reloads itself, and that bookmark is now gone.

Note that, at the moment, any user can delete any other user's bookmarks. If you want things to work differently, you'll need to add a check to the delete function in views.py.

To be able to update bookmarks, we will need to create add an update link, make it go to a page where bookmarks can be edited, and then let that page be saved. (If we wanted to make this slicker, the first step would be to put the edit page in a form using e.g. jQuery UI.)

We already have a form that lets us enter bookmarks, and we can re-use it as the editing area. As a reminder, open index.html and notice the following fragment:

... fixme

Now, let's create a template for the editing interface. Create a new file, edit.html, as a sibling of index.html, and give it the following contents:


 * there is an "update" link
 * this goes to a bookmark.html-like page, but with the form fields
 * When you submit that, it saves, and redirects you to /

Enabling the built-in admin
...needs to be written

Handling user-uploaded media
So far, our website has only included visual effects created by the site admin. It's time to change that by letting your users upload custom images for tags.

A note about dependencies: For Django's image support to work properly, your system must have the Python Imaging Library. This is available either as "PIL", as "Image", or as "pillow".

Because that introduces a lot of complexity, this tutorial glosses over it. If you are especially excited about this, ask a TA to work with you on it.

Here are the basic steps:

First, you need to install PIL. Follow the instructions here to do so.

Second, you need to modify your virtualenv to permit system site packages to work. To do that, do:

Now you should be able to import Image in your python prompt. If not, talk to a TA.

We will need to enhance the tag model to accept an uploaded file. Open up bookmarks/model.py in your favorite editor. Change the Tag class definition so it looks like this:

You then need to create and execute a schema migration. Then, commit the changed models files and the the new migration files to git.

When that is done, you should add a new form to the tag.html view that lets users upload a new image. You will need to add a corresponding view that processes the form, and then redirects the user back to the tag view.

One final note: Because this section deals with file uploads, it will work inconsistently on Heroku. Heroku does not promise to keep any uploaded files. Your two options would be (1) reconfigure your app to use a different storage engine, for example uploading your images to a service like Amazon S3; or (2) you could use a different hosting service, such as OpenShift, that does not have this behavior.

Installing django-debug-toolbar and what it's useful for
Django's operation can be somewhat opaque. When a page does not show the information you were expecting, there can be a great number of reasons: perhaps some data was not saved to the database, or perhaps the variable name the template was expecting did not match the name you provided in the context from the view.

Django debug toolbar is an open source Django app that gives you more insight into how Django is working. In this section, we will show you how to install it, demonstrate at least one thing it is useful for, and explain at least one gotcha that you should be aware of.

Now is a good time to make sure you are using a Django app that actually works! If your current branch is not something you're confident of, this is a good moment to create a new branch based on known-working code. To do that:

First, we will need to install it. Typically, you would need to add it to requirements.txt for your own project. In the case of the tutorial, we already configured requirements.txt to have it, but we will show you how to add it as if we hadn't.

Open requirements.txt in your favorite text editor. Make sure the following line appears:

Once that is done, you would run the following command. (It is safe to run it now, even though it may not be needed).

This reads requirements.txt and ensures your virtualenv has all the packages installed, downloading and installing them if necessary.

Now that it is available, we need to tell Django to enable it. django-debug-toolbar requires a few adjustments to your settings.py, which you can find in myproject/settings.py. First, look for MIDDLEWARE_CLASSES. Add the following string as the final indented line in the sequence, and make sure there is a comma at the end of the line before it:

Configure the list of IP addresses will be able to see the debug toolbar. To do that, add this to the end of myproject/settings.py:

Finally, make sure it appears in the list of INSTALLED_APPS. If not, added it to the end of that list as follows:

Now that it is installed, stop and start your runserver. Then take a look at http://127.0.0.1:8000/ -- do you see the new toolbar in the top right corner?

If so, great!

We'll show you two neat tricks the debug toolbar can offer you. First, if you click on the SQL box, you will see the page expand into a list of all the SQL queries that your front page executed, and how long they took. If your pages are loading slowly, you may find that you can trim down the number or the complexity of these queries.

Second, if you click Templates, you can see which templates were rendered, and (excitingly) what data was passed to them. This is the simplest way to see what information was passed to the templates. Although the interface is somewhat complicated, it is more helpful than repeatedly reloading a page with different "print" statements in it!

There are two gotchas that one must be aware of when using django-debug-toolbar:


 * 1) By default, it only works when your DEBUG is True. Most of the time, this is a good fit; it means users out on the 'net can't view this advanced interface. (That's because when you "deploy" an app, you are supposed to set DEBUG to False.)
 * 2) It "absorbs" redirects. Any time that your code would generate a redirect in the browser, the debug toolbar "catches" that and lets you see what is going on. This can be great, but it can also get annoying.

You can read more about these features, how to change them, and what else the debug toolbar can do on its official website!

Writing your own tests
Automated testing is a way to run code that verifies your code. There are a few benefits to writing tests:


 * If you wrote the tests before you wrote the code, then you can vividly see that your code worked: the test failed before you wrote the code, and it passes after you wrote it.


 * Having a test suite gives you confidence that a change you are making does not break existing functionality. Since many, many bug fix accidentally break other functionality, this is of huge importance.


 * Having a test suite gives other people confidence that your app does the things it says it does. This is helpful when showing the code to another developer or when listing its features to another stakeholder.

Django has your back here: it comes with a built-in ability to run tests if you write them, and it comes with a test suite of its own.

First, let's take a look at how to run tests. The auth app that we have been using for user login provides a test suite. You can run that test suite as follows:

You'll see a lot of . characters print out, and finally a message like:

Ran 181 tests in 7.764s

OK Destroying test database for alias 'default'...

Congratulations! You can now rest assured that the auth app works properly. (This is good, since you are relying on it!)

In the rest of this section, you'll see how to write your own tests and learn other helpful tips about testing.

Running and writing tests for bookmarks
We can run just the tests for our bookmarks app as follows:

We'll see something like:

Creating test database for alias 'default'... . -- Ran 1 test in 0.000s

OK

It may be surprising that there is even one test to run, given that so far, we have not paid any attention to tests! Take a look at bookmarks/tests.py; in there, you'll see some automatically created hints by Django about how to write new tests, and one simple test.

Let's deconstruct that test right now:

This is a Python class named SimpleTest, inheriting from django's TestCase class. This inheritance is required for Django to discover the test when you run manage.py test. (You can read more about that here.)

It has one method; the fact that it starts with test_ is also essential to it being discovered. Within the method, there is a text description (known as a docstring), and finally, a one-line body.

This body is the heart of the test. It calculates a value, and then asserts that is equal to a stored, known-correct value.

To make this clearer, let's change the test so that it asserts that 1+1 is 3. To do that, replace the class with the following:

Now run the bookmarks test suite:

As the test runs, you see a F printed rather than pleasing, relaxing .. Additionally, once all the tests are over, Django's test runner prints the details of what failed:

=
========================================================= FAIL: test_basic_addition (bookmarks.tests.SimpleTest) -- Traceback (most recent call last): File ".../bookmarks/tests.py", line 16, in test_basic_addition self.assertEqual(1 + 1, 3) AssertionError: 2 != 3

Now that we have a basic understanding of tests, go into bookmarks/tests.py and remove the SimpleTest class entirely.

Adding a new test
Writing tests for a Django app is similar to interacting with the app in the manage.py shell. One key difference is that every test case starts out with a blank database, rather than using your app's database.

Let's add a test for your bookmarks app that verifies the index view: namely, that if there is a bookmark in the database, it gets passed to the template in a context variable called bookmarks.

To do that, open up bookmarks/tests.py and add the following to the end of the file.

This class has just one method, which begins with test_ so that it can be picked-up by the Django test runner.

In the test database, which starts out as blank, there are no users, so we must create one. (You can change that through test fixtures.)

Then we create one bookmark, and ask a the Django test client to load the home page. The test client is like a special, Django-aware web browser. Instead of doing an HTTP request against the actual website, it calls code directly within the Django app. This gives it access to, for example, the template context.

Finally, we extract the bookmark list sent to the template, and we verify that the bookmark we created shows up in the list.

If we run this test, we will see the familiar (if terse) output from the test runner:

Creating test database for alias 'default'... . -- Ran 1 test in 0.000s

OK Destroying test database for alias 'default'...

Hooray! You have written your first test.

Notice that test only validates that we pass the correct data to the template. We do not yet validate that the template actually renders the bookmark! To do that, you would need to test response.body.

One other aspect of testing that can be very helpful is using tests to help you refactor your code. (Refactoring is the process of taking code that works and making it simpler to understand without breaking it.)

In particular, our index view contains some string manipulation to normalize tags. That code, so far, has been somewhat under-specified; we do not really have a statement anywhere of precisely how it should work. One very valid exercise would be to remove that code from index, move it into a function, and then write at few test cases to cover it with tests.

At some point, you may be wondering how you know if you have written enough tests. One answer is by measuring your code's coverage, which is the fraction of your web app that the test suite executes. You can be more confident that the code works properly if it is covered by your test suite. Django does not come with coverage tools built-in, but with the help of django-coverage you can take those measurements. Code coverage is not a perfect way to determine if you have written enough tests, but getting your code to 100$ covered is a solid first goal.

Providing your data as an API
APIs, short for application programming interfaces, are a way for your Django site to provide services to other bits of code.

One common use for APIs is making data available to your own Javascript code. Another common use is making them available for other people to use in their own Python scripts or in Javascript widgets. In this section, you will learn how to add a Django app named tastypie to your project and see how it can easily make your data available.

Deciding on the API we'll provide
Before going forward, we should decide what kind of information we will export, and how we will export it.

For this example, since all the bookmarks on the site are public, we will provide a list of all bookmarks. We will let users of the API filter the bookmarks by the user, if they like.

Adding Tastypie as a dependency
First, we will edit requirements.txt to add a dependency on django-tastypie. To do so, we simply add one line to the end of requirements.txt:

Now that our project depends on it, you can simply run (so long as the virtualenv is activated):

That will download and install all the dependencies for the app, which will bring in tastypie.

Finally, add it to INSTALLED_APPS, toward the end. Within that sequence, add this on a line on its own:

With all those changes made, let's commit:

configure our API via api.py and add it to urls.py
Within bookmarks, create a new file called api.py. Let the contents of that file be:

Tastypie is based on "Resources"; you can read more about it in its own tutorial. In this example, we do just a handful of things:


 * We limit users of the API to only be able to get from you, rather than use other HTTP verbs like PUT and DELETE to upload or modify data. (You can enable PUT and similar modification features yourself, but you will likely want to learn about authorization within Tastypie first. The simplest such mode is to enable SessionAuthentication.)


 * It will make available the queryset provided, which is all the bookmarks.


 * The resource_name setting configures one URL fragment that will be required, namely that it will be available at .../bookmark/.

For this file to be invoked in request processing, we have to tie it into a url. So let's edit urls.py. First, add a new import to the top of the file:

Below the imports, add this on a line of its own:

Within the urlpatterns sequence, add this on a line of its own:

Now your API should be live and on the web! Visit http://127.0.0.1:8000/api/bookmark/?format=json and you should see a machine-readable list of all the bookmarks on your site!

With all that done and working, now is a great time to commit. Run:

and make sure you git add any new files. With that addressed, run:

Filtering, and further directions
Our API lets us filter by any of the fields it returns. So, for example, you can filter the results by user by visiting:

http://localhost:8000/api/bookmark/?user__username=yourself&format=json

Doing this requires no changes to your code. Cool, huh?

One further direction would be to summarize the tags as a list of strings, rather than as a list of tag URIs. You could look into the dehydrate method that Tastypie lets you override.

Further directions

 * Exporting tags

Regular expressions
Text patterns can be encoded as regular expressions. You've already seen them in urls.py. We won't go into great depth here, but we will recommend some resources so you can understand them well, visualize them, and know how to write Python code that takes advantage of them.

First, get to know how they work. To do that, I recommend visualizing them.

Visualizing
To start with that, open up http://www.regexper.com/ and enter in one of the regular expressions we used. For example, enter this: ^$

Then, click the Display button (or hit ENTER on your keyboard). Contrast that with these examples:

^bookmarks\/$

(This is similar to the URL for the bookmarks view. We had to escape the slash character (/) by putting a backslash (\) before it because the regexper.com tool uses JavaScript-esque regular expressions rather than Python regular expressions. Let's ignore that for now.)

You'll see that it matches the exact string of "bookmarks/"; it also constrains things so that the line ends after the word "bookmarks/". If you remove that "$" character, the string "bookmarks/ahoy" would also match.

Executing regular expressions
You can execute regular expressions in a few ways.

First, there is the Python API for that. You can see it in action here:


 * http://codepad.org/SFjk5YzF

You can also use http://regexpal.com/ to execute regular expressions

More reading and resources

 * http://regexpal.com/ has a great Quick Reference that you can find by clicking on those words in the top right of the web page.


 * http://www.diveintopython.net/regular_expressions/street_addresses.html explains more about why regular expressions are nice, and how to use them well.

Learn about relational databases
Django uses a relational database to store your data. In this section, we will explain the very basics of relational databases, give you pointers to more information, and show you how to explore your data.

Within the tutorial, we have used Django's ORM to access our data. That has obscured how the data is actually stored... so now it is time to take a quick look.

SQL is a query language for creating, reading, and modifying databases. SQL queries look something like:

SELECT title FROM bookmarks

You can get access to a SQL shell from within Django by running:

and you can now execute the above query.

These databases are considered relational because the normal way to use them involves using references between tables to avoid repetition of data. In our tutorial, for example, a Bookmark pointed to a User. This is not the only way to build the app; we could have embedded all of the user's information within each Bookmark. That would have a major downside: the various copies of the information could get out of date.

This tutorial used sqlite, a very popular relational database, but there are plethora of others, including:


 * MySQL
 * Postgres
 * Oracle
 * Microsoft SQL Server

Django can be used with any of these. sqlite is convenient because it requires no setup step; however, popular hosting services typically use MySQL or Postgres. sqlite has one major downside -- it cannot support multiple changes to the database made while each other are being made. That limitation is completely OK for an app running on your laptop.

Even though Django's ORM looks just like Python, it does complex work behind the scenes. This work deserves some respect! One common source of slowness in web applications is executing too many SQL queries.

Exploring your app's data
The simplest way to explore your app's data visually is by finding the database.db file and opening it in a graphical browser. On Linux, Mac, or Windows, install Firefox, and then install the https://addons.mozilla.org/en-us/firefox/addon/sqlite-manager/ add-on.

From there, you can open your database and browse around.

Try to run the above SQL query against your database within SQLite Manager.

Other tools

 * Use http://sqlzoo.net/ to learn more about SQL by trying interactively to write queries that solve problems.


 * Use http://sqlfiddle.com/#!7/781d4/1 to try SQL queries within the web browser.