Django for Designers/Whats next

From OpenHatch wiki

Part 7: Exercises for the reader

Updating and deleting bookmarks (the last two parts of CRUD)

Let's make it possible to delete a bookmark. The changes we will make to support that are to add a delete button next to each bookmark, add a a view that handles that delete button, and hook all that up to urls.py.

To make the delete button appear, edit bookmark.html so that it has the following text, top to bottom. The only change we are making is the addition of a new div toward the bottom.

<li>
    <a class="bookmark-link" href="{{ bookmark.url }}">{% if bookmark.title %}{{ bookmark.title }}{% else %}{{ bookmark.url }}{% endif %}</a>
    <div class="metadata"><span class="author">Posted by {{ bookmark.author }}</span> | <span class="timestamp">{{ bookmark.timestamp|date:"Y-m-d" }}</span>
    {% if bookmark.tag_set.all %}| <span class="tags">
        {% for tag in bookmark.tag_set.all %}
            <a href="{% url 'bookmarks.views.tag' tag.slug %}">{{ tag.slug }}</a></span>
        {% endfor %}
    {% endif %}
    </div>
    {% if request.user.is_authenticated %}
    <div class="actions"><form method="POST" action="{% url bookmarks.views.delete bookmark.id %}>
        {% csrf_token %}
        <input type="submit" value="Delete">
    </form></div>{% endif %}

We added a very dense block there. Here is an explanation, piece-by-piece:

  • It only renders if there is a user logged-in.
  • It looks for a URL corresponding to a view (that we have not yet written) called bookmarks.delete.
  • It passes the bookmark ID to that view as a URL parameter.
  • It provides a CSRF token, and a submit button labeled Delete.

To make that work, the next step to to take is to write our view. Open up bookmarks/views.py and add the following to the end of the file:

def delete(request, bookmark_id):
    if request.method == 'POST':
        b = get_object_or_404(Bookmark, pk=int(bookmark_id))
        b.delete()
    return redirect(index)

Namely, if the request is a POST, we try to find the corresponding bookmark and delete it. No matter what, we redirect back to the start.

Finally, we need to adjust urls.py to be aware of this view. To do that, open up urls.py in your editor, and add this line anywhere within the urlpatterns sequence (for example, underneath the final current URL pattern):

    url(r'^delete/(\d+)/$', 'bookmarks.views.delete'),

(The (\d+) means, "any sequence of digits, 1 or longer, captured." Capturing it permits the variable to be passed to the Python view function.)

Save all of that, and now try loading up the home page. If you see a bookmark you want to delete, click the delete button. What should appear to happen is that the page reloads itself, and that bookmark is now gone.

Note that, at the moment, any user can delete any other user's bookmarks. If you want things to work differently, you'll need to add a check to the delete function in views.py.

To be able to update bookmarks, we will need to create add an update link, make it go to a page where bookmarks can be edited, and then let that page be saved. (If we wanted to make this slicker, the first step would be to put the edit page in a form using e.g. jQuery UI.)

We already have a form that lets us enter bookmarks, and we can re-use it as the editing area. As a reminder, open index.html and notice the following fragment.

{% block bookmark_widget %}
    {% if request.user %}
    <div id="new-bookmark-widget">
        <form method="post">
        {% csrf_token %}
        <h3>Bookmark</h3>
        {{ form.as_p }}
        <p><button id="new-bookmark-submit">Submit</button>Submit</button>
        </form>
    </div>
    {% endif %}
{% endblock %}

Within that fragment, make one change. Change this:

        <form method="post">

to this:

        <form method="post" action="{% url bookmarks.views.index %}">

That way, it POSTs to the index view.

We will want the same widget in our new page. So we will extend index.html rather than extending the base. Create a new file called edit.html with the following contents:

{% extends 'index.html' %}

{% block subheader %}Edit bookmark{% endblock %}

{% block content %}{% endblock %}

This way, we empty out the content section, but keep the rest of the page layout intact. (It would probably be even better to extract the bookmark_widget to a separate block, and then have edit.html extend base.html instead of index. But hey, this works!)

Make sure to inform git you'll want that file to be part of the next commit:

# in django-for-designers/myproject
$ git add bookmarks/templates/edit.html

We'll need a view to render that form and to accept changes. So, open up views.py and add a new function to the end:

def edit(request, bookmark_id):
    b = get_object_or_404(Bookmark, pk=int(bookmark_id))
    context = {
        'form': BookmarkForm(instance=b),
    }
    return render(request, 'edit.html', context)

In this snippet, we grab a bookmark by ID number instantiate a Django form based on, and pass that to the template.

Intriguingly, we do not need any special code to handle the submitted form. If it gets submitted to the index view, the existing machinery will handle it properly!

We still need to inform urls.py about the new view. To do that, open urls.py and add this line as the final line in the urlpatterns sequence:

    url(r'^edit/(\d+)/$', 'bookmarks.views.edit'),

We do also need to change the base template so that it links to our edit functionality. Open base.html and find the following:

    {% if request.user.is_authenticated %}
    <div class="actions"><form method="POST" action="{% url bookmarks.views.delete bookmark.id %}>
        {% csrf_token %}
        <input type="submit" value="Delete">
    </form></div>{% endif %}

and replace it with:

    {% if request.user.is_authenticated %}
    <div class="actions"><form method="POST" action="{% url bookmarks.views.delete bookmark.id %}>
        {% csrf_token %}
        <input type="submit" value="Delete">
    </form> | <a href="{% url bookmarks.views.edit bookmark.id %}">edit</a></div>{% endif %}

Congratulations! You can now edit your bookmarks.

You'll do well to commit at this point.

# in django-for-designers/myproject
$ git commit -a -m 'Update and delete functionality'


Handling user-uploaded media

So far, our website has only included visual effects created by the site admin. It's time to change that by letting your users upload custom images for tags.

A note about dependencies: For Django's image support to work properly, your system must have the Python Imaging Library. This is available either as "PIL", as "Image", or as "pillow".

Because that introduces a lot of complexity, this tutorial glosses over it. If you are especially excited about this, ask a TA to work with you on it.

Here are the basic steps:

First, you need to install PIL. Follow the instructions here to do so.

Second, you need to modify your virtualenv to permit system site packages to work. To do that, do:

$ virtualenv --system-site-packages .

Now you should be able to import Image in your python prompt. If not, talk to a TA.

We will need to enhance the tag model to accept an uploaded file. Open up bookmarks/model.py in your favorite editor. Change the Tag class definition so it looks like this:

class Tag(models.Model):
    bookmark = models.ManyToManyField(Bookmark)
    slug = models.CharField(max_length=50, unique=True)
    image = models.ImageField()

You then need to create and execute a schema migration. Then, commit the changed models files and the the new migration files to git.

When that is done, you should add a new form to the tag.html view that lets users upload a new image. You will need to add a corresponding view that processes the form, and then redirects the user back to the tag view.

One final note: Because this section deals with file uploads, it will work inconsistently on Heroku. Heroku does not promise to keep any uploaded files. Your two options would be (1) reconfigure your app to use a different storage engine, for example uploading your images to a service like Amazon S3; or (2) you could use a different hosting service, such as OpenShift, that does not have this behavior.

Installing django-debug-toolbar and what it's useful for

Django's operation can be somewhat opaque. When a page does not show the information you were expecting, there can be a great number of reasons: perhaps some data was not saved to the database, or perhaps the variable name the template was expecting did not match the name you provided in the context from the view.

Django debug toolbar is an open source Django app that gives you more insight into how Django is working. In this section, we will show you how to install it, demonstrate at least one thing it is useful for, and explain at least one gotcha that you should be aware of.

Now is a good time to make sure you are using a Django app that actually works! If your current branch is not something you're confident of, this is a good moment to create a new branch based on known-working code. To do that:

# in django-for-designers
$ git branch my-debug-toolbar-work origin/pre-part-5
$ git checkout my-debug-toolbar-work

First, we will need to install it. Typically, you would need to add it to requirements.txt for your own project. In the case of the tutorial, we already configured requirements.txt to have it, but we will show you how to add it as if we hadn't.

Open requirements.txt in your favorite text editor. Make sure the following line appears:

django-debug-toolbar

Once that is done, you would run the following command. (It is safe to run it now, even though it may not be needed).

# in django-for-designers
$ pip install -r requirements.txt

This reads requirements.txt and ensures your virtualenv has all the packages installed, downloading and installing them if necessary.

Now that it is available, we need to tell Django to enable it. django-debug-toolbar requires a few adjustments to your settings.py, which you can find in myproject/settings.py. First, look for MIDDLEWARE_CLASSES. Add the following string as the final indented line in the sequence, and make sure there is a comma at the end of the line before it:

    'debug_toolbar.middleware.DebugToolbarMiddleware',

Configure the list of IP addresses will be able to see the debug toolbar. To do that, add this to the end of myproject/settings.py:

INTERNAL_IPS=('127.0.0.1',)

Finally, make sure it appears in the list of INSTALLED_APPS. If not, added it to the end of that list as follows:

    'debug_toolbar',

Now that it is installed, stop and start your runserver. Then take a look at http://127.0.0.1:8000/ -- do you see the new toolbar in the top right corner?

If so, great!

We'll show you two neat tricks the debug toolbar can offer you. First, if you click on the SQL box, you will see the page expand into a list of all the SQL queries that your front page executed, and how long they took. If your pages are loading slowly, you may find that you can trim down the number or the complexity of these queries.

Second, if you click Templates, you can see which templates were rendered, and (excitingly) what data was passed to them. This is the simplest way to see what information was passed to the templates. Although the interface is somewhat complicated, it is more helpful than repeatedly reloading a page with different "print" statements in it!

There are two gotchas that one must be aware of when using django-debug-toolbar:

  1. By default, it only works when your DEBUG is True. Most of the time, this is a good fit; it means users out on the 'net can't view this advanced interface. (That's because when you "deploy" an app, you are supposed to set DEBUG to False.)
  2. It "absorbs" redirects. Any time that your code would generate a redirect in the browser, the debug toolbar "catches" that and lets you see what is going on. This can be great, but it can also get annoying.

You can read more about these features, how to change them, and what else the debug toolbar can do on its official website!

Writing your own tests

Automated testing is a way to run code that verifies your code. There are a few benefits to writing tests:

  • If you wrote the tests before you wrote the code, then you can vividly see that your code worked: the test failed before you wrote the code, and it passes after you wrote it.
  • Having a test suite gives you confidence that a change you are making does not break existing functionality. Since many, many bug fix accidentally break other functionality, this is of huge importance.
  • Having a test suite gives other people confidence that your app does the things it says it does. This is helpful when showing the code to another developer or when listing its features to another stakeholder.

Django has your back here: it comes with a built-in ability to run tests if you write them, and it comes with a test suite of its own.

First, let's take a look at how to run tests. The auth app that we have been using for user login provides a test suite. You can run that test suite as follows:

# in django-for-designers/myproject
$ python manage.py test auth

You'll see a lot of . characters print out, and finally a message like:

Ran 181 tests in 7.764s

OK
Destroying test database for alias 'default'...

Congratulations! You can now rest assured that the auth app works properly. (This is good, since you are relying on it!)

In the rest of this section, you'll see how to write your own tests and learn other helpful tips about testing.

Running and writing tests for bookmarks

We can run just the tests for our bookmarks app as follows:

# in django-for-designers/myproject
$ python manage.py test bookmarks

We'll see something like:

Creating test database for alias 'default'...
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

It may be surprising that there is even one test to run, given that so far, we have not paid any attention to tests! Take a look at bookmarks/tests.py; in there, you'll see some automatically created hints by Django about how to write new tests, and one simple test.

Let's deconstruct that test right now:

from django.test import TestCase


class SimpleTest(TestCase):
    def test_basic_addition(self):
        """
        Tests that 1 + 1 always equals 2.
        """
        self.assertEqual(1 + 1, 2)

This is a Python class named SimpleTest, inheriting from django's TestCase class. This inheritance is required for Django to discover the test when you run manage.py test. (You can read more about that here.)

It has one method; the fact that it starts with test_ is also essential to it being discovered. Within the method, there is a text description (known as a docstring), and finally, a one-line body.

This body is the heart of the test. It calculates a value, and then asserts that is equal to a stored, known-correct value.

To make this clearer, let's change the test so that it asserts that 1+1 is 3. To do that, replace the class with the following:

from django.test import TestCase


class SimpleTest(TestCase):
    def test_basic_addition(self):
        """
        Tests that 1 + 1 always equals 2.
        """
        self.assertEqual(1 + 1, 3)

Now run the bookmarks test suite:

# in django-for-designers/myproject
$ python manage.py test bookmarks

As the test runs, you see a F printed rather than pleasing, relaxing .. Additionally, once all the tests are over, Django's test runner prints the details of what failed:

======================================================================
FAIL: test_basic_addition (bookmarks.tests.SimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../bookmarks/tests.py", line 16, in test_basic_addition
    self.assertEqual(1 + 1, 3)
AssertionError: 2 != 3

Now that we have a basic understanding of tests, go into bookmarks/tests.py and remove the SimpleTest class entirely.

Adding a new test

Writing tests for a Django app is similar to interacting with the app in the manage.py shell. One key difference is that every test case starts out with a blank database, rather than using your app's database.

Let's add a test for your bookmarks app that verifies the index view: namely, that if there is a bookmark in the database, it gets passed to the template in a context variable called bookmarks.

To do that, open up bookmarks/tests.py and add the following to the end of the file.

from django.contrib.auth.models import User
from bookmarks.models import Bookmark

class BookmarkViewTest(TestCase):
    def test_bookmark_shows_up(self):
        # Create a user to own the bookmark
        me = User.objects.create(username='me')

        # Create a sample bookmark
        mark = Bookmark.objects.create(user=me, title='Title of the song', url='http://example.com/')

        # Visit the home page
        c = Client()
        response = c.get('/')

        # Make sure it has the bookmark data
        as_sent_to_template = response.context['bookmarks']
        self.assertTrue(mark in as_sent_to_template)

This class has just one method, which begins with test_ so that it can be picked-up by the Django test runner.

In the test database, which starts out as blank, there are no users, so we must create one. (You can change that through test fixtures.)

Then we create one bookmark, and ask a the Django test client to load the home page. The test client is like a special, Django-aware web browser. Instead of doing an HTTP request against the actual website, it calls code directly within the Django app. This gives it access to, for example, the template context.

Finally, we extract the bookmark list sent to the template, and we verify that the bookmark we created shows up in the list.

If we run this test, we will see the familiar (if terse) output from the test runner:

# in django-for-designers/myproject
$ python manage.py test bookmarks
Creating test database for alias 'default'...
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
Destroying test database for alias 'default'...

Hooray! You have written your first test.

Notice that test only validates that we pass the correct data to the template. We do not yet validate that the template actually renders the bookmark! To do that, you would need to test response.body.

One other aspect of testing that can be very helpful is using tests to help you refactor your code. (Refactoring is the process of taking code that works and making it simpler to understand without breaking it.)

In particular, our index view contains some string manipulation to normalize tags. That code, so far, has been somewhat under-specified; we do not really have a statement anywhere of precisely how it should work. One very valid exercise would be to remove that code from index, move it into a function, and then write at few test cases to cover it with tests.

At some point, you may be wondering how you know if you have written enough tests. One answer is by measuring your code's coverage, which is the fraction of your web app that the test suite executes. You can be more confident that the code works properly if it is covered by your test suite. Django does not come with coverage tools built-in, but with the help of django-coverage you can take those measurements. Code coverage is not a perfect way to determine if you have written enough tests, but getting your code to 100$ covered is a solid first goal.

Providing your data as an API

APIs, short for application programming interfaces, are a way for your Django site to provide services to other bits of code.

One common use for APIs is making data available to your own Javascript code. Another common use is making them available for other people to use in their own Python scripts or in Javascript widgets. In this section, you will learn how to add a Django app named tastypie to your project and see how it can easily make your data available.

Deciding on the API we'll provide

Before going forward, we should decide what kind of information we will export, and how we will export it.

For this example, since all the bookmarks on the site are public, we will provide a list of all bookmarks. We will let users of the API filter the bookmarks by the user, if they like.

Adding Tastypie as a dependency

First, we will edit requirements.txt to add a dependency on django-tastypie. To do so, we simply add one line to the end of requirements.txt:

django-tastypie

Now that our project depends on it, you can simply run (so long as the virtualenv is activated):

# in django-for-designers
$ pip install -r requirements.txt

That will download and install all the dependencies for the app, which will bring in tastypie.

Finally, add it to INSTALLED_APPS, toward the end. Within that sequence, add this on a line on its own:

    'tastypie',

With all those changes made, let's commit:

# in django-for-designers
$ git commit -a -m 'Adding dependency on tastypie'

configure our API via api.py and add it to urls.py

Within bookmarks, create a new file called api.py. Let the contents of that file be:

from tastypie.resources import ModelResource
from bookmark.models import Bookmark


class BookmarkResource(ModelResource):
    class Meta:
        allowed_methods = ['get']
        queryset = Bookmark.objects.all()
        resource_name = 'bookmark'

Tastypie is based on "Resources"; you can read more about it in its own tutorial. In this example, we do just a handful of things:

  • We limit users of the API to only be able to get from you, rather than use other HTTP verbs like PUT and DELETE to upload or modify data. (You can enable PUT and similar modification features yourself, but you will likely want to learn about authorization within Tastypie first. The simplest such mode is to enable SessionAuthentication.)
  • It will make available the queryset provided, which is all the bookmarks.
  • The resource_name setting configures one URL fragment that will be required, namely that it will be available at .../bookmark/.

For this file to be invoked in request processing, we have to tie it into a url. So let's edit urls.py. First, add a new import to the top of the file:

from bookmarks.api import BookmarkResource

Below the imports, add this on a line of its own:

bookmark_resource = BookmarkResource()

Within the urlpatterns sequence, add this on a line of its own:

    (r'^api/', include(bookmark_resource.urls)),

Now your API should be live and on the web! Visit http://127.0.0.1:8000/api/bookmark/?format=json and you should see a machine-readable list of all the bookmarks on your site!

With all that done and working, now is a great time to commit. Run:

# in django-for-designers
$ git status

and make sure you git add any new files. With that addressed, run:

# in django-for-designers
$ git commit -a -m 'Added an API with Tastypie'

Filtering, and further directions

Our API lets us filter by any of the fields it returns. So, for example, you can filter the results by user by visiting:

http://localhost:8000/api/bookmark/?user__username=yourself&format=json

Doing this requires no changes to your code. Cool, huh?

One further direction would be to summarize the tags as a list of strings, rather than as a list of tag URIs. You could look into the dehydrate method that Tastypie lets you override.

Further directions

  • Exporting tags

Regular expressions

Text patterns can be encoded as regular expressions. You've already seen them in urls.py. We won't go into great depth here, but we will recommend some resources so you can understand them well, visualize them, and know how to write Python code that takes advantage of them.

First, get to know how they work. To do that, I recommend visualizing them.

Visualizing

To start with that, open up http://www.regexper.com/ and enter in one of the regular expressions we used. For example, enter this:

^$

Then, click the Display button (or hit ENTER on your keyboard). Contrast that with these examples:

^bookmarks\/$

(This is similar to the URL for the bookmarks view. We had to escape the slash character (/) by putting a backslash (\) before it because the regexper.com tool uses JavaScript-esque regular expressions rather than Python regular expressions. Let's ignore that for now.)

You'll see that it matches the exact string of "bookmarks/"; it also constrains things so that the line ends after the word "bookmarks/". If you remove that "$" character, the string "bookmarks/ahoy" would also match.

Executing regular expressions

You can execute regular expressions in a few ways.

First, there is the Python API for that. You can see it in action here:

You can also use http://regexpal.com/ to execute regular expressions

More reading and resources

  • http://regexpal.com/ has a great Quick Reference that you can find by clicking on those words in the top right of the web page.


Learn about relational databases

Django uses a relational database to store your data. In this section, we will explain the very basics of relational databases, give you pointers to more information, and show you how to explore your data.

Within the tutorial, we have used Django's ORM to access our data. That has obscured how the data is actually stored... so now it is time to take a quick look.

SQL is a query language for creating, reading, and modifying databases. SQL queries look something like:

SELECT title FROM bookmarks

You can get access to a SQL shell from within Django by running:

# in django-for-designers/myproject
$ python manage.py dbshell

and you can now execute the above query.

These databases are considered relational because the normal way to use them involves using references between tables to avoid repetition of data. In our tutorial, for example, a Bookmark pointed to a User. This is not the only way to build the app; we could have embedded all of the user's information within each Bookmark. That would have a major downside: the various copies of the information could get out of date.

This tutorial used sqlite, a very popular relational database, but there are plethora of others, including:

  • MySQL
  • Postgres
  • Oracle
  • Microsoft SQL Server

Django can be used with any of these. sqlite is convenient because it requires no setup step; however, popular hosting services typically use MySQL or Postgres. sqlite has one major downside -- it cannot support multiple changes to the database made while each other are being made. That limitation is completely OK for an app running on your laptop.

Even though Django's ORM looks just like Python, it does complex work behind the scenes. This work deserves some respect! One common source of slowness in web applications is executing too many SQL queries.

Exploring your app's data

The simplest way to explore your app's data visually is by finding the database.db file and opening it in a graphical browser. On Linux, Mac, or Windows, install Firefox, and then install the https://addons.mozilla.org/en-us/firefox/addon/sqlite-manager/ add-on.

From there, you can open your database and browse around.

Try to run the above SQL query against your database within SQLite Manager.

Other tools

  • Use http://sqlzoo.net/ to learn more about SQL by trying interactively to write queries that solve problems.