Automated testing

From OpenHatch wiki
Revision as of 19:04, 3 February 2012 by imported>Paulproteus (→‎We regularly run Automated Testing)

This is a page about improving or modifying OpenHatch.

We call that "Hacking OpenHatch," and there is a whole category of pages about that.

The purpose of this page is to show you how to write automated tests within the OpenHatch codebase.

If you already know how software testing works, skip to Details specific to OpenHatch.

Tests: An overview

When you run:

python test 

and you'll see a bunch of dots. Dots mean success.

This runs the many tests that are part of the OpenHatch code.

In general, you really should write a test if you add new functionality. This page explains how and when to write new tests and how to run the tests we have.

What a basic test looks like

Imagine this is in mysite/base/

def multiply(x, y):
    return x * y

Then this would be in mysite/base/

import mysite.base.views

class TestMultiplication(django.test.TestCase):
    def test_return_one(self):
        # Make a dictionary that should return 1
                         mysite.base.views.multiply(7, 5))

When a test fails

When a test fails you will see "FAILED" followed by the test_name, along with the Traceback and the failure summary at the end (e.g. FAILED (failures=2, errors=1, skipped=9))

To force a failure, maybe you are just curious to see what it will look like, you can add: self.assertTrue(False) to a test case that you are interested in running.

Getting your local dev OpenHatch set up to run tests

To run tests correctly you'll need to have subversion installed -

$ apt-get install subversion

Then run the full suite of tests --

$ python test

Read the official Django testing guide

The official guide on Django testing is quite good. It says, "The best part [about writing tests for Django code] is, it's really easy."

We use the Django "unit test" style of writing tests.

General testing tips

How to write code that is easy to test

If you are writing a function, make it accept arguments for its data, rather having it calculate the input itself. For example:


def multiply(x, y):
    return x * y

Less good

def multiply(x):
    return x * y

It's okay to rely on things like system settings and database content, but in general if your functions are simpler, they are easier to test.

Details specific to OpenHatch

We regularly run Automated Testing

OpenHatch's Automated Testing is run by Jenkins, with the interface on the virtual machine donated by GPLHost @

Where to write your tests

In general, add tests to the same Django app as you are editing. For example, if you made changes to base/, then add a test in base/

The test files are kind of sprawling. It doesn't really matter where within the file you add your test. I would suggest adding it to the end of the file.

The OpenHatch test case helper class

In mysite/base/ there is a TwillTests class. It offers the following convenience methods:

  • login_with_client
  • login_with_twill

About fixtures

If you inherit from TwillTests, you get some data in your database. You can rely on it.

To run your tests

What app did you write your test in? Let's pretend it was in base:

python test base

To run your tests quickly

The normal test runner uses MySQL, and has to do a bunch of database setup and teardown. If you want the tests to run faster, you can use a different settings file that uses an in-memory SQLite database.

./bin/sqlite_mysite test base

The tests will run about five times faster that way.

To run just a few specific tests

./bin/sqlite_mysite test base.Feed base.Unsubscribe.test_unsubscribe_view

The structure here is app.class.method. So if you want to just run your own new test, you can do it that way.

Mocking and patching

This section is important, but we haven't written it yet. Oops.

Testing with Twill, versus the Django test client

To make a long story short:

The Django test client is good at introspecting how the function worked internally.

Twill tests are good because they let you say "Click on the link called log in".

We should write more about this. Maybe you, dear reader, can say some more.