Continuous integration of Django projects with Jenkins

Check out the new site at https://rkblog.dev.

23 July 2012 Comments

Jenkins is a tool for watching or managing jobs. Either external like cronjobs or executing "build and test" jobs of your Django projects. There is even more, but in this article I'll focus on a basic Jenkins setup - building and testing a Django project. For Django there is django-jenkins that allows an easy integration with Jenkins tools like nice visualization of code coverage, pep8/pylint/pyflakes code violations.

In this article I'll setup a local Jenkins configuration that will look on a folder with the code. In real world Jenkins would watch a repository and execute a job when a new changeset shows up.

Jenkins

On Jenkins web page we can find packages for various OS. We can also download a WAR file for local execution (you will need a JAVA JRE and maybe few other dependencies).

Download the WAR file.
From a terminal launch it with
java -jar jenkins.war
Jenkins will start and will be available at http://localhost:8080/

The main part of the page are jobs (none at first run). On the left we have a menu with few handy elements. "New" can be used to add new jobs. "Manage Jenkins" allows configuring Jenkins and also installing plugins - and we need few for django-jenkins.

For django-jenkins we need two plugins: Violations and Cobertura. For local tests also File System SCM plugin will be useful. After installation you may need to restart Jenkins for them to appear as installed. If the plugin list is empty try to download it by going through tabs or hitting the install button.

Creating a job for a Django project

Click on "New" and create new job. Enter a name and choose "Build a free-style software project". If you already have a similar job you can copy it too.

Next we configure it:

Usually Jenkins would look on a repository for changes, but using File System SCM we can use a plain local folder with a Django project. Good for playing with Jenkins.

Poll SCM will make Jenkins check the given repository for any changes since last build. If there will be new changes it will launch the job. The */30 * * * * means that it will check every 30 minutes.

Usually for build instructions we would give dependency installation and then run all tests we need. In this case I have everything installed on the local system so I just launch tests. manage.py jenkins is a part of django-jenkins (more on that later). It will launch tests and create reports files used by Jenkins.

Cobertura plugin can generate code coverage reports. Set reports/coverage.xml as the path for report. Code executions statistics are configured like so:

Just set reports/junit.xml as the path. Violations pluging will allow yo visualize pep8, pylint/pyflakes violations:

Jenkins is configured. We just need to add django-jenkins to that project and it will work - Jenkins will build the applications and create reports.

Django-Jenkins

We can install it the usual way:

pip install django-jenkins

Then we add it to INSTALLED_APPS:

django_jenkins

Next we define a list of tasks for django-jenkins. The base ones are:

JENKINS_TASKS = (
        'django_jenkins.tasks.with_coverage',
        'django_jenkins.tasks.django_tests',
        'django_jenkins.tasks.run_pep8',
        'django_jenkins.tasks.run_pyflakes',
    )

We also need to install other python packages: pep8, pylint and pyflakes. After that you can test everything executing manage.py jenkins. If it succeeds then the project is configured and ready for action with Jenkins.

By default tests of Django and everything in INSTALLED_APPS will be launched. That's not the best thing for the reports. We can define PROJECT_APPS that will hold a list of our apps for for testing. For example:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.admin',
    'javascript_settings',
    'django_jenkins'
)
PROJECT_APPS = (
    'jsdemo',
    'testapp',
)
INSTALLED_APPS += PROJECT_APPS

What Jenkins can tell us?

In our test configuration Jenkins will check every 30 minutes for changes in the project folder. If there will be any it will launch a build. From a build we can get a lot of useful informations. The main information - did the build succeed. We can also check:

Console: log from the executed tasks. Sometimes it may fail at downloading packages from pypi. You can retrigger a task to repeat it when pypi is online again.
Coverage Report: reports about code coverage with visualization of branches/conditionals coverage.
Violations: This will cover pep8, pylint/pyflakes violations.

You will find more info on django-jenkins Tutorial and Continuous Integration with Jenkins.

RkBlog

Django web framework tutorials, 23 July 2012

Check out the new site at https://rkblog.dev.