RkBlog

Hardware, programming and astronomy tutorials and reviews.

Profiling Django apps and Python scripts with cProfile and Kcachegrind

Profiling execution time of every part of your Python code in a quick and easy way.

Some time ago I wrote about profiling Django with hotshot with mod_python that is now dead. To profile execution time of Python scripts as well as Django views, tests, or management commands we use cProfile and its wrappers. There are also converters that provide data for KCachegrind. Let us see what we have available to use.

Profiling with cProfile

The simplest way is to run a Python script with cProfile on:

python -m cProfile -o profiling_data.pyprof my_script.py

We will have to install pyprof2calltree to convert cProfile result file to kcachegrind compatible:

pyprof2calltree -i profiling_data.pyprof -k
Selenium tests profile result

Profiling of two Selenium tests where sleep takes most of the time

This solution is good for plain scripts that can be executed and end. In case of Django it can be used to profile tests (see what's the slowest part for example) or management commands. To profile them you have to use local manage.py and not global django-admin command.

There are also simple wrappers like profilestats, which provides a decorator that can be applied to a function (or class based view) and will dump profiling data to a hardcoded file (cachegrind.out.profilestats).

For Django views (and other parts) django-extensions will be much better. Install them, add them to INSTALLED_APPS and then you can start the development server with:

python manage.py runprofileserver --kcachegrind --prof-path=/some/folder/for/logs/

In given folder you will get profiling result files for every view executed during use of that development server.

Getting kcachegrind

Kcachegrind is a part of KDE applications and under Linux it's easy to install it. Under Windows you can use the KDE4 for Windows installer or Qcachegrind. For OSX you can use Kcachegrind in one of Linux/Unix software repository systems as a part of KDE apps. You can also run desktop Linux in a virtual machine.

RkBlog

5 May 2014;

Comment article