Lettuce - scenario based tests for Django and other frameworks
Using Lettuce to test Django applications using expressive scenarios
lettuce is a test system driven by scenarios used in behavior driven development (BDD). The scenarios are described expressively in English and can be read and understood even for non programmers. Test system like lettuce parsers the scenarios and executes required tests for each step.
Lettuce can be used with Django, other web frameworks or scripts. The installation is typical - pip install lettuce. For Django you also have to add 'lettuce.django' to INSTALLED_APPS.
Lettuce tests structure
For an app you want to create Lettuce tests you have to create a sub folder called features. In this folder you then create a pair of files *.py and *.feature (both named identical). For example index.feature and index.py would do. The "feature" file contains scenarios, while ".py" file contains python code executed for steps used in scenarios.Here is a basic .feature file:
Each scenario file starts with a feature description. It's not parsed by Lettuce. The Scenarios are. In our example the scenario says that the main page works. To check it we have two steps - when I access it the server must respond with a 200 response code.
Scenarios may be and usually are longer. They start with Given part and end with Then part that checks the result of code executed in the "Given" part:The ".py" file can look like so:
Each scenario must be mapped to a function. To do that we use a @step decorator that takes the step string as an argument. Variables from the string are mapped via regular expressions. In the example the URL and the response code are mapped that way. There can also be steps run before/after each scenario or before/after all tests.
The "world" object in tests is persistent between each step call. Before "all" we set the Django test client under world.browser, and then in scenario test steps we use it to perform our tests.
To run the tests use python manage.py harvest command. Lettuce by default uses 8000 port for its server and will fail if the development server is using that port. Lettuce will search for tests in every app from INSTALLED_APPS and will execute them when found:
To prevent code duplication of basic or frequently used steps you can use a terrain.py file located in features folder (available for all scenarios in that folder) or in project main folder (available to every application tests).I can put the @befire.all call in such file, as it's always needed:
Lettuce and Django test environment
Due to Google App Engine and other limiting environments Lettuce by default doesn't use the Django test database or its environment. You may find on the net various create/teardown configurations for Lettuce but the don't seem to work nicely with current Django and Lettuce versions. You can create and use test settings file and use it for lettuce tests, or use/wait for the django testserver pull request to be merged. With that change no create/teardown actions are needed.
Using Steven changes I could easily extend the tests to use the database:
In this example I'm creating a "test" user (which could be done better with Factory Boy) and log him in. I'm also using world.response.content to see the server response (HTML page in real application. In this case the view just HttpResponse some text).
Vanilla lettuce would use the database from settings file and would not clean inserted data so you would quite quickly run into integrity errors on the user table.
Splinter - using Selenium with Lettuce
splinter is a handy tool that can replace the Django test Client with a similar object that will use Selenium to do browser based tests:
Selenium tests are handy for acceptance and frontend tests where you can test what user would see. For splinter the tests could look like so:
I had to remove few steps as you can just login a user from the tests.
Lettuce and Django-Jenkins
Django-jenkins can handle Lettuce tests. Just add 'django_jenkins.tasks.lettuce_tests', to JENKINS_TASKS in your settings file. In Jenkins in the job configuration for "Publish JUnit test result report" you have to add "reports/lettuce.xml" (you then should have something like "reports/junit.xml, reports/lettuce.xml").
That should connect the lettuce tests with Jenkins - but check if Jenkins will notice that a lettuce test failed.
More on Lettuce
- Web development fun with Lettuce and Django
- The Biggest Mistakes Django Developers Make When Using Lettuce
- Django Full Stack Testing and BDD with Lettuce and Splinter
- Testing Django - part 2 - lettuce