RkBlog

Hardware, programming and astronomy tutorials and reviews.

Temporary files in Django for tests and on the fly file manipulation

Using InMemoryUploadedFile in Django applications for file manipulation

Sometimes you need to do some operations on files on the fly in the code and place the result for form validation or save it with a model instance. When you want to write forms tests files may also be needed. To avoid using extra static files for tests you can use some temporary files. All those (and more) can be done with in-memory / temporary file-like objects. On the web there are StringIO or ContentFile usage examples but they wont work always - or as good as Django InMemoryUploadedFile object would. Here is an example of InMemoryUploadedFile usage for tests and other similar cases.

InMemoryUploadedFile is a Django object, used in form file uploads, but it also can be used on its own. Using StringIO.StringIO as a temporary file with recent Django versions may results in exceptions when Django tries to call a methods that doesn't exist. Using ContentFile on a for example image data may result in broken image file. InMemoryUploadedFile is a Django friendly solution for such file operations. I used it in tests and also in thumbnail generation in ckeditor before saving the result via Django File Storage API.

An example usage would look like so:
import Image
import StringIO

def get_temporary_text_file():
    io = StringIO.StringIO()
    io.write('foo')
    text_file = InMemoryUploadedFile(io, None, 'foo.txt', 'text', io.len, None)
    text_file.seek(0)
    return text_file

def get_temporary_image():
    io = StringIO.StringIO()
    size = (200,200)
    color = (255,0,0,0)
    image = Image.new("RGBA", size, color)
    image.save(io, format='JPEG')
    image_file = InMemoryUploadedFile(io, None, 'foo.jpg', 'jpeg', io.len, None)
    image_file.seek(0)
    return image_file

StringIO.StringIO() is a file-like class allowing to mock real file objects. When creating a temporary text file I use "write" method - like in normal file objects. For temporary image PIL saves the image to the StringIO object. When you have the content in StringIO or in other object you can wrap it around InMemoryUploadedFile. This class takes few arguments: file (file object), field name (when used in forms), file name, mime type, size, charset. When using stand-alone the field name can be "skipped" with None. The same can be done for charset. Setting the file to its start with seek(0) prevents some issues with Django recognizing it as an empty file (Storage API, file upload).

Usage

InMemoryUploadedFile can be used to put a file from some uncommon source and then use it in forms validation, model instances or tests. When using non-local storage apis it can help avoiding writing on local storage before sending files to the storage backend (storage API). In tests such temporary files may be used to make a form pass or fail with an error depending on your needs - and without adding extra static files for that job.

Application tests example

I have a basic "testapp" application with a mode:
from django.db import models

class TestModel(models.Model):
    image = models.ImageField(upload_to='test/', blank=True, verbose_name='Test')
And a view:
from django.views.generic.edit import CreateView

from testapp.models import *

class UploadImageView(CreateView):
    model = TestModel
    success_url = '/'

upload_image = UploadImageView.as_view()
Followed by a form - template:
<form action="./" method="post" enctype="multipart/form-data">
    {% csrf_token %}
    <table>
        {{ form }}
    </table>
    <input type="submit" value="Add" />
</form>
To test this form such tests could be used:
import Image
import StringIO

from django.core.files.uploadedfile import InMemoryUploadedFile
from django.core.urlresolvers import reverse
from django.test import TestCase

def get_temporary_text_file():
    io = StringIO.StringIO()
    io.write('foo')
    text_file = InMemoryUploadedFile(io, None, 'foo.txt', 'text', io.len, None)
    text_file.seek(0)
    return text_file


def get_temporary_image():
    io = StringIO.StringIO()
    size = (200,200)
    color = (255,0,0,0)
    image = Image.new("RGBA", size, color)
    image.save(io, format='JPEG')
    image_file = InMemoryUploadedFile(io, None, 'foo.jpg', 'jpeg', io.len, None)
    image_file.seek(0)
    return image_file


class ImageUploadViewTests(TestCase):
    def test_if_form_submits(self):
        test_image = get_temporary_image()
        response = self.client.post(reverse('upload-image'), {'image': test_image})
        self.assertEqual(302, response.status_code)

    def test_if_form_fails_on_text_file(self):
        test_file = get_temporary_text_file()
        response = self.client.post(reverse('upload-image'), {'image': test_file})
        self.assertEqual(200, response.status_code)
        error_message = 'Upload a valid image. The file you uploaded was either not an image or a corrupted image.'
        self.assertFormError(response, 'form', 'image', error_message)
When you fire those tests both will pass - when using image as the file the form will pass, and for text file it will return an error.
RkBlog

17 July 2012;

Comment article