utils.media package¶
Media & Thumbnailing¶
This document explains how the utils.media package enables us to serve public and private media (user-uploaded) files. We also give an outline of the workings of our thumbnailing implementation.
Media types¶
We differentiate between two types of media: public and private. Public means that the files can be served without any kind of authentication. Requests for private files have to be checked by Django first before we offload serving the file using [django-sendfile2](https://github.com/moggers87/django-sendfile2).
Public¶
The public files are saved in MEDIA_ROOT/public/. These files can be served by nginx like one would with a regular [MEDIA_ROOT](https://docs.djangoproject.com/en/2.1/ref/settings/#media-root). The files should be served at MEDIA_URL/public/.
Private¶
The private files are saved in MEDIA_ROOT where the public folder is the exception. This is a legacy implementation from our previous thumbnailing code. We could decide to migrate these files to a dedicated private folder in the future. Even though the files are not saved in such a folder they are located at MEDIA_URL/private/. This path should be available to concrexit so that the media files can be served correctly: the signature (more on that below) should be valid and match the path.
Signatures¶
To make sure the private media files stay private we have implemented a signature-based security mechanism based on the idea behind the [Thumbor](https://thumbor.readthedocs.io/en/latest/security.html)’s security implementation. This signature can be extended to contain more information, like we did for our thumbnails.
The value of the signature, which is a dictionary, must at least contain one key: serve_path. This is the path that is used in utils.media.views.private_media() to determine the location of the media file that should be served. If this path matches the path in the url the file will be made available to the user. A second, optional, key is attachment which is used by the sendfile backend to force a download if the value is True.
sig_info = { 'serve_path': f'{settings.MEDIA_ROOT}/<image location>' }
print(signing.dumps(sig_info))
'eyJzZXJ2ZV9wYXRoIjoiL21lZGlhLzxpbWFnZSBsb2NhdGlvbj4ifQ:1gsbnC:QJTqFUWY6HxMBTEIxYPl9V1yf48'
The signature is appended to the location using a query parameter with the key sig and is valid for 3 hours as implemented by utils.media.views._get_image_information.
https://<base url>/media/private/<image location>?sig=eyJzZXJ2ZV9wYXRoIjoiL21lZGlhLzx...
To get the url of a media item you can use utils.media.services.get_media_url(). Never use this to get a url directly from user input!
Thumbnails¶
To make sure we do not have to serve full-size images to users every time they open a page we decided to thumbnail our images. This functionality is provided as a templatetag (utils.templatetags.thumbnail) and service (utils.media.services.get_thumbnail_url()).
Every thumbnail only needs to be generated once, when the thumbnail exists the tools mentioned above will return the direct media url relative to the root of the concrexit instance. If the image is private the url will contain the signature as well.
If the thumbnail does not yet exist the returned url will be the url of the generation function (utils.media.views.generate_thumbnail). This url can then be called by the browser of the user creating the thumbnails on-demand preventing blocking of the page or large workloads when uploading multiple photos at once. Once the thumbnail is generated the user will be redirected to the real image.
The url to the thumbnail generation route is signed with a signature that extends the signature we use to serve private media files. More information about the signature can be found in the next section.
The thumbnailing tools should never be used to create thumbnails from user input directly! They do not protect against directory traversel and assume the path is correct. They are meant to be used with a path from an ImageField or similar.
digraph thumbnails { get[label="get_thumbnail_url"] generate[label="generate_thumbnail"] public[label="serve thumbnail as public media file"] private[label="serve thumbnail as private media file"] get -> generate [label="no thumbnail"] get -> public [label="thumbnail exists & is public"] get -> private [label="thumbnail exists & is private"] generate -> public [label="redirects"] generate -> private [label="redirects"] }Signatures¶
The signature used for the generation of thumbnails extends the signature to load a private media file with keys used for the generation. The signature contains all information required to generate the thumbnail.
{
'visibility': 'public',
'size': '300x300',
'fit': 0,
'path': 'image.jpeg',
'thumb_path': 'thumbnails/300x300_0/image.jpeg',
'serve_path': '/media/public/thumbnails/300x300_0/image.jpeg'
}
The signature protects us against path tampering and thus path traversal and DDoS attacks. We also know that a user can only get a valid signature if they accessed a page providing them with that signature.
Submodules¶
utils.media.services module¶
-
utils.media.services.get_media_url(path, attachment=False)[source]¶ Get the url of the provided media file to serve in a browser. If the file is private a signature will be added. Do NOT use this with user input :param path: the location of the file :param attachment: True if the file is a forced download :return: the url of the media
-
utils.media.services.get_thumbnail_url(path, size, fit=True)[source]¶ Get the thumbnail url of a media file. NEVER use this with user input. If the thumbnail exists this function will return the url of the media file, with signature if necessary. Does it not yet exist a route that executes the
utils.media.views.generate_thumbnail()will be the output. :param path: the location of the file :param size: size of the image :param fit: False to keep the aspect ratio, True to crop :return: direct media url or generate-thumbnail path
utils.media.views module¶
Utility views
-
utils.media.views.generate_thumbnail(request, request_path)[source]¶ Generate thumbnail and redirect user to new location
The thumbnails are generated with this route. Because the thumbnails will be generated in parallel, it will not block page load when many thumbnails need to be generated. After it is done, the user is redirected to the new location of the thumbnail.
- Parameters
request (HttpRequest) – the request
- Returns
HTTP Redirect to thumbnail