Security, Sensitive Data Dealing and Pgcrypto in Django

                Security is the most priority thing when we launch our app in production mode for live users, django provides us amazing features to server our API's securely. But if we don't deal with them properly its useless, like NEVER deploy your production code with `DEBUG = True`, because if any function raises an error it dumps a lot of metadata from environment.

                By default Django records the full traceback for the exception raised, each traceback frame’s local variables, and the HttpRequest’s attributes. However, sometimes certain types of information may be too sensitive and thus may not be appropriate to be kept track of, for example a user’s password or credit card number.

Getting start with example on how to deal with sensitive data:

            I have created a project named `sensitivedata` with single app server. The best practice is to keep our all secret data like `API_KEYS`, `SECRET_KEYS`, `S3_BUCKET_KEYS` and other sensitive data in environment varibles, so its not pushed anywhere in repository and directly handled at server.

            In this demo I am keeping project's `SECRET_KEY` and `DATABASE_URL` in environment variables, you can deal with environment variables from either os.environment['VARIABLE_NAME'] or you can use `django-environ` library, right now I am using `django-environ` library.

You can simply install it by pip or easy_install

pip install django-environ

Reading Environment Variables:

To read environment variable

import environ

env = environ.Env()

SECRET_KEY = env('SECRET_KEY')

You can also set some default value in case you dont have that parameter in environment by providing default parameter.

SECRET_KEY = env('SECRET_KEY',default="je817(l*((ozr-n!ldtwz=w#%&0(odq!yx-6p3%&kbfb#=&by$")

And For database:

env.db('DATABASE_URL', default='mysql://root:newpwd@localhost:3306/sensitivedb')

           Now you can export all your environment variables or you can also update settings using the same library to auto read env variables. You can export variables by

export DATABASE_URL='mysql://root:newpwd@localhost:3306/sensitivedb'

export SECRET_KEY=''e817(l*((ozr-n!ldtwz=w#%&0(odq!yx-6p3%&kbfb#=&by$

Dealing with Sensitive Parameters in Request Response Cycle:

           Django offers amazing set of function decorators to help you control which information should be filtered out of error reports in a production environment (DEBUG is set to False): `sensitive_variables()` and `sensitive_post_parameters()`.

If your code handles sensitive information in local variables inside a function, you can mark them as sensitive using `sensitive_variables()` decorator for `GET` requests and `sensitive_post_parameters()` for `POST` requests.

           Let's take an example, I am writing a sample model Profile which has OneToOne Relationship with User model and has `country`, `phone_number` and `passport_number`, where `phone_number` and `passport_number` is sensitive data.

from django.dispatch import receiver

from django.contrib.auth.models import User

from django.db.models.signals import post_save

from phonenumber_field.modelfields import PhoneNumberField

class Profile(models.Model):

    user = models.OneToOneField(User, on_delete=models.CASCADE)

    country = models.CharField(max_length=128, null=True)

    phone_number = PhoneNumberField(null=True)

    passport_number = models.TextField(null=True)

@receiver(post_save, sender=User)

def create_user_profile(sender, instance, created, **kwargs):

    if created:

        Profile.objects.create(user=instance)

Here I have used third party django-phonenumber-field which can be installed from

`pip install django-phonenumber-field`

Once these is installed, you need to add `phonenumber_field` in installed_apps in your settings.py file

INSTALLED_APPS = [

    ....,

    'phonenumber_field',

    ..

]

Now, inside our views.py we need to use the `sensitive_variables()` and `sensitive_post_parameters()` decorator to deal with sensitive data, where we are explicitly describing the sensitive parameters, which will be presented as `********` in case of any exception.

import json

from django.shortcuts import render

from django.http import HttpResponse

from django.views.decorators.debug import (

                                                       sensitive_variables,

                                                       sensitive_post_parameters

                                                                      )

from server.models import Profile

@sensitive_variables('phone_number', 'passport_number')

def getUserProfile(request):

    data = {

        'phone_number':str(request.user.profile.phone_number),

        'passport_number':request.user.profile.passport_number

    }

    return HttpResponse(json.dumps(data), content_type="application/json")

@sensitive_post_parameters('phone_number', 'passport_number')

def updateUserProfile():

    profile = request.user.profile

    profile.phone_number = request.POST['phone_number']

    profile.passport_number = request.POST['passport_number']

    profile.save()

    return HttpResponse("+OK")

     In above code you can see `phone_number` and `passport_number` are explicitly defined as sensitive parameters.

Pgcrypto with Sensitive Data:

     We are encrypting the sensitive data before storing it in our database and decrypting it in our app for further operations. These can be done by writing encyption and decryption helper functions, which can be called when we are making db calls.

In these demo, we have used `django-fernet-fields` library which does these tasks for us, we simply need to install it and then import its predefined EncyptedFields. Also if we are using our own custom field or any third party field that too can be encrypted easily, like in our case we are using `phonenumber_field`, so lets encypt the same fields ie.. `phone_number` and `passport_number`.

     Update our models.py file, where we have imported EncyptedTextField, and updated passport_number field from TextField to EncyptedTextField:

from fernet_fields import EncryptedTextField

class Profile(models.Model):

    user = models.OneToOneField(User, on_delete=models.CASCADE)

    country = models.CharField(max_length=128, null=True)

    phone_number = PhoneNumberField(null=True)

    passport_number = EncryptedTextField(null=True)

Now lets store the data from admin panel and have a look at stored data in database.

:Saving user’s profile details:

         Now, in case if we want to encrypt our own custom field or any third party field like we have `phone_number` field, so that can also be encrypted easily, simply import `EncryptedField` and write a class how we have written below.

Again update server/models.py file, where we have taken extra field `encypted_phone_number`:

from phonenumber_field.modelfields import PhoneNumberField

from fernet_fields import EncryptedTextField, EncryptedField

class PhoneEncryptedField(EncryptedField, PhoneNumberField):

    pass

class Profile(models.Model):

    user = models.OneToOneField(User, on_delete=models.CASCADE)

    country = models.CharField(max_length=128, null=True)

    phone_number = PhoneNumberField(null=True)

    passport_number = EncryptedTextField(null=True)

    encrypted_phone_number = PhoneEncryptedField(null=True, blank=True)

Lets update the details again from admin panel and see the details in database:

:Updating profile details:

:Raw Query Result:

Conclusion:

        Django provides amazing features to deal with sensitive data if used properly and API_KEYS/ SECRET_KEYS can easily be secured by keeping them in environment variables so that it is only available with authorized person also we can easily encrypt sensitive data before storing in database which can be decrypted in app very easily or we can use existing libraries to perform the task like we did using `django-fernet-fields`.

Code can be found at github repository : `https://github.com/pawan3103/sensitivedatadealing`