Web Services in Python - Part III - String Resources and Some More Web Methods

For this phase of the project we'll be altering the way we pass error messages to the client. We'll still be using our ClientException class but rather than storing the strings directly in our scripts we're going to create a strings resource file and access each one using an alias. This will not only allow us to reuse duplicate error messages should the need arise but bring us one step closer to multi-lingual support in our application. Let's take a look at the resource manager.

utilities/resources.py


import os
import uuid
import json

class ResourceHandler(object):
    _instance = None

    @classmethod
    def _initialize(cls):
        if not cls._instance is None:
            return

        strings_file = os.path.dirname(os.path.realpath(__file__)) + '/strings.res'
        cls._instance = { 'strings_res': {} }

        if not os.path.exists(strings_file):
            print 'WARNING: No strings resource file found.'
            return

        try:
            stringsStr = open(strings_file).read()
        except:
            raise Exception('Failed to open strings resource file %r.' % (strings_file))

        try:
            strings = json.loads(stringsStr)
        except:
            raise Exception('Failed to parse strings file %r.' % (strings_file))

        cls._instance['strings_res'] = strings

The initialization method will be responsible for loading any resource files we wish to use. For now all we have is our aptly named strings.res file which will allow us to keep a comprehensive list of all literals used by the application.

    @classmethod
    def getString(cls, id):
        ResourceHandler._initialize()
        strings_res = cls._instance['strings_res']
        try:
            uid = uuid.UUID(id)
        except
            uid = None

        if uid is None:
            uid = uuid.UUID(strings_res['aliases'][id])

        id = str(uid)
        string = strings_res['strings'][id]

        return { 'number': id, 'string': string }

To grab the string we simply make a call to this getString method and either pass in the UUID associated with the string or an alias for easier reading. Let's take a look at our strings resource file for this phase of the project.

utilties/strings.res


{
    "strings": {
        "df23e710-af3c-40bc-9681-02c48cedd3e4""User name must be specified."
        "665608f9-4181-417f-aabd-8fc444188a13""No application with id {0} found."
        "f732479c-78c3-4c45-b9d1-62755bee5c8c""Username must be alphanumeric and may include only dashes and underscores."
        "c2e4bb07-14bf-4353-b3b3-4e227c578fce""The passport application is read only."
        "1057b9fa-b800-401f-b467-da15a885d5f4""Incorrect username or password."
        "90f49ba4-8b23-4eef-a9ee-40b704bdd8dd""Could not find application '{0}'."
        "78a7782f-1598-4075-912a-97eba803b605""Application name must be specified."
        "8f18fc32-325b-4edc-a19e-4ddcd8171448""Application name must be alphanumeric and may include only dashes and underscores."
        "d3e1f219-a49d-4fa1-a4d9-066f3908bd4a""Application with name '{0}' already exists."
        "924f43c1-d365-4a63-8d42-2fd010dfc30c""Root user is read only."
        "3d06eba1-1f3a-4df4-a869-01205d06145f""The value {0} is not a valid UUID."
        "1c99f90e-9db5-49ba-be40-be6c6eccea25""The value {0} is not a valid integer."
        "80f1dbfe-d9b5-4adf-aff1-270a35904fcd""No user with id {0} found."
        "d0546dcd-9552-4ac4-850c-160790c037d5""User with username {0} already exists."
        "affd5bb6-9c1c-448d-8edd-59b334d9384c""Password value must be supplied."
        "307c61e3-6b78-43c7-9504-e150d0128397""Not authorized."
        "8335f93d-d317-4846-959f-fed9065c8c16""Stronger password required."
    }, 
    "aliases": {
        "PASSPORT_IS_READ_ONLY""c2e4bb07-14bf-4353-b3b3-4e227c578fce"
        "USER_NAME_REQUIRED""df23e710-af3c-40bc-9681-02c48cedd3e4"
        "DUPLICATE_USER_NAME""d0546dcd-9552-4ac4-850c-160790c037d5"
        "WEAK_PASSWORD""8335f93d-d317-4846-959f-fed9065c8c16"
        "ROOT_IS_READ_ONLY""924f43c1-d365-4a63-8d42-2fd010dfc30c"
        "DUPLICATE_APP_NAME""d3e1f219-a49d-4fa1-a4d9-066f3908bd4a"
        "PASSWORD_REQUIRED""affd5bb6-9c1c-448d-8edd-59b334d9384c"
        "USER_ID_NOT_FOUND""80f1dbfe-d9b5-4adf-aff1-270a35904fcd"
        "INVALID_INTEGER""1c99f90e-9db5-49ba-be40-be6c6eccea25"
        "INVALID_USER_NAME""f732479c-78c3-4c45-b9d1-62755bee5c8c"
        "INVALID_UUID""3d06eba1-1f3a-4df4-a869-01205d06145f"
        "APP_ID_NOT_FOUND""665608f9-4181-417f-aabd-8fc444188a13"
        "APP_NAME_NOT_FOUND""90f49ba4-8b23-4eef-a9ee-40b704bdd8dd"
        "INVALID_APPLICATION_NAME""8f18fc32-325b-4edc-a19e-4ddcd8171448"
        "APP_NAME_REQUIRED""78a7782f-1598-4075-912a-97eba803b605"
        "INVALID_CREDENTIALS""1057b9fa-b800-401f-b467-da15a885d5f4"
        "NOT_AUTHORIZED""307c61e3-6b78-43c7-9504-e150d0128397"
    }
}

Obviously it would be difficult to keep track of our strings in code using a UUID so it's possible to refer to them using aliases. Having a UUID will allow us to merge these literals with other projects down the road should the need arise. Let's take a look at how we'll be using this in our BaseController class.

controllers/basecontroller.py


# imports

from ..utilities.resources import ResourceHandler

class BaseController(object):

    # ...

    def getException(self, id, source, value = None, parameters = None):
        string_info = ResourceHandler.getString(id)

        string = string_info['string']
        number = string_info['number']

        if not parameters is None:
            string = string.format(parameters)
        return ClientException(source = source, 
                    value = value,
                    message = string,
                    number = number)

To save ourselves some keystrokes we're going to use a getException method which will grab the appropriate string from the ResourceHandler and return the constructed ClientException. For now this will be the only location where we'll be making a call to the ResourceHandler. Next, in order to allow the creation of applications We'll need to make some changes to our Storm mapping classes.

models/data.py


# Some new imports
import random
import string

class Application(Storm):
    __storm_table__ = 'application_info'
    applicationid = Int(primary=True)
    name = Unicode()
    users = ReferenceSet(   applicationid,
                'ApplicationUser.applicationid',
                'ApplicationUser.userid',
                'User.userid')
    def __init__(self, name):
        self.name = name

    @staticquerymethod
    def create(store, name):
        result = store.add(Application(name))
        store.commit()
        return result

    @staticquerymethod
    def get(store, id):
        return store.get(Application, id)

    @staticquerymethod
    def getAll(store):
        return store.find(Application, True)

    @staticquerymethod
    def findByName(store, name):
        return store.find(Application, Application.name.like(name)).one()

    @querymethod
    def save(store, self):
        store.commit()

In addition to our already existing findByName method the Application class we now have the ability to create new and save altered application entities. We now also have the ability to retrieve an application by it's id number. Our User class will have many similar changes.

class User(Storm):
    __storm_table__ = 'user_info'
    userid = Int(primary=True)
    uniqueid = UUID()
    username = Unicode()
    password = Unicode()
    salt = Unicode()
    email = Unicode()
    emailverified = Int()
    active = Int()
    challenges = ReferenceSet(userid, 'Challenge.userid')

    def __init__(self, username, email=u'', emailverified=1, active=1):
        self.username = username
        self.email = email
        self.emailverified = emailverified
        self.active = active

    @staticquerymethod
    def create(store, username, password, email=u'', emailverified=1, active=1):
        result = store.add(User(username, email, emailverified, active))
        store.commit()
        result.setPassword(password)
        return result

    @staticquerymethod
    def get(store, id):
        return store.get(User, id)

    @staticquerymethod
    def getAll(store):
        return store.find(User, True)

    @staticquerymethod
    def findByUsername(store, username):
        return store.find(User, User.username == username).one()

    @staticquerymethod
    def findByUniqueId(store, id):
        return store.find(User, User.uniqueid == id).one()

    @querymethod
    def setPassword(store, self, password):
        self.salt = unicode(''.join(random.choice(string.ascii_uppercase + \
            string.ascii_lowercase + string.digits) for x in range(20)))
        self.password = unicode(SHA256.new(password + self.salt).hexdigest())
        store.commit()

    def authenticate(self, challenge, response):
        value = challenge.read()
        result = SHA256.new(self.password + value).hexdigest()
        return result == response

Some changes to take note of:

  • We can now find a user by their unique id. Since the service will never make the user's database id publicly known this is typically how we will be looking up users.
  • Setting the password will always be handled by the setPassword method. This applies even when a new user is being created.
  • In the last phase I incorrectly placed a querymethod decorator above the authenticate method. Since the method has no need for the database store this was unncessary so it has been removed.

Basic Validation

Now that we're accepting user input we're going to have to come up with a mechanism for validation. For now we're just going to use a file filled with various useful validation functions but down the road we'll be moving towards a more robust approach.

utilities/validation.py


import re

def isStrongPassword(password):
    if len(password) < 8:
        return False

    strength = len(password) // 6
    strength = strength + (0 if re.search('[A-Z]', password) is None else 1)
    strength = strength + (0 if re.search('[a-z]', password) is None else 1)
    strength = strength + (0 if re.search('[0-9]', password) is None else 1)
    strength = strength + (0 if re.search('[^A-Za-z0-9]', password) is None else 1)
    return strength > 3

def isValidName(name):
    if len(name) < 5:
        return False
    if len(name) > 20:
        return False

    return not re.search('^[A-Za-z0-9_-]+$', name) is None

Both functions are fairly self explanatory but it should be pointed out that these are not language agnostic. Unfortunately Python's built in Regular Expression library doesn't work quite correctly for all languages so we're going to have to come back to this later.

Now that we've taken a look at the overhead changes let's see what's changed in our ServiceController.

controllers/servicecontroller.py


import uuid

from .basecontroller import BaseController
from ..utilities import validation
from ..models.data import Application, User, Challenge, UserLogin

class ServiceController(BaseController):
    @BaseController.restrictedmethod
    @BaseController.httpget
    def getApplicationMethod(_self, name):
        if name is None:
            raise _self.getException(id='APP_NAME_REQUIRED', source='name')

        app = Application.findByName(name)
        if app is None:
            raise _self.getException(id='APP_NAME_NOT_FOUND',
                source='name',
                parameters=[name])
        return { 
            'Application' : { 
                'Id' : app.applicationid,
                'Users' : [user.username for user in app.users] } }

Since the BaseController class is now responsible for creating our ClientException instances we can remove the class import to avoid confusion. Our getApplication method is noticeably easier to read without the lengthy error messages cluttering up our code. All that's left to do is add a few new methods to support the necessary functionality for our Passport service.

@BaseController.restrictedmethod
@BaseController.httpget
def getApplicationsMethod(_self):
    return {
        'Applications' : [ 
            { 
                'Id' : app.applicationid,
                'Name' : app.name 
            } for app in Application.getAll() ] }

@BaseController.restrictedmethod
@BaseController.httpget
def saveApplicationMethod(_self, id, name):
    if name is None:
        raise _self.getException(id='APP_NAME_REQUIRED',
            source='name')

    app = Application.findByName(name)
    if not app is None and app.applicationid != id:
        raise _self.getException(id='DUPLICATE_APP_NAME',
            source='name',
            value=name,
            parameters=[name])
    if not validation.isValidName(name):
        raise _self.getException(id='INVALID_APPLICATION_NAME',
            source='name',
            value=name)

    if not id is None:
        app = Application.get(id)
        if app is None:
            raise _self.getException(id='APP_ID_NOT_FOUND',
                source='id',
                value=id,
                parameters=[id])
        if app.name.lower() == 'passport':
            raise _self.getException(id='PASSPORT_IS_READ_ONLY',
                source='id',
                value=id)
        app.name = name
        app.save()
    else:
        app = Application.create(name)

    return { 'Application': { 'Id' : app.applicationid } }

The getApplications method is simple enough in that it simply calls our new getAll method on our Application data model and projects the results. These projected results will act as our view models for the time being. In the future it would be better to create a class that is responsible for translating one type of model into another but this approach will work for the time being. As you can see our saveApplication method benefits greatly from our string resources mechanism due to all the potential error messages that must be conveyed to the client. As with the data models we're going to create some similar methods for the User entities.

@BaseController.restrictedmethod
@BaseController.httpget
def getUserMethod(_self, username):
    if username is None:
        raise _self.getException(id='USER_NAME_REQUIRED', source='username')

    user = User.findByUsername(username)
    if user is None:
        raise _self.getException(id='USER_NAME_NOT_FOUND',
            source='username',
            parameters=[username])
    return { 'User' : { 'Id' : str(user.uniqueid) } }

@BaseController.restrictedmethod
@BaseController.httpget
def getUsersMethod(_self):
    return {
        'Users' : [ 
            { 
                'Id' : str(user.uniqueid),
                'Name' : user.username 
            } for user in User.getAll() ] }

@BaseController.restrictedmethod
@BaseController.httpget
def saveUserMethod(_self, id, username, password):
    if username is None:
        raise _self.getException(id='USER_NAME_REQUIRED', source='username')
    if password is None:
        raise _self.getException(id='PASSWORD_REQUIRED',
            source='password')

    user = User.findByUsername(username)
    if not user is None and user.userid != id:
        raise _self.getException(id='DUPLICATE_USER_NAME',
            source='username',
            value=username,
            parameters=[username])
    if not validation.isStrongPassword(password):
        raise _self.getException(id='WEAK_PASSWORD',
            source='password')
    if not validation.isValidName(username):
        raise _self.getException(id='INVALID_USER_NAME',
            source='username',
            value=username)
    if not id is None:
        user = User.findByUniqueId(id)
        if user is None:
            raise _self.getException(id='USER_ID_NOT_FOUND',
                source='id',
                value=id,
                parameters=[id])
        if user.username == 'root':
            raise _self.getException(id='ROOT_IS_READ_ONLY',
                source='id',
                value=id)
        user.username = username
        user.save()
    else:
        user = User.create(username, password)

    return { 'User': { 'Id' : str(user.uniqueid) } }

Aside from the password validation there's almost no difference in the flow of these methods and our Application related methods. While we do have some additional fields that can be set in the User entity we'll save these features for a future date. For now we're nearly finished. The next step is adding some methods for managing application permits.

@BaseController.restrictedmethod
@BaseController.httpget
def removeUserPermitMethod(_self, userid, applicationid):
    try:
        userid = uuid.UUID(userid)
    except:
        raise _self.getException(id='INVALID_UUID',
            source='userid',
            value=userid,
            parameters=[userid])

    user = User.findByUniqueId(userid)
    if user is None:
        raise _self.getException(id='USER_ID_NOT_FOUND',
            source='userid',
            value=userid,
            parameters=[userid])
    if user.username == 'root':
        raise _self.getException(id='ROOT_IS_READ_ONLY',
            source='userid',
            value=userid)

    try:
        applicationid = int(applicationid)
    except:
        raise _self.getException(id='INVALID_INTEGER',
            source='applicationid',
            value=applicationid,
            parameters=[applicationid])

    app = Application.get(applicationid)
    if app is None:
        raise _self.getException(id='APP_ID_NOT_FOUND',
            source='applicationid',
            value=applicationid,
            parameters=[applicationid])
    if app.name.lower() == 'passport':
        raise _self.getException(id='PASSPORT_IS_READ_ONLY',
            source='applicationid',
            value=applicationid)

    if len([ u for u in app.users if user.userid == u.userid]) != 0:
        app.users.remove(user)
        app.save()

    return None

@BaseController.restrictedmethod
@BaseController.httpget
def addUserPermitMethod(_self, userid, applicationid):
    try:
        userid = uuid.UUID(userid)
    except:
        raise _self.getException(id='INVALID_UUID',
            source='userid',
            value=userid,
            parameters=[userid])

    user = User.findByUniqueId(userid)
    if user is None:
        raise _self.getException(id='USER_ID_NOT_FOUND',
            source='userid',
            value=userid,
            parameters=[userid])
    if user.username == 'root':
        raise _self.getException(id='ROOT_IS_READ_ONLY',
            source='userid',
            value=userid)

    try:
        applicationid = int(applicationid)
    except:
        raise _self.getException(id='INVALID_INTEGER',
            source='applicationid',
            value=applicationid,
            parameters=[applicationid])

    app = Application.get(applicationid)
    if app is None:
        raise _self.getException(id='APP_ID_NOT_FOUND',
            source='applicationid',
            value=applicationid,
            parameters=[applicationid])
    if app.name.lower() == 'passport':
        raise _self.getException(id='PASSPORT_IS_READ_ONLY',
            source='applicationid',
            value=applicationid)

    if len([ u for u in app.users if user.userid == u.userid]) == 0:
        app.users.add(user)
        app.save()

    return None

These methods will allow us to alter which users have access to which applications. In order to implement this capability fully we're going to need to make a change to our authenticateChallenge method.

@BaseController.httpget
def authenticateChallengeMethod(_self, application, username, response, id):
    user = User.findByUsername(username)
    challenge = Challenge.findByUniqueId(id)
    if challenge is None or user is None or not user.authenticate(challenge, response):
        raise _self.getException(id='INVALID_CREDENTIALS', source='password')

    if not application is None:
        app = Application.findByName(application)
        if app is None:
            raise _self.getException(id='APP_NAME_NOT_FOUND',
                source='application',
                value=application,
                parameters=[application])
        if not user in app.users:
            raise _self.getException(id='NOT_AUTHORIZED',
                source='application',
                value=application)
    else:
        app = None

    login = UserLogin.create(user = user, application = app)        
    _self.setCookie(BaseController.loginCookieName, str(login.uniqueid))
    return None 

That should do it. Now when the user attempts to log in the method will check for the application parameter. If it is present a login will only be created if the user is authorized for that application.

Phase III Links: Download | Demo

Conclusion

We now have all the basic functionality we'll need for utilizing this service as a basic authentication manager for an external project. As with the previous phase a simple static HTML page has been provided to demonstrate some of the service's behavior but we won't be going over it in this article. With our Passport system in place it's time to switch gears back to ASP.Net MVC. Next week we'll be looking at creating master pages for our html views and furthering our development on multi-lingual support.

Quick Links: << Previous: Authentication and Login Sessions