18 Jul 2013 Code Refactoring Django Projects

When I started worked with one django project and I encountered the following issues

  • limited knowledge on package dependencies (i.e python modules)
  • Taking so much time to setup development environment and no test cases
  • Code is less organized and coding standards PEP08 are not met.

It leads to much cost of development for upcoming sprints. So I decided that I will put extra effort to refactor existing code to make changes quickly and test the functionality.

I think we need to do iterative approach of * Prepare critical test scenarios * Prepare test data * Refactor source code * Execute test cases

use pip to do package management

Prepare Setup

This approach will be iterative so we need setup.py to install test environment quickly.I developed setup.py through setuptools

import os
from setuptools import setup
from setuptools import find_packages

def read(fname):
    return open(os.path.join(os.path.dirname(__file__), fname)).read()

def strip_comments(l):
    return l.split('#', 1)[0].strip()

def reqs(path):
    return list(filter(None, [strip_comments(l) for l in
                open(path).readlines()]))

setup(  # url = "http://packages.python.org/ha",
    name='PACKAGE NAME',
    version='0.0.X',
    author='AUTHOR',
    author_email='AUTHOR@SITE',
    description='PACKAGE DESCRIPTION',
    license='MIT',
    keywords='',
    packages=reqs('./install/app-list.txt'),
    include_package_data=True,
    zip_safe=False,
    long_description=read('README.md'),
    classifiers=['Development Status :: 0 - Beta', 'License :: MIT'],
    scripts=reqs('./install/scripts.txt'),
    install_requires=reqs('./install/requirements.txt'),
    )

Through above script we can easily maintain all required packages in install/requirements.txt

I can use pip to install requirement packages in developer system

pip install -r requirements.txt

Prepare Test Cases

I think py.test provide quick way to write test cases with prefix of 'test_' to method instead of alternative test suites like py unittest

Refactoring Source Code

  • Tidy Source Code
  • Refactoring Source Code

Using pytidy.py to clean existing code

pytidy tool will work on single python file. So the following script will help to apply for all files

echo '#!/bin/bash
file=$1
cp $file $file.bk
mv $file $file.src
python pytidy.py $file.src $file
#rm $file.src' > test.sh
chmod +x pytidy.sh
find . -name '*.py' -exec ./pytidy.sh {}\;

Refactoring Source Code

Python is dynamic language, sometimes it is very difficult to identify issues due to refactoring. So I would like to recommended

  • Identify external facing components like WEB API,REST and ensure that those components are not effected or corrective decisions are made
  • Django Models fields may effected and try to avoid model changes. If you decided to change model field names then ensure database changes are consistent
  • Changes in url.py considered and it may effect URL Routing(if you decided that url pattern may change then 404 handler has to consider old routing)

Database Refactoring

Generally it is not recommended to database refactoring when there is very limited test cases to valid the functionality.

I see following options database refactoring

  • Use South to Track the database changes and apply those changes in stage or production environments
  • Re-Create Database with the following approach
python manage.py dumpdata|python -m json.tool >data.json
#Make Necessary changes in data.json
# Create new database with new database settings in settings.py
python manage.py syncdb
python manage.py migrate
python manage.py loaddata data.json
  • Refer Book for advanced concepts on database refactoring

What's next ?

After various sprints,we have good number of test cases and source organized with less technical debut and ready for next sprint requirements. Now, we need to start * thinking about in-line documentation wih examples and expose API Documentation through sphinx * continue to use pylint

comments powered by Disqus