Cygora is finally online as private beta!

I finally released my project Cygora (www.cygora.com), a social platform which allows to organize any kind of meeting by any user in his city.
My project is a brand new site in its genre, because it allows an huge flexibility for the end user and is 100% focused on the organization management, by allowing the definition of participation rules, like gender, age of subscribers, max number of participants and more. The site is developed with the main goal of being user-friendly and to offer the opportunity to find interesting (and even crazy) activities to do with other people.
From a technical perspective, I consider it an extremely important milestone for my career as a developer, because I used several interesting technologies (Python, Django, PostgreSQL, RabbitMQ, Celery, Redis, git, virtualenv, pip, AngularJS, Bootstrap…) and I configured a modern software architecture for a web project, by using Amazon Web Services (AWS). This should allow me to easily and automatically scale my app both horizontally than vertically. I also used cool tools like New Relic and Loggly, in order to monitor my app and to discover what’s going on on my EC2 instances.
I’m very proud of the work I’ve done, but obviously this is just the beginning and I have to continue in this direction if I hope to become a future web entrepreneur! :)
These are the current numbers of Cygora:

400 Unit tests written (This does not make the project bug-free, but working in TDD helped me a lot in fast fixing during my continuos refactoring… indeed the way to go!)

1400 Versioned files (many classes, short methods, DRY approach)

300 Commits on the git repository

14 (Django) Apps to distribute responsibilities across the project

3 Developed libraries (at least) that I can potentially release as open source projects in the future (a simple newsletter app, a mini framework to dynamically create translations model classes based on app languages, a mini framework which allows to add comments to any model in a Django app)

9 MB Of deployable source code (including static assets)

100+ Deploys to test the release cycle

2 Initial localizations for the application (English and Italian)

1 Year of hard work in the night and weekends (in which apart from the development of Cygora I learned: Python, Django, RabbitMQ, Celery, AWS and much more)

Creating a custom AMI with Postgis and its dependencies in order to deploy Django + GeoDjango on Amazon Elastic Beanstalk

While the installation of PostgreSQL + Postgis on my development machine (my beloved MacBook Pro) has been very easy, thanks to MacPorts, installing the necessary software on Amazon Elastic Beanstalk (in order to move my project Cygora.com from local to the cloud) has been an hard challenge!
Theoretically you can customize an environment by using configuration files in which you can specify packages and other resources to install, but the problem is that in the Amazon 64bit Linux distribution for Python (which is an extremely customized version of Red Hat) you don’t have apt (for which postgis packages are available), instead you have to rely on yum. Is possible to install extra repositories for yum (see here: http://postgis.net/install) in order to easily install postgis… but honestly I have no idea which repository should be the right one for Amazon Linux, so… it’s been painful, but I opted for an “old school” style installation, by downloading and compiling the missing packages by myself. So, after launching my EC2 instance I did connect to it via SSH and:

1. Switch to root user:

sudo su -

2. Update all the installed packages (which Amazon doesn’t update very often!):

yum update -y

3. Install development tools and necessary libraries (some of them, like “graphviz” are not required for GeoDjango and you can aovid their installation if you want… I’m reporting all my libraries as a future reference for myself :P)

yum install -y python-devel libpcap libpcap-devel libnet libnet-devel pcre pcre-devel gcc gcc-c++ libtool make libyaml libyaml-devel binutils libxml2 libxml2-devel zlib zlib-devel file-devel postgresql postgresql-devel postgresql-contrib geoip geoip-devel graphviz graphviz-devel gettext libtiff-devel libjpeg-devel libzip-devel freetype-devel lcms2-devel libwebp-devel tcl-devel tk-devel

4. Download and compile proj:

wget http://download.osgeo.org/proj/proj-4.8.0.zip
unzip proj-4.8.0.zip && cd proj-4.8.0
./configure && make && sudo make install
cd ..

5. Download and compile geos:

wget http://download.osgeo.org/geos/geos-3.4.2.tar.bz2
tar -xvf geos-3.4.2.tar.bz2 && cd geos-3.4.2
./configure && make && sudo make install
cd ..

6. Download and compile gdal (this library is the most SLOW to compile and depending on the type of instance that you have launched it may takes up to a couple of hours… be patient!):

wget http://download.osgeo.org/gdal/1.10.1/gdal1101.zip
unzip gdal1101.zip && cd gdal-1.10.1
./configure --with-python=yes && make && sudo make install
cd ..

7. Download and install postgis:

wget http://download.osgeo.org/postgis/source/postgis-2.1.1.tar.gz
tar -xvf postgis-2.1.1.tar.gz && cd postgis-2.1.1
./configure && make && sudo make install

8. Update installed libraries (this step is necessary to avoid issues related to invalid library paths):

sudo echo /usr/local/lib >> /etc/ld.so.conf
sudo ldconfig

It’s also a nice idea to export the environment variable LD_LIBRARY_PATH (as /usr/local/lib/:$LD_LIBRARY_PATH).
Once you have installed all the necessary software on your machine you can create a custom AMI by going to: EC2 > instances > select your instance > create AMI. To use that AMI as the default one for your application you have to specify its id in your Elastic Beanstalk environment configuration.

ps. If you are interested in Elastic Beanstalk + Django topic, you should subscribe to my RSS feed and follow my on Twitter (@daveoncode)… I will write a series of posts on running my project in the cloud using Amazon Web Services :)

Django 1.6 has been finally released… these are the changes I introduced into my codebase

Django 1.6 has been finally released, but this update is quite obtrusive, in fact…

Issue n.1: tests not found!

the dev team has changed significantly how unit tests are discovered and my old approach stopped working (no tests to run found).
In my previous system I created a “tests” package for each app in my project, in these packages I created a test class for each tested class, named like “TestClassName” and I exposed these classes at “tests” package level by customizing its __init__.py file.
But starting with Django 1.6 the default test runner doesn’t care about “tests” modules/packages, it looks instead for file matching the pattern “test*.py” and since this pattern is CASE SENSITIVE (aaaarghh!!!) my test CLASSES (so they start with an uppercase letter) are ignored! Fortunately is very simple to override the default runner in order to match another pattern :)
This is my custom runner:

from django.test.runner import DiscoverRunner

class TestRunner(DiscoverRunner):

    def __init__(self, pattern=None, top_level=None, verbosity=1, interactive=True, failfast=False, **kwargs):
        super(TestRunner, self).__init__('Test*.py', top_level, verbosity, interactive, failfast, **kwargs)

and in the settings module:

TEST_RUNNER = 'myapp.TestRunner.TestRunner'

Issue n.2: PyCharm Django Tests configuration is now broken!

If you are using PyCharm as your python IDE, the “Django tests” run configuration is now broken (honestly I don’t understand why the guys of JetBrains have set up a such perverse way to run Django tests by writing their own python modules LOL)… so I defined a simple python run configuration in which I call “python manage.py test“… it’s not so pleasant, like the Django tests run configuration because is basically the same output of the shell script without the “green bar” and the red/yellow/green buttons that show the results of the run tests… but at the moment it does the job, and the important thing is that, differently from launching tests from the shell I can use breakpoints in the IDE to stop code execution and debug the situation! (like before)
Uh… the good part is that now my tests are executed ~40% faster!!! (10 seconds vs 14 seconds in the old run configuration)

Issue n.3: Where is gone GeoIP?!

Another problem I faced is that the GeoIP wrapper around MaxMind api has been moved from django.contrib.gis.utils to django.contrib.gis.geoip (and this change seems not documented in the release notes!)

Creating class based view decorators using simple Django function decorators

I love decorators because they allow to extend class/functions capabilities and to be be added and removed easily. Django offers a set of decorators but unfortunely they are designed for functions or class methods only and instead I want to add them to the whole class in order to avoid to override the orignal method just to add the decorator annotation above it. Fortunately wrapping the simple Django decorators in order to use them against classes (basically View classes) is quite simple and I’m gonna show you what I done.
For example let’s consider we have a view that require a POST method. If we use a simple function view we can write the following:

@require_POST
def myView(request):
	return HttpResponse('hello world!')

but, if we use class views (and we should!), we are forced to write:

class MyView(SomeKindOfParentClassView):
	# ...imagine several methods here

	@require_POST
	def dispatch(self, request, *args, **kwargs):
		return HttpResponse('hello world!')

wouldn’t be cleaner and more elegant instead to simply write the following?

@require_POST
class MyView(SomeKindOfParentClassView):
	# ...imagine several methods here

…and this is the class decorator implementation:

from django.views.decorators.http import require_POST as post_only

def require_POST(View):
    View.dispatch = method_decorator(post_only)(View.dispatch)
    return View

Basically I redefined the class method and returned the class itself.
This was really simple since the concrete implementation of the decorator has been added using the existend Django decorators… but what if we want to create a custom decorator? For example in my app I created a decorator that enable access to a view only if it’s requested via ajax, this is how I implemented it:

def require_AJAX(SomeKindOfParentClassView):
    def ajaxOnly(function):
        def wrap(request, *args, **kwargs):
            if not request.is_ajax():
                return HttpResponseForbidden()
            return function(request, *args, **kwargs)

        return wrap

    View.dispatch = method_decorator(ajaxOnly)(View.dispatch)
    return View

And the decorated view looks like this:

@require_POST
@require_AJAX
class MyView(SomeKindOfParentClassView):
    # ...imagine several methods here

As you can see that annotations are really readable, immediate and self-describing.

How to make AngularJS and Django play nice together

In order to make AngularJS working as I wish in my Django app, these are the settings that I’ve adopted:

1. Differentiate Angular templates symbols from Django ones:

Both Angular than Django use doble curly braces to mark variables and/or expressions ({{ myVar }}).
In order to have the full control on how and by who our templates are rendered, I redefined the Angular interpolations signs in the config() method of my client app.

$interpolateProvider.startSymbol('{$');
$interpolateProvider.endSymbol('$}');

2. Change the default Angular Content-type header used in POST method:

Angular defines the “Content-Type” header as “application/json” for ajax POST, but Django doesn’t understand that content properly and as result, the POST data is not an object as we expect but rather a string! So, I modified the default content type as “application/x-www-form-urlencoded” (which is the format used by jQuery and other JavaScript libraries).

$httpProvider.defaults.headers.post['Content-Type'] = 'application/x-www-form-urlencoded';

3. Assign the CSFR token to each ajax call automagically

In Django templates we can use the tag {% csrf_token %} inside a form to print an hidden input containing the token. But when it comes to making an xhr post request, how can we pass the token in an effective and DRY manner? The answer I gave myself is to set a default http header for ajax calls containing the value of the token obtained by reading the session cookie (in this way this stuff is handle 100% by JavaScript).

$http.defaults.headers.post['X-CSRFToken'] = $cookies.csrftoken;

Differently from points 1 and 2, this is done in the run() method, since $cookies is an Angular service and can’t be used in config() block (in that function only provider objects can be used).
In order to use $cookies we have also to import “angular-cookies.js” in addition to the base “angular.js“.

The final configuration is the following:

angular.module('myapp', ['ngCookies']).
    config([
    '$httpProvider', 
    '$interpolateProvider', 
    function($httpProvider, $interpolateProvider) {
        $interpolateProvider.startSymbol('{$');
        $interpolateProvider.endSymbol('$}');
        $httpProvider.defaults.headers.post['Content-Type'] = 'application/x-www-form-urlencoded';
    }]).
    run([
    '$http', 
    '$cookies', 
    function($http, $cookies) {
        $http.defaults.headers.post['X-CSRFToken'] = $cookies.csrftoken;
    }]);

UPDATE:

Starting from Angular 1.2, you have also to set default headers in order to use Django helper method is_ajax() in your views:

$httpProvider.defaults.headers.common['X-Requested-With'] = 'XMLHttpRequest';