Installing and configuring Linux Mint in dual boot on an Apple Macbook Pro with retina display

Preface

I’ve been an happy mac OSX user for almost 10 years, but recently something in my mind has changed… I don’t know… it’s like if… suddenly I’ve started to feel myself like a child which simulates to drive his father’s car (when it comes to the OS) when the reality is that I’m an adult that actually wants to drive his car and has a regular license and the experience to do that :P
So I decided to try the switch from OSX to Linux.
Linux’s world is a mess, there are a bazillion of distros and several “GUI engine” (I don’t know if this is a correct/acceptable term for talking about GNOME, KDE, Cinnamon and so on, but I’m a newbie in the Penguin’s planet :P) so initially my first target has been Ubuntu since it’s the most famous and supported version (and it has for example a dedicated site on my beloved stackoverflow newtork: askubuntu). Unfortunately most famous and adopted doesn’t mean “the best” (can you hear me windows users?!) and in fact after playing with Ubuntu 15 and 14 and by following a suggestion of a friend I tried Linux Mint and as far I tested it’s far better than Ubuntu: it’s more stable, more responsive, it has a better gui free from the several little bugs I found in Ubuntu, and most important currently it has the best support for HiDPI displays thanks to Cinnamon (Ubuntu’s Unity offers a way to scale the interface in order to match the highest pixel density of a retina, but you have to set it manually, Mint instead automatically detect your monitor and apply the right resolution… moreover even by setting the right resolution in Ubuntu, the login screen will always looks bad and several ui components like icons in notification popups will looks blurred… on Mint all the OS elements will automatically looks gorgeous like in OSX!). So I’m currently running Linux Mint Cinnamon 17.1 (code name “Rebecca”) on my macbook pro retina mid 2012.
I opted for a dual boot installation (with Linux starting as the default system) just to be able to use XCode in the future for an eventual iOS app development or something like that requiring OSX.
The installation (even in dual boot) is quite simple and, despite the many tutorial I found on the web, DOES NOT REQUIRE any esoteric third party software like ReFind, LinuxFlashDriveInstaller, BootMyPenguinToApple and so on (actually I invented all these names since I don’t remember the real ones, but you should get it).
So lets see how to install it…

1. Requirements

1. an external hard disk for backup
2. a flashdrive (less than 2gb needed!)
3. an internet connection
4. a cable adapter (you have to connect the macbook to the internet without wifi since you will install the necessary wifi drivers only later)

2. Backup your stuff

This is a very important step in order to avoid the loss of important files and hours/days/weeks or years of work!
If you never used TimeMachine is time to try it out and create a full backup on an external hard disk.
TimeMachine is awesome, you can restore your previous machine state even if you completely erase your primary disk! But if it’s awesome for OSX it’s pretty useless if you want to move your data to a new OS, since it use an incremental backup system that only OSX can properly use. So my suggestion is to have an external HD with 2 partitions: one for the time machine backup and another to manually copy the files you wish to use/conserve for another OS or for the future.

3. Partitioning (preparing the hard disk for running an additional OS)

Open disk utils (it’s an utility app included in OSX to manage disks), select your machine’s disk and add 2 new partitions (using the “+” button at the bottom of the window), one will be used to install Linux Mint, the second one will be used as swap partition. A swap partition is not mandatory but it’s highly recommend, since for example Linux needs it to allow features like hibernation. So the Linux partition should be at least 20gb, the swap partition a few gb, ideally not more than your available ram (in my case 8gb). You can choose any filesystem type for the new partitions, since we will properly format them during Linux installation. Even the partition name doesn’t matter (I choose “LINUX” and “SWAP”). Once you apply the defined partition scheme, your main partition (where OSX is installed) will be shrinked to make room for the new ones by preserving all the current content on the disk (obviously you should have enough free space on it). The partitioning may requires several minutes depending on the size and disk type of your machine.

4. Create a bootable Linux flash drive

Download the iso from Mint’s website (in my case I downloaded the 64bit Cinnamon edition).
Once downloaded you have to convert the iso into an img.
In the terminal app execute the following commands:

sudo hdiutil convert -format UDRW -o /path/to/mint.img /path/to/mint.iso
sudo mv /path/to/mint.img.dmg /path/to/mint.img

Now you have to write the image to the flash drive. Once the flash drive is plugged into the usb port, run the command:

diskutil list

in order to figure to which disk node it has been assigned to (disks “nodes” are in the form /dev/diskN, where N is an incremental number), then run the command:

diskutil unmountDisk /dev/diskN

(where N is the number related to your flash drive).
Now you have to write the image to the flash drive by running the following command:

sudo dd if=/path/to/mint.img of=/dev/rdiskN bs=1m

Note 1: “rdisk” it’s not a typo, but a trick to speed up the write process
Note 2: this command will replace all the data on the flash drive!
Note 3: it may takes some minutes to complete, and no progress input will be displayed, the only visible feedback will be the flashing cursor in the terminal
Note 4: once finished, or during one of the previous step, OSX may show an alert saying something like “disk X is not readable” and offering an option to format it. In this case just press “ignore” (or anyway the button that dismiss the alert without actions)

5. Installation

Once the image has been written to the flash drive (you will see in the terminal the report of written data once the command has completed his job), restart your mac by holding the alt (option) key pressed once you hear the typical booting sound. You will see a screen from which you can select from which disk to boot, choose your flash drive!
Before installing Linux Mint, ensure that: you are connected to the network via network cable and that your machine is connected to the power source.
By booting your mac using the flash drive containing Linux Mint image, you will “land” to the “portable” version of it (that is a working and ready to try Linux Mint OS). On the desktop you’ll see a DVD icon for the actual installation, double click it and proceed to the real installation.
Once you reach the installation screen “Installation type“, be very careful, this is the most important and “dangerous” step. There is a list of several options, but you have to choose the last one: “something else” (since it allows you to specify exactly what to do).
In the next screen you will see a panel with all available disks/partitions and a combo at the bottom “Device for boot loader installation”.
Now you have to find the partition you previously created for Linux (you should figure this out by reading its size and available space and in my case was “/dev/sd4″), then click on “change”, select a file system (the preferred one for Linux is “Ext4 journaling file system“), check “format partition“, set “/” as mount point, press “OK” and confirm your action.
Then you have to find the partition related to the swap, click “change” and select “use as swap area“.
Finally, change the selected option in “Device for boot loader installation” by specifying the proper Linux partition.
Then proceed with the installation by confirming and following the wizard.
At the end of the installation you will be asked to restart your machine, if you get stuck in a black screen and your machine does not restart… just press enter (it’s an embarrassing “little” bug in the GUI that may happen XD).
Your machine should reboot and automatically load Linux Mint OS (you can boot into Mac OSX by holding alt key during the booting).
Now you should have a working mac OSX/Linux Mint dual boot machine… but for Linux, there are a couple of things to do in order to have a fully working OS… for example you can’t currently use the wifi, but we are going to fix this and other issues in the “post installation” section…

6. Post installation

6.1 Installing updates

Once your machine reboots into Linux Mint, the OS should display an icon in the bar displaying available updates, by clicking it the software manager will open up, select all the available updates and install them.

6.2 Installing missing (wifi) drivers

Open the driver manager (just press the command key or “super” as it’s called in Linux and type “driver manager”), it will shows several drivers available for the NVidia (or whatever graphic card your machine has), but DON’T CHANGE THE DEFAULT DRIVER automatically installed by Mint (which should be the only open source one in the list), since as far I saw and even if it sounds odd, the proprietary drivers on Linux won’t work (black screen once installed and rebooted!). The same problem happens with Ubuntu (both 14 than 15).
However the driver for the “airport extreme” (the wifi), can be safely installed (and it’s the only one, available). Once installed you should be able to connect to the internet using the wifi (if not try rebooting your mac).

6.3 Macbook trackpad, external keyboard and mouse setup

There are some configurations to change and some “hacks” to do in order to have a working and conformable input setup. The trackpad works out of the box, but it doesn’t scroll using two fingers as I was used to do nor it has natural scrolling (inverted scrolling -> move up do go down and vice versa like on a mobile device).
So the first thing to do via GUI is to open the “mouse and touchpad” preference and under “Scrolling / Panel layout” choose “two-finger scrolling” and check “enable horizontal scrolling“. As you can see there is a ready-to-click option “enable natural scrolling” which actually does exactly what it says, but leave it unchecked, because it would break natural scrolling on an Apple Magic Mouse! (we will fix things in order to have a natural scrolling on both soon…)

6.3.1 Pairing that F@$%&*’ Magic Mouse!

Blue tooth pairing is automatically handled by Mint and setting up a (Apple) keyboard is dead simple and fast, you just need to turn the keyboard on and the OS will ask you to type a code on the keyboard for the pairing… BUT, setting up a blue tooth mouse is just… IMPOSSIBLE (via GUI), due to a weird bug, for which once the mouse is discovered and the OS opens a popup to insert the pairing code (which is 0000 by default), as soon you click on the input field in order to write the code, that damned popup gets closed, so the result is that you don’t have the opportunity to provide the code… so pairing fails :(
But, fuck off… let’s take the control and manually pairing the mouse using the terminal (fortunately we need to do this only once)!
We need a couple of packages that provides blue-tooth handling functionalities, so let’s get them:

apt-get install -y bluez-tools bluez-hcidump 

then follow the instructions under “solution”: here. I used that post to fix my problems too, but in my case I simply passed the passcode using the retrieved device mac address and I didn’t added it as “trusted device” as described in the tutorial (anyway my magic mouse is properly recognized each times).

6.3.2 Get rid of the “ctrl+v” that Linux OS automatically assigns to the middle mouse button (and that with an Apple Magic Mouse it’s a pain in the ass!!!)

Yep… the third button click on Linux it’s equivalent to a “paste” command, but using a mouse that doesn’t have physical buttons, accidental ctrl+v are very, very easy (I was going crazy by seeing pieces of urls/comments/variables added to my code each time I used the mouse!).
So, the solution is to remove the third button by the used layout, by running:

sudo sh -c "echo 'pointer = 1 0 3 5 4 7 6 8 9 10 11 12' >> ~/.Xmodmap"
sudo xmodmap ~/.Xmodmap

Buttons are numbered incrementally from 1 to 12 and the sequence from left to right means: button one, button two (so middle button), button 3 and so on… By specifying an invalid (missing) button like 0 or 13 we are removing it’s binding. With this command we are also redefining the order of “buttons” 4, 5, 6, 7 that are those related to scrolling gestures in order to have a natural scrolling effect even on the magic mouse… and this settings will also used for the trackpad… so now all should works perfectly!!!

6.3.4 Configuring specific apps in order to properly handle an HiDPI monitor (retina display)

Cinnamon does a great job in order to display the OS gui properly on an HiDPi monitor, but certain applications (mainly Java and QT ones) don’t look very good. Unfortunately these ones can “fixed” only if the vendor has implemented support for the different display (such JetBrains IDEs, chrome, firefox and so on), to find out the required settings in order to display these applications properly read this great resource on Linux Arch wiki.

6.3.5 Configuring the keyboard in order to use European letters like ü, á, ñ…

On Mac OSX even if you use an US keyboard you can hold a key to select one of it’s related “variants” with “special signs” (for example by pressing and holding “i” you can then select an “í” from an inline tooltip that shows up).
On Mint the only way I found in order to use special letters is to switch from “English (US)” keyboard layout to “English (US, International with dead keys)“, this one allows to print special letters by pressing right alt + letter key (so to print “í”, you have to press “i” + the alt key on the right of the keyboard).
Unfortunately the “International with dead keys” as a strange mappings for quotes and brackets, so I can’t set it as the default layout, instead when I need to type those letters I temporally switch from a layout to another using the icon in the system toolbar… it’s a reasonable effort :^)

AngularJS: centralized application loading status handling using http interceptors

It’s a common need in a JavaScript application (especially a single page app) to show a spinner and/or a message while data is getting retrieved from the server (that is while the ajax call is in progress). In fact an ex colleague asked me how I manage this in AngularJS just a couple of weeks ago and I want to share my implementation since I think that’s the cleanest and more effective way to achieve the goal using native framework’s features. Moreover it’s also useful to handle ajax errors in a centralized and generic way and I’m gonna show that too.
AngularJS has a very useful feature: HTTP Interceptors, these ones are services that get automatically called on each ajax request step (before an ajax call, after an ajax call and so on) once they are registered using the $httpProvider.
So they are the perfect place where to write our logic for generic error handling and loading state, in fact the official documentation says:

For purposes of global error handling, authentication, or any kind of synchronous or asynchronous pre-processing of request or postprocessing of responses, it is desirable to be able to intercept requests before they are handed to the server and responses before they are handed over to the application code that initiated these requests. The interceptors leverage the promise APIs to fulfill this need for both synchronous and asynchronous pre-processing.

So what I do in my app is to create the following interceptor:

angular.module('myapp').
    service('LoadingInterceptor', 
    ['$q', '$rootScope', '$log', 
    function($q, $rootScope, $log) {
        'use strict';

        return {
            request: function(config) {
                $rootScope.loading = true;
                return config;
            },
            requestError: function(rejection) {
                $rootScope.loading = false;
                $log.error('Request error:', rejection);
                return $q.reject(rejection);
            },
            response: function(response) {
                $rootScope.loading = false;
                return response;
            },
            responseError: function(rejection) {
                $rootScope.loading = false;
                $log.error('Response error:', rejection);
                return $q.reject(rejection);
            }
        };
    }]);

It makes use of $rootScope to store the loading state “globally”, which is set to true as soon the request is created (request method) and set to false when the request is invalid (requestError) or the call has been completed (response or responseError).
It also logs errors in case of requestError and responseError.

Then I register the interceptor in the module config() block:

.config(['$httpProvider', function($httpProvider) {
    $httpProvider.interceptors.push('LoadingInterceptor');
}]);

As you can see it’s registered as a simple plain string rather than using the usual dependency injection (AngularJS knows how to properly load the service).

Regarding the display of a message/spinner to the user, in my templates all I have to do is something like:

<div data-ng-if="loading">
    Loading...
</div>

That’s really cool! clean! simple! effective! :)

UPDATE:

I realized that my interceptor doesn’t work properly when concurrent xhr requests are involved.
For example, if a request A starts and then a request B starts, if B completes its job before A, the loading status is defined as “false“. To fix this issue I modified it by tracking the count of initialized requests and completed ones, so the final reliable implementation of the interceptor is the following:

angular.module('crs').service('LoadingInterceptor', ['$q', '$rootScope', '$log', 
function ($q, $rootScope, $log) {
    'use strict';

    var xhrCreations = 0;
    var xhrResolutions = 0;

    function isLoading() {
        return xhrResolutions < xhrCreations;
    }

    function updateStatus() {
        $rootScope.loading = isLoading();
    }

    return {
        request: function (config) {
            xhrCreations++;
            updateStatus();
            return config;
        },
        requestError: function (rejection) {
            xhrResolutions++;
            updateStatus();
            $log.error('Request error:', rejection);
            return $q.reject(rejection);
        },
        response: function (response) {
            xhrResolutions++;
            updateStatus();
            return response;
        },
        responseError: function (rejection) {
            xhrResolutions++;
            updateStatus();
            $log.error('Response error:', rejection);
            return $q.reject(rejection);
        }
    };
}]);

Regular Expressions in Python: how to match english and non english letters

Ok, this is a quick (and I hope super-helpful) tip on how to match foreign languages letters like (ö, è…) in a python regex.
As everybody knows, matching letter signs is just a matter of using [a-z] or \w (the latter will also match underscores!) but unfortunately letters with “decorations” are not matched by these selectors. If you want to match them, you have to use unicode selectors (something like [\u00D8-\u00F6]), but python can automatically match all the unicode variants by simply passing the flag re.UNICODE to compile(). So this:

re.compile('[^\W_]', re.IGNORECASE | re.UNICODE)

will match any english and non english letter.
But let me explain… \w matches letters and underscores, \W (note it’s uppercased) as opposite match all but letters and undescores, so [^\W_] will match letters only (thanks to the negation ^).
Bear in mind: the flag re.UNICODE as reported in python docs :

“Makes several escapes like \w, \b, \s and \d dependent on the Unicode character database”

A stupid demonstration:

# -*- coding: utf-8 -*-
import re

ENGLISH_CHARS = re.compile('[^\W_]', re.IGNORECASE)
ALL_CHARS = re.compile('[^\W_]', re.IGNORECASE | re.UNICODE)

assert len(ENGLISH_CHARS.findall('_àÖÎ_')) == 0
assert len(ALL_CHARS.findall('_àÖÎ_')) == 3

ps: not all languages have implemented the unicode flag, for example JavaScript had not …I love Python :)

Frameworks, libraries, dependencies and philosophies

In these days, at the company where I work, we are discussing about which front-end framework to adopt for our projects.
The two candidates are currently AngularJS and Backbone, but the discussion has stimulated my thoughts about libraries and frameworks in general and I want to speak about this topic and about opinions from colleagues and other people working in the IT whose I disagree . (I’m such a badass that sticks to his ideas until someone is able to provide me a far valid, far exhaustive and far more convincing list of motivations than mine).
So, in this post I would like to express my point of view about frameworks, libraries and dependencies in a language-agnostic way.
First of all I want to provide the definitions of “framework” and “library” in order to be sure that you (the reader) will consider these terms as I do. I’m providing them by taking inspiration from a relevant question on StackOverflow: What is the difference between a framework and a library? and by arbitrarily rearranging the responses in that discussion in order to provide a more personal and deeper definition.

Library:

A library performs specific, well defined operations like handling http connections, data serialization, cryptography and so on. It’s composed by several classes and/or functions and you (the developer) have the control over it, you decide when and where to call such “utilities” in your code.

Framework:

A framework is instead far more complex than a library and it offers a lot of different functionalities. It follows the so called “Hollywood principle” (if you love buzz words like many “chatterboxes” do) for which you (the developer) don’t have the full control over it, but instead you have to follow and agree to the concepts and procedures defined by the framework. A framework goes beyond the programming language, it has its architecture, its philosophy and is composed by different layers of code (and often also by different languages and technologies, it could be composed by a mix of Python + JavaScript + HTML + SQL + YAML for example).

So, by comparing AngularJS with Backbone we are comparing two different things, since the first is a complete front-end framework, it offers all that you need to create a JavaScript application (templating, controllers, data binding, localization…) and you don’t have the “low-level” control on the code, since you don’t manually create instances of your controllers neither you render your UI components (it’s the framework that does the work for you when it’s ready), on the other hand Backbone is just a library which provides an abstraction layer on the MVC design pattern, but you have to use several third party libraries to handle dom manipulation, templating, cookies and so on.

Now, I want to report a series of sentences coming from ideas/doubts/observations I heard/read on the web or by colleagues and below each one I will reply with my opinion.

1. Does I need a framework to do X?

If you are a professional developer (or you aim to be) which is implementing something else beyond an “hello world” demo, you should absolutely use some kind of framework to do your job!

2. …but I don’t want to use a framework, I prefer to write my code from scratch

Writing code from scratch is an useless waste of time and effort! Why should you re-invent the wheel each time? Why don’t rely on a tool which already exists and effectively solves common needs for an application development and that’s already developed, used and tested by a lot of people (who have a strong experience on the involved language and on several aspects of developing, like: performance, security, scalability, maintainability and so on)?
And if you choose to write stuff from scratch, unless you are writing a pile of shitty spaghetti code, in the end you will implement some kind of abstraction layer in order to reuse functionalities in your app… so you will build your own framework… so once again: why don’t you use something that already exists and it’s being actively developed, tested, patched and for which people are writing extensions, tutorials, plugins and are constantly reporting issues and experiences?
I have to say that I was myself guilty in the past, I was used to write all I needed from scratch, only because I’m able to do that and I felt somehow “stronger”, “smarter” by acting that way… but it’s not the truth. The truth is that using a framework consciously is an evidence of being a mature programmer, who understands that is better to invest time on application specific features rather than writing low-level common utilities.
If you are an experienced programmer who love to write low-level API, it’s more useful to contribute to these frameworks, since they are usually open source projects available on github or write “plugins”/”extensions” for them.
Don’t reinvent the wheel, even if you can do it better, please avoid it, it’s a waste of time if you are alone (the development it’s only a part of the work, you have to maintain it, document it, fix bugs and so on).

3. Frameworks are hard to learn, I prefer to use a bunch of small simple libraries

That’s true, frameworks require a considerable time investment in order to be mastered by the developer (I learned Python in 2 days by myself and it took some months to get confident with the Django framework for example), but definitely it worth! You will be far more productive once you get used to it, you will write less code, in less time and without to worrying about how to design your application skeleton because is the framework itself which provide you the path to follow. On the other hand, learning a library is quite simple, but that’s just because libraries are focused on a single limited scope.
I don’t like to introduce many dependencies in my projects, I usually rely on third party code only if it dramatically improves my productivity and this is where a framework comes into play! It’s just one dependency, yes an huge dependency, but I can trust that each components it provides works as expected, since they are tested by the team that developed the framework and by people that use it, on the other hand by choosing several libraries arbitrarily in order to compose my own software stack, I’m not sure that each component will works smoothly as expected.
Let’s suppose to use libraries: A, B, C, D… how can I be sure that C hasn’t some kind of conflict or subtle incompatibility with A, B or D? Theoretically if A, B, C and D are just simple libraries that handle one and only one specific task, there won’t be any issue, but from my experience I can say that theory and practice are two different things… such issues are a concrete problem and this is specially true if we are using a dynamic scripting language like JavaScript (or Python, Ruby, PHP and so on) in which you can write atrocious hacks like runtime classes/functions (re)definition, variables overriding and so on. The worst thing is that such issues are often very, very, very HARD to spot!

4. A framework is heavy and I think I will use just use a little part of it… it’s overkilling!

Someone talks about monolithic frameworks and considers projects like Django (talking about server side stuff) or AngularJS (talking about client side stuff) overkilling so prefers to use multiple dedicated libraries. In my opinion you will never use a framework at 100% of its capabilities, but if you are not taking advantage of at least 50/60% of its features, this means that your project is so tiny and simple that you really just need few things or, if that’s not the case, you are therefore writing by yourself functionalities that are already provided by the framework and you are not aware of (or that you deliberately decided to not use).
Adopting a framework is an important choice that must not be taken easy, it must be carefully considered, planned, discussed.
As I said in several occasions to other colleagues, the adoption of a framework is like a marriage, you know that won’t be the perfect one, like there won’t be the perfect woman/man of your life, but anyway you found the one you like the most and that provides all or many of the functionalities you need, but of course it has also some flaw or trait you don’t like.
So the point is that you have to evaluate all these aspects and once you choose a framework you have to “get along with it”, you have to share its philosophy and style and you have to use it like it’s expecting to be used, otherwise you’ll find yourself struggling with the framework in order to make it behave like it hasn’t been intended to behave, and you’ll waste time and effort you could spare.

5. Ok, let’s use a framework or a set of libraries… but I want to be able to switch from one to another easily, so I’m gonna write some wrappers/adapters!

Switching from a framework to another in a big project will never be easy (that’s why you have to choose it carefully)! Writing an abstraction layer by creating wrappers around original API is in my opinion a waste of time! Don’t get me wrong it’s not about laziness, but instead because I’m a pragmatic developer who doesn’t live in a magic world where ponies run on the rainbow, I decided instead to take the pill from Morpheus which brings me to the real world in which I’m aware of the difficulty/impossibility of maintain such code base (that in the moment in which you have actually to switch the underlying framework will probably reveal its fragility and uselessness).
In my opinion, this approach is not agile, this is not feasible at all… it looks to me just insane!

I think that I said all I have to say about the topic, I hope you enjoyed my post and I would like to read your point of view in the comments.

High level URL manipulation using native Python API

While developing my new project I faced the need of manipulating an URL in order to change its query-string.
Basically my goal was to provide a parameter with a default value if not already defined and to add another new one.
The modules I used are urlparse and urllib, and in a few lines of code I achieved my goal in an high level programming fashion (I mean, without regex or low level “hacks”).
So let’s start… the first step is to parse the URL string using urlsplit:

from urlparse import urlsplit

url_data = urlsplit(url_string)

Supposing url_string is a string holding a valid url like “http://www.mysite.com/path/?a=1&b=2″, the urlsplit will return a SplitResult object which is a named tuple.
A named tuple is a subclass of tuple, a class which behaves like it but offer a way to initialize it using pre-defined keywords arguments and to refer them later, so for example is possible to create a named tuple called “CreditCard” in this way:

from collections import namedtuple

CreditCard = namedtuple('CreditCard', 
'number, secure_code, expire, owner')

and use it in this way:

card = CreditCard(number=1234567890, 
                  secure_code=123, 
                  expire='2018/06/06', 
                  owner='Peter Parker')
print '{}\'s card number is: {} and'
      'has this secure code: {}'.format(
          card.owner, card.number, card.secure_code
      )

One cool feature of named tuple is that you can update one of its field without to have to recreate the object by yourself using the method _replace (it will returns a new tuple with the updated value… remember that tuples are IMMUTABLE objects!).
So to change the owner of the previously defined credit card you will do:

card = card._replace(owner='Bruce Wayne')

(to be honest I don’t know why they decided to mark this helpful method as “protected” using the underscore prefix… but this does’n really matter)

ok… you should get it now. Let’s back to SplitResult… the tuple has the following properties (they are all strings objects):

  • scheme (http, https…)
  • netloc (www.mysite.com)
  • path (/path/)
  • query (the query-string)
  • fragment (what comes after the “#” sign)

So, what I needed was to manipulate the query-string, but once parsed out from the original URL it’s just a raw string and to avoid to mess up with string manipulation I used parse_qs, which returns a python dictionary:

from urlparse import parse_qs 

qs_data = parse_qs(url_data.get('query'))

A dictionary is very handy in order to manipulate query-string parameters, so now all I have to do is something like:

if not 'target_parameter' in qs_data:
    qs_data['target_parameter'] = ['tp1']
qs_data['extra_parameter'] = ['ex1']

You may be wondering about value assignment as a list instead of simple strings, well, this is because parse_qs returns a dictionary with keys and values as sequences since a parameter can be supplied with multiple values (ie: “?cat=ACTION&cat=HORROR&cat=COMEDY”).

Now that the query string data has been updated all I have to do is to serialize it back to a simple string and update the original SplitResult.

from urllib import urlencode

url_data._replace(query=urlencode(qs_data, True))

The second argument passed to urlencode (True) tells the function that we are passing sequences as values so it will handle them according.
The new modified url can be now retrieved by calling:

url_data.geturl()

to summing up, this is the full code:

from urllib import urlencode
from urlparse import urlsplit, parse_qs

# parse original string url
url_data = urlsplit(url_string)

# parse original query-string
qs_data = parse_qs(url_data.get('query'))

# manipulate the query-string
if not 'target_parameter' in qs_data:
    qs_data['target_parameter'] = ['tp1']
qs_data['extra_parameter'] = ['ex1']

# get the url with modified query-string
url_data._replace(query=urlencode(qs_data, True)).geturl()

That’s all folks! If you enjoyed this post don’t forget to share it using the buttons below ;)