Andy McKay linked to a bit of interesting data from his twitter account. I wanted to be able to visualize the data so I used it as an excuse to try out flask, a python micro web framework, and some other stuff. It was all a bit of a yack shave because I could have done the whole thing as a static page but what the hell. It also led me to finally moving my blog off of slicehost and over to linode which of course led to an upgrade of wordpress which led to trying wordpress behind nginx and so on and so forth. Such is the life of a techie.

Anyway without further ado, click through to see all locations in BC where a marijuana grow operation was found by the police in the last year. And here is the raw data.

The visualization shows a circle at the location and the size is proportional to how many plants were seized. A scaling factor was applied to the circles so that they are still visible, and proportional, at all zoom levels.

Also, don’t be surprised if the whole thing comes crashing down because this was thrown together as an experiment.

Installing supervisor on Natty Nat will undoubtedly throw the error ImportError: cannot import name fixtag

The root cause of this error is the python-meld3 library and the following import statement:

from xml.etree.ElementTree import fixtag

The problem has been fixed in version 0.6.7 of python-meld3. Therefore, before you install supervisor, do the following on the command line as root:

aptitude install python-meld3
pip install meld3==0.6.7

The above will install version 0.6.5 of python-meld3 and then pip will upgrade it to the latest. You can then install supervisor without any problems.

If you want you can read the gory details of the problem in python-meld3.

The discussion on stackoverflow about handling different dataType returns from AJAX calls really got me thinking about the best pattern for POSTing form data to the server.

This really ties in nicely to my previous blog post about Patterns of Organization when mixing MVC frameworks with javascript. If you are using your own RESTful API when creating a new object in your domain (a.k.a. POSTing some form to create a new foobar) what is the best pattern to follow?

Let’s first assume that you are POSTing your form via AJAX because you just popped up a lightbox that contained the form rather than doing a whole round trip to render a whole new page with the form. Therefore you have some code that looks like this:

$("form#foobar").submit(function() {
    $.ajax({type: 'POST',
            url: '/foobar',
            data: $(this).serializeArray(),
            success: function(data) {
                // do something with data;
            }
    });
});

Under the old MVC pattern you probably re-rendered the form if the user gave you bad data or forgot to fill out a field and then on success, you either re-directed them to the new page showing them their new data or immediately rendered that new page in the POST controller. However, in the javascript example above it becomes quite cumbersome to expect multiple possible returns. We do not want to be parsing the returned HTML to figure out if it is a re-rendered form because of a validation failure or if it is the HTML of our new item. The obvious solution is to return JSON which is easy to inspect. But I really like the idea of sending back an error HTTP response code if validation fails.

Firstly, if I am within my own application and retrieving data I do not want to write code to turn JSON into HTML. I know how I want my data to look on the page so why not just return it straight away into HTML. This allows me to keep my markup in templates where it should be. By returning validation failures as, well, failures, it allows me to instantly handle two different return types – JSON for an error code return, and HTML for a new item return. If JSON is returned I can easily pick out the error message and update the DOM of the form to display the erred fields. But if my POST was successful, I then kill the form and slide in my new HTML to update my existing list of data all nicely formatted.

Now my javascript looks something like this:

$("form#foobar").submit(function() {
    $.ajax({type: 'POST',
            url: '/foobar',
            data: $(this).serializeArray(),
            dataType: 'html',
            success: function(html) {
                $("div#data").append(html);
            },
            error: function(XMLHttpRequest, textStatus, errorThrown) {
                var errorObject = JSON.parse(XMLHttpRequest.responseText);
                // update form with stuff in errorObject
            }
    });
});

I’ve been thinking a lot lately about the emerging patterns in web development in the post Web 2.0 world. AJAX and javascript are mainstream now and no longer just parlor tricks. It is de rigueur to implement RESTful web services and have an API. But with all these AJAX calls whizzing about and JSON, XML, HTML, and whatnot being passed to and fro, it is easy to end up with a tangled mess. The one thing about mixing your web framework with javascript is that you now have a lot of code going on that leaks outside of your nice, neat Model-View-Controller (MVC) pattern.

I have seen javascript used extremely sparingly in an application all the way up to extremely heavy usage with javascript controlling not only logic but the entire layout of the page. But to keep a nice separation of responsibilities I believe the server side code should be responsible for laying out the structure of a page. We can then componentize the dynamic data bits and create a nice structure whereby javascript widgets are responsible for making AJAX calls to fill in those components.

For example, you have a page which is rendered by your MVC framework. But on that page is a div that contains a list of dynamic data. When the page is rendered there is just a placeholder for the list. Using JQuery event handlers you then make an AJAX call to fetch the data from your own RESTful API and populate the div.

What I like about this pattern is:

  • you still have a strong MVC pattern
  • you are eating your own dog food in terms of your API
  • initial page loads are quick
  • you can constantly poll for new data to give your application a real-time feel

An important point to note in this pattern is that you can still have your MVC framework responsible for rendering the data into HTML. Your RESTful API should accept a format parameter to set whether the return type is JSON, XML, or HTML. Internally, your own application can use the HTML return data type so that you do not have to write a bunch of javascript to turn the JSON into HTML. This way you can keep all your markup in templates and not scattered in javascript code.

If you are running Pylons 0.9.7 with the latest “stable” AuthKit 0.4.4 you may run into some troubles using the forward AuthKit method. If you are like me, you followed all the smatterings of examples scattered about the Pylons and AuthKit site only to keep receiving a ForwardRequestException anytime you hit an authorize protected controller.
Continue reading »

In Python, you can declare a function within another function and the scope of local variables in the outer function carry through to the inner function. However, you have to be careful with the re-assignment of the variable inside of the inner function due to some funkiness that may or may not be a bug (I say its a bug but who am I?)

Continue reading »

With JQuery UI Tabs v.3 there isn’t an option to make the tabs bookmarkable. In other words when you click on the tabs the URL does not change therefore you can’t link or bookmark anything other than the default tab.

Here is how I solved the problem… [UPDATED!]

Continue reading »

If you don’t want simpletal filtering out   then the following function needs to be added to simpleTAL.py:

 def skippedEntity (self, name):
     self.log.info ("Recieved skipped entity: %s" % name)
     self.characters( unichr(sgmlentitynames.htmlNameToUnicodeNumber.get (name, 65533)))

If you are using HTML templates with SimpleTAL it will pass through HTML comments (previous to version 2.2 this was not true.)  However, the same cannot be said of XML templates.  In version 3.8 of SimpleTAL, support was introduced for passing through HTML comments but only if you have PyXML installed! Continue reading »

This is one miserable problem with a very simple solution. It has to do with multiple python interpreters in mod_python v.3.2.8 and multiple virtual hosts. To remedy the situation, add the following line inside your <VirtualHost> declaration in your apache conf file:

PythonInterpreter main_interpreter

Restart server and all will be better. For more info see trac ticket #3455

UPDATE: All is not well in the world of mod_python and trac.  The above solution does improve matters but there are still issues.  Many are suggesting to use mod_wsgi with trac to manage multiple sites on virtual hosts.  Is this the beginning of the end of mod_python?

© 2012 rootsmith blog Suffusion theme by Sayontan Sinha