Gareth Latty (Lattyware)

Mark and Recapture decorators in Python.

Decorators are a nice little feature in Python, allowing you to transform functions on the fly in a method that looks nice in code. One use I find myself gravitating towards is as a callback marker - avoiding ugly ‘connect’ function calls. This is a great use that is easy to do, however it does have a downside - decorators happen before instantiation, meaning that the callbacks don’t have `self` filled.

My solution to this problem is the idea of a mark and recapture decorator. We create a decorator that marks a function, then, once the class is constructed, we recapture it.

def callback(*args):
    def decorate(f):
        try:
            f._marks.add(args)
        except AttributeError:
            f._marks = {args}
        return f
    return decorate

Here is our decorator. We take advantage of the fact that Python functions are first class objects, and simply mark them by adding an attribute to them. We use a set so that we can run the decorator multiple times to use the same function for multiple callbacks.

def connect_callbacks(obj):
    for _, f in inspect.getmembers(obj, inspect.ismethod):
        try:
            for mark in f.__func__._marks:
                connect(mark, f)
        except AttributeError:
             pass

This is out connection function. This searches through the given object and finds all the marked functions. We do this by trying to access the `_marks` attribute on the `__func__` attribute. The `__func__` attribute is the original, unbound function the method runs, which is the one we marked earlier. We then run connect (which would be your callback creating function) with the mark (which can be any data you want) and the function, or pass if it doesn’t have a mark. We can easily run this connection function in the constructor for our object and the mark and recapture pattern gives us what we want.

Follow this link for an example of the naive version (which gives us the unbound method) vs this method on ideone (which gives you the output too!), along with code highlighting.

Generating words at random - what I learnt from Ludum Dare 22.

I posted this up on the Ludum Dare blog as well, but thought it’d go well here too.

So, I didn’t manage to finish Ludum Dare 22 as I had to travel home from Uni halfway through and ran out of time.

My aim was to create a procedurally generated universe and allow the player to travel around finding out if they are alone as sentient life in the given universe. Given the time issues I really didn’t get much done, but I did focus on a particular problem, I wanted to name planets so players could remember where they had been. How do you create words that are pronounceable without just having planets called ‘Fork’ and ‘Television’. Words like these:

• fanglas
• jubbensetrier
• amenet
• moquiets
• mystilaxation
• consutey
• untive
• curchers
• anchottollon
• symborse
• prasting
• weeloats
• dupliquding
• autobency
• proscolicends

Well, the answer came in the form of Markov chains, a cool little trick that allows you to do this quite simply. Afterwards this still intrigued me, and I finally had some time to finish up my script,  wordgenerator.

It’s a Python library and command line application, so it’s usable by pretty much anyone. If you have trouble thinking up names for things in general, it can be a great help, and as a library it goes hand-in-hand with any procedurally generated content. It’s GPLv3ed, so feel free to use it in any way that fits the license. The above is actual output from my script. You can change the output via a variety of options (explained in the above link) and by changing the input dictionary of words to generate from, for example, using an Italian one:

• impiate
• aliersi
• inaudartererai
• ottardiscrerei
• addoluccio
• deredicassella
• coibinarei
• impresto
• accreste
• storano

While nothing revolutionary (Markov chains are pretty well known), the script performs pretty well and saves a bit of work. I think it’s pretty cool, and surprisingly funny to see the output you get, so if you find yourself needing names in your next Ludum Dare game, feel free to use it.

Teaching CompSci - Code Reuse

Something I have noticed as a big trend in Computer Science is the act of telling students to forgo the standard library when trying to solve tasks. I get this - when teaching someone the basics of CompSci, you have to start with basic problems, and, for obvious reasons, most of those have already been solved by the standard library.

This has lead to questions asking students to do things without using the standard library, and that’s fine. I get the purpose of that, it makes sense, and there is nothing inherently wrong with it. The issue is with the fact that it is not being made clear to students that this is an academic exercise. I constantly see people re-implementing trivial (or not-so-trivial) functions that are provided by the standard library, and constantly hear people referring to functions provided by a language as ‘cheating’ or ‘taking the easy way out’.

Not reinventing the wheel is something that makes a good programmer. Code reuse has so many benefits - increased stability and less time working on a problem can only be a good thing, and yet I constantly see students getting this misunderstanding that they should implement everything by hand.

Does this mean people teaching should have to think up simple, and yet currently unsolved problems? Of course not. The answer is to simply state clearly, in every case you give a problem like this, that the optimal solution is to simply use the standard library, but for the case of this exercise, that isn’t allowed. This needs to be stated clearly, otherwise, you punish students for following good practice and finding the best solution.

There are many horror stories about reimplementing functionality, and I strongly believe that this kind of teaching causes these horrors.

So I implore all teachers and lecturers, make it clear that what you are asking is only for the purpose of the exercise, and bad practice.

Ludum Dare #22

So, I’ve officially declared I’m going to go in for Ludum Dare again, so I’ll have to make a game in 48 hours.

It’s a really fun thing to do, so I’m definitely looking forward to giving it another go, and hopefully do a bit better this time. I really liked my last entry ‘Unrest’ - which got top 25 for both innovation and theme, but this time I want to try and make something with more gameplay, and hopefully a little more polish and flair - I’m no artist, but I want to try and get better graphics and some sound done this time.

If you want to follow my progress, I’ll be updating on the big Ludum Dare blog. My plan is to do a timelapse this time too, so you can watch me work afterwards.

Here is to hoping it’s an interesting theme and I can think of something good!

Lazy Attributes - Efficient attribute generation in python.

How often do you end up with a simple class with a load of attributes that need to be generated based on other data? It happens a lot, and often, you will do it two ways.

Some will just cache the data, generating it in the constructor. This method works well enough if you want to access those attributes all the time. They are generated as the class is created, then can be accessed quickly.

Some will create functions to generate the data, creating it upon request. This is also a nice enough method, the property builtin hides the ugliness of ‘getter’ methods, and if you are creating loads of objects of the class and not accessing the attributes often, this is great. And it means no overhead on object creation.

However, what if you are creating a lot of objects, but sometimes need to access those attributes a lot? Neither offers us a great solution. If we do the former, we end up having slowdowns creating objects, if we go for the latter, we’ll get hit when we access those attributes. What if we need the best of both worlds?

Well, there is. Due to the beauty of python’s decorators, we can create a simple function to create lazy attributes. These are not created until the attribute is requested, however, when it is, it caches the value, so it can call it up without generating it again. Note that this decorator will create attributes with the name of the function prefixed with an underscore.

def lazy(func):
	""" A decorator function designed to wrap attributes that need to be
	generated, but will not change. This is useful if the attribute is  
	used a lot, but also often never used, as it gives us speed in both 
	situations."""
	def cached(self, *args):
		name = "_"+func.__name__
		try:
			return getattr(self, name)
		except AttributeError:
			value = func(self, *args)
			setattr(self, name, value)
			return value
	return cached

You simply then add the decorator to any ‘getter’ method, and use property like always. It looks just like the second example, except with the decorator added in. In some simple tests:

0 Accesses:
C: 0.01067304611206
G: 0.00860595703125
L: 0.00849795341491

1000 Accesses:
C: 0.08756899833679
G: 1.58509612083435
L: 0.65241217613220

You can see that when not accessing the attribute, the Lazily generated class takes roughly the same amount of time as the generated on the fly class, while the cached class takes longer as it generates the attribute upon creation. On the other hand, with 1000 accesses of the attribute, you can see the lazily generated class massively outperforming the generated one as it only needs to do it once per object.

Naturally, situations differ, and this isn’t always appropriate, in the above example, cached gives such a small increase in time for object creation, you would just use it, as it’s still faster when accessing, however, if you had a lot of cached values, lazily generating them may become more and more worthwhile, especially if they take a long time to compute.

If you want to run the test for yourself, then feel free to grab a copy of the source code.