Basics of MetaProgramming

Before going to have practical examples and usage of meta programming first we understand what is meta programming? The answer to this question lies in the word ‘meta’. We have heard the word meta data that is ‘data about data’. In a similar way, meta programming is actually programming about/for programming or you can say it is the code that manipulates code.

Metaprogramming is a deeper magic that you might not need usually but sometimes we have problems and their solution lies in that magic.

The journey of metaprogramming (magic) starts from simple decorators and end up to the insane part for python, the meta classes but in this part of article we will only cover decorators.

Let’s start our journey from a simple problem of logging. We have different functions and we want that whenever a function is executed, timestamp and function's name are also printed. Simplest solution that comes in mind is to simply use print statements wherever you need like this.

> mul(3, 2)
> 1425845487.15
> mul
> 6
> div(3, 2)
> 1425845492.12
> div
> 1>>> def add(x, y):
> ... print(time())
> ... print('add')
> ... return x + y
> ...
> def sub(x, y):
> ... print(time())
> ... print('sub')
> ... return x - y
> ...
> def mul(x, y):
> ... print(time())
> ... print('mul')
> ... return x \* y
> ...
> def div(x, y):
> ... print(time())
> ... print('div')
> ... return x / y
> ...
> add(2,3)
> 1425845460.5
> add
> 5
> sub(3, 2)
> 1425845478.74
> sub
> 1

> type(10)
> <class 'int'>
> type(10.1)
> <class 'float'>
> type('Hello World')
> <class 'str’>
> > >

But isn’t looks weird? We are writing almost same lines everywhere, which is not a good idea. We are repeating similar things in all functions. It should be in one single place in order to follow the line “Don’t repeat yourself (DRY)”.What can we do? To solve the problem first we have to understand python basic concept that everything is an object.

Everything is an object: In python everything is an object even methods and class. Methods and class are first class object in python. They can be used same as any other common variable. Every object has some type. Since classes are object, they have type too. Let’s see some simple examples and don't worry we are moving towards our goal, the solution to the problem.

> type(str)
> <class 'type'>
> type(int)
> <class 'type'>
> class Foo(object):
> ... pass
> ...
> type(Foo)
> <class 'type'>
> foo = Foo()
> type(foo)
> <class '**main**.Foo'>
> def bar(object):
> ... pass
> ...
> type(bar)
> <class 'function'>
>

So we are now confirmed that everything is an object so we can manipulate methods to solve our problem. Manipulation of methods is done though decorators.

Function Decorator:

Decorator are function that takes a function and returns a modified function. We can use decorator to modify our functions which will add our desired functionality to our actual function. Decorators are easy to use and understand so let’s do it.

>>> from time import time
>>> def log(func):
... def wrapper(*args, **kwargs):
... print(time())
... print(func.__name__)
... return func(*args, **kwargs)
... return wrapper
...
>>> @log
... def add(x, y):
... return x + y
...
>>> @log
... def sub(x, y):
... return x - y
...
>>> @log
... def mul(x, y):
... return x * y
...
>>> @log
... def div(x, y):
... return x / y
...
>>> add(3,2)
1425846110.02
add
5
>>> sub(3, 2)
1425846121.06
sub
1
>>> mul(3, 2)
1425846140.7
mul
6
>>> div(3, 2)
1425846147.38
div
1
>>>

Wow, that looks pretty good. Now we have nice elegant code with less lines but with same functionality. Also this is a good idea with respect to SoC (Separation of Concerns).

In most cases in object oriented environment we work with classes. Suppose we have a class with lots of method like this (below), although it has only four methods but just imagine it has lots of functions and we have to repeat our decorator a lot more time like this.

class Binary(object):
    def __init__(self, x, y):
        self._x = x
        self._y = y
    @log
    def add(self):
        return self._x + self._y

    @log
    def sub(self):
        return self._x - self._y

    @log
    def mul(self):
        return self._x * self._y


    @log
    def div(self):
        return self._x / self._y

What if we can remove these decorators on every method which is still repetition. Why not find a way to make it more simple and clean by applying a single decorator to class which will perform logging in all methods in a class. Here it is.

Class Level Decorator:

def decorate_methods(decorator):
    def decorate(cls):
        for attribute in cls.__dict__:
            if callable(getattr(cls, attribute)):
                setattr(cls, attribute, decorator(getattr(cls, attribute)))
        return cls
    return decorate

What we are doing here is that, we are passing class to decorator function. This function takes another decorator function as an argument to decorate class methods. Class decorator first garbs all methods from class dictionary and replaces dictionary with modified methods (after applying given decorator).

@decorate_methods(log)
class Binary(object):
def __init__(self, x, y):
self._x = x
self._y = y

def add(self):
return self._x + self._y

def sub(self):
return self._x - self._y

def mul(self):
return self._x * self._y

def div(self):
return self._x / self._y

Now test it in interactive shell.
>>> b = Binary(3, 2)
>>> b.add()
1426083188.75
add
5
>>> b.sub()
1426093132.35
sub
1

Here arises an another problem. What if we don’t want logging in every function but we want to decide at call time weather we need logging or not. This can be done by adding arguments to the wrapped function.

>>> def log(func):
...     def wrap(*args, log=False, **kwargs):
...         if log:
...             print(func.__name__)
...         return func(*args, **kwargs)
...     return wrap
...
>>> @log
... def add(x, y):
...     return x + y
...
>>> add(3,2)
5


>>> add(3,2, log=True)
add
5
>>>

Type Validation:

There are situations when we want to restrict some input arguments types. In above examples, we can pass int, double, float etc. Here we will find a way to validate arguments type that we intended to be passed by user.

>>> def validator(typ=int):
...     def decorator(functn):
...         def wrapper(*args, **kwargs):
...             if isinstance(args[0], typ) and isinstance(args[1], typ):
...                 return functn(*args, **kwargs)
...             else:
...                 raise TypeError('Both arguments should be of type {}'.format(typ))
...         return wrapper
...     return decorator
...
>>> @validator(int)
... def add(a, b):
...     return a + b
...
>>> add(2,3)
5
>>> add(2,3.5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in wrapper
TypeError: Both arguments should be of type <class 'int'>
>>>

These are simple tasks that can be done using decorators. In second part of this article we will talk about meta classes to manipulate classes and their method and we will talk about how django framework uses meta classes to do fancy things.