A base class to define functors.

Updated Gribouillis 2 Tallied Votes 711 Views Share

A functor, or function object is sometimes a convenient way to encapsulate helper functions and data needed to achieve a small task, and also to share data between calls to the same function. This snippet defines a base class [b]functor[/b] which permits an easy functor creation for python 2.

# python 2.7
# functors.py
import sys

__version__ = '0.0.2'
__all__ = ["functor", "staticfunctor", "classfunctor"]

class metafunctor(type):
    wrapmore = staticmethod(lambda f: f)
    
    """helper metaclass for functors"""
    def __new__(meta, class_name, bases, new_attrs):
        if "__init__" in new_attrs:
            raise TypeError("__init__() forbidden in functor class '%s'" % class_name)
        klass = type.__new__(meta, class_name, bases, new_attrs)
        def wrapper(*args, **kwd):
            return klass()(*args, **kwd)
        wrapper = meta.wrapmore(wrapper)
        from functools import update_wrapper
        update_wrapper(wrapper, klass)
        wrapper._functor_class = klass
        return wrapper

class metastaticfunctor(metafunctor):
    wrapmore = type('_staticmethod', (staticmethod,), dict())
    
class metaclassfunctor(metafunctor):
    wrapmore = type('_classmethod', (classmethod,), dict())


class functor(object):
    """A base class to define functors, a special kind of function objects.
    
    Author: Gribouillis, for the python forum at www.daniweb.com
    License: This program has been placed in the public domain
    Created and published June the 27th 2011
    
    Tests: tests can be run on the command line with 'python -m doctest -v Functor.py'
    
    Usage:
        Subclassing functor defines a function instead of a class, for example
        
            class myfunction(functor):
                def __call__(self, what):
                    print "called with argument", repr(what)
        
            myfunction("hello world") # myfunction is now a function and not a class
            
        Each time myfunction is called, a *new* instance of the hidden class is created and
        the instance's __call__ method is called instead. The hidden class itself can be accessed
        as myfunction._functor_class.
        
        The benefit of defining functors instead of functions is to encapsulate helper data
        and methods for the task executed by the function. Here is a more complete example
        of a functor to reverse the letters in each word of a sentence
        
        >>> class reverse_words(functor):
        ...     'A functor to reverse words in a sentence.'
        ...     def __call__(self, sentence):
        ...         return self.regex.sub(self.subf, sentence)
        ...     import re
        ...     regex = re.compile(r'[a-zA-Z_]\w*')
        ...     def subf(self, match): # helper functions and data are encapsulated in the functor
        ...         return match.group(0)[::-1]
        >>> print reverse_words("please reverse words in this test sentence.")
        esaelp esrever sdrow ni siht tset ecnetnes.
    """
    __metaclass__ = metafunctor
    __slots__ = ()
    
    def __call__(self, *args, **kwd):
        raise NotImplementedError(
            ("__call__() not implemented for functor", self.__class__.__name__))

if sys.version_info >= (3,):
    exec("""class functor(functor, metaclass = metafunctor):
        __slots__ = ()""")
functor = functor._functor_class


class staticfunctor(functor):
    __metaclass__ = metastaticfunctor
    __slots__ = ()
staticfunctor = staticfunctor._functor_class

class classfunctor(functor):
    __metaclass__ = metaclassfunctor
    __slots__ = ()
classfunctor = classfunctor._functor_class
Gribouillis 1,391 Programming Explorer Team Colleague

The above functor base class can also be used to define methods, static methods and class methods, here is an example

class A(object):
    def __init__(self):
        self.value = None
        
    class foo(functor):
        def __call__(self, this, n):
            this.value = n

    class sbar(functor):
        def __call__(self, stuff):
            print "sbar called with", repr(stuff)
    sbar = staticmethod(sbar)
    
    class cbar(functor):
        def __call__(self, cls, stuff):
            print "cbar called with", cls, repr(stuff)
    cbar = classmethod(cbar)

a = A()
a.foo("value set by the foo functor")
print a.value
a.sbar("static call with instance")
A.sbar("static call with class")
a.cbar("classmethod call")
""" my output --->
value set by the foo functor
sbar called with 'static call with instance'
sbar called with 'static call with class'
cbar called with <class '__main__.A'> 'classmethod call'
"""

Notice that the 'method functor' has 2 implicit arguments: the functor object and the A instance.

commented: interesting indeed +15
TrustyTony 888 ex-Moderator Team Colleague Featured Poster

Wikipedia function object Python section is using simple object with __call__ and __init__ for start value as Accumulator. Could you give example of benefit of your approach disallowing the __init__?

Gribouillis 1,391 Programming Explorer Team Colleague

There is a big difference with a traditional functor. A traditional functor is an object initialized once and called several times. It maintains a state between the calls. In my approach, an instance is automatically created every time the function is called. The benefit is that the functor has exactly the same interface as a function: it does not need to be instantiated explicitely, and it can be called recursively, or in different threads. The idea here is not particularly to keep a state between subsequent calls (such static data could be stored in the class' dict, for example self.__class__.n += x ). The key idea is encapsulation: from the client code, the fact that it is a functor is an invisible implementation detail.

Since the class is automatically instantiated for each call, there is no need for an __init__ function. Initialization code can be written in the __call__ method if it is needed. The data stored in self is meant to be shared with the helper methods during an execution, but not between calls.

It's a much more dynamic object than a traditional functor.

matricol -8 Junior Poster in Training

python looks cool and simple but where can I begin .. do you have maybe a video tut ? thanks for the help

Ezzaral 2,714 Posting Sage Team Colleague Featured Poster

Perhaps you can begin by reading "Starting Python", prominently stickied at the the top of the forum, rather than hijacking someone's code snippet with off-topic questions.

Gribouillis commented: Good idea ! +13
Gribouillis 1,391 Programming Explorer Team Colleague

In a chain reaction, each atom explodes once and the pieces go collide other atoms which explode, etc. This example uses a functor chain_reaction(item, explode) which takes an initial item (atom) and an explosion function to recursively "collide" new atoms. Each atom must be generated only once in the chain reaction. This functor can be useful to traverse any graph in python

class chain_reaction(functor):
    """Generate item and each item created by calling explode(item), and recursively.
    Each item is generated only once and exploded once.
    arguments:
        @ item : a hashable python object
        @ explode : a function explode(single_arg) --> sequence of items
    """
    def __call__(self, item, explode):
        self.marked = set()
        self.explode = explode
        return self.reaction(item)
    
    def reaction(self, item):
        if item in self.marked:
            return
        self.marked.add(item)
        exp = self.explode(item)
        yield item
        for x in exp:
            for y in self.reaction(x):
                yield y
                

# EXAMPLE: CRAWLING THE WEB IS A CHAIN REACTION
if __name__ == "__main__":
    from urllib2 import urlopen
    from BeautifulSoup import BeautifulSoup
    from itertools import islice
    
    def explode_url(url):
        try:
            soup = BeautifulSoup(urlopen(url).read())
            for link in soup.findAll('a'):
                try:
                    if link["href"].startswith("http://"):
                        yield link["href"]
                except Exception:
                    pass
        except Exception:
            pass
    
    reaction = chain_reaction("http://www.python.org", explode_url)
    
    for link in islice(reaction, 0, 15):
        print link
        
""" my output -->
http://www.python.org
http://docs.python.org/devguide/
http://bugs.python.org/
http://python.org/about
http://wiki.python.org/moin/
http://www.python.org/moin/?action=rss_rc
http://moinmo.in/
http://www.jurawiki.de/
http://www.jurawiki.de/RecentChanges?action=rss_rc&ddiffs=1
http://www.jura.uni-sb.de
http://www.bundesverfassungsgericht.de/pressemitteilungen/bvg11-049.html
http://cmiskp.echr.coe.int/tkp197/view.asp?action=html&documentId=860968&portal=hbkm&source=externalbydocnumber&table=F69A27FD8FB86142BF01C1166DEA398649
http://www.echr.coe.int/echr/Homepage_EN
http://www.coe.int/
http://www.coe.int/lportal/web/coe-portal/accessibility
"""
Gribouillis 1,391 Programming Explorer Team Colleague

With a little work, the previous functor can be turned into a decorator wrapping the explosion function. Here is the new formulation with an application to traversing a hierarchy of directories

from functools import update_wrapper

class chain_reaction(functor):
    """Decorate an explode function to create a chain reaction.
    
       Usage:
           
           @chain_reaction
           def explode(atom):
               'function with a single atom argument, returning a sequence of atoms'
               ...
               yield another_atom # for example
               
            for atom in explode(an_atom):
                # loop over atoms recursively exploded
        
        An atom can be any hashable python object.
    """
    def __call__(self, explode):
        self.marked = set()
        self.explode = explode
        wrapper = lambda item: self.reaction(item)
        update_wrapper(wrapper, explode)
        return wrapper
    
    def reaction(self, item):
        if item in self.marked:
            return
        self.marked.add(item)
        exp = self.explode(item)
        yield item
        for x in exp:
            for y in self.reaction(x):
                yield y
                
if __name__ == "__main__":
    from itertools import islice
    import os
    
    @chain_reaction
    def hierarchy(folder):
        """Recursively generate the descendant directories of a directory"""
        p, dirnames, filenames = next(os.walk(folder))
        return (os.path.join(folder, d) for d in dirnames)
    
    root =  os.path.expanduser("~")
    for d in islice(hierarchy(root), 0, 100):
        print d
TrustyTony 888 ex-Moderator Team Colleague Featured Poster

Neat hack, at least if you do not consider that you can get same result as example use with oneliner:

import os
from itertools import islice
print('\n'.join(value[0] for value in islice(os.walk(os.path.expanduser("~")), 0, 100)))

Thanks for the os.path.expanduser('~'). It actually works also in Windows.

Gribouillis 1,391 Programming Explorer Team Colleague

Neat hack, at least if you do not consider that you can get same result as example use with oneliner:

import os
from itertools import islice
print('\n'.join(value[0] for value in islice(os.walk(os.path.expanduser("~")), 0, 100)))

Thanks for the os.path.expanduser('~'). It actually works also in Windows.

I agree that the example was somewhat artificial. Here is a variation which shows how easily you can use chain_reaction to display a tree. I include an improved version of chain_reaction() as an attached file

if __name__ == "__main__":
    from easygui import codebox
    from itertools import islice
    import os
    
    @chain_reaction(unique=False)
    def levels((p, n, maxdepth)):
        if os.path.isdir(p) and (maxdepth is None or n < maxdepth):
            for name in sorted(os.listdir(p)):
                yield (os.path.join(p, name), n + 1, maxdepth)


    def dirlines(p, maxdepth=None):
        for q, n, d in levels((p, 0, maxdepth)):
            s = ("    " * n) + os.path.basename(q)
            yield s + "/" + ("..." if maxdepth is not None and n >= maxdepth else "") \
                                                                   if os.path.isdir(q) else s
            
    home = os.path.expanduser("~")
    
    codebox(msg = home, text="\n".join(islice(dirlines(home, maxdepth=3), 0, 100)))
TrustyTony 888 ex-Moderator Team Colleague Featured Poster

Yes this example is less trivial, you can save effort of doing and debugging yourself depth-first walk:

import os
import itertools as it

def  process(name, depth, max_depth=None):
    return (depth*'   '+ (name + '/...' if max_depth and depth >= max_depth else
                        (name + '/' if max_depth else name)
                        )
           )

def dir_to_depth(current, depth, max_depth, function=process ):
    yield function(os.path.basename(current), depth, max_depth)
    if depth < max_depth:
        listing = sorted(os.listdir(current))
        dirs = [dn for dn in listing if os.path.isdir(os.path.join(current,dn))]
        for d in dirs:
            for deeper in  dir_to_depth(os.path.join(current, d), depth + 1, max_depth, function):
                yield deeper
        dirs = set(dirs)
        for f in [fn for fn in listing if fn not in dirs]:
            yield function(f, depth+1)

print('\n'.join(it.islice(dir_to_depth(os.path.expanduser('~'), 0, 3), 0, 100)))
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.