This snippet composes functions of the form func(infile, outfile) which read an input file and transform its content to write an output file. A function compose((f, g, h,...)) is defined which returns a new such file transformation function which successively applies f, g, h, using in-memory temporary files.

Edited 5 Years Ago by Gribouillis: n/a

# Title: Compose file transformation functions in python
# Date: 2011 Oct 27 
# Author: Gribouillis for the python forum at
# License: public domain
# Use this code freely in your programs.

    from io import StringIO # python 3
except ImportError:
    from cStringIO import StringIO # python 2

def last_marked(iterable):
    """Generate pairs (islast, item) with the iterable's items
    The boolean value 'islast' is False unless the item is the last in the sequence.
    it = iter(iterable)
        prev = next(it)
    except StopIteration:
    for item in it:
        yield False, prev
        prev = item
    yield True, prev
def compose(file_transforms):
    """Compose a sequence of file transformation functions to create a new one.

    Here a file transformation function has the signature
        def func(infile, outfile):
    where infile is a file opened for reading and outfile is a file opened for
    compose() takes a sequence of file transformation functions and returns a
    new file transformation function which runs each function one after the other,
    using StringIO (memory files) as temporary files. For example, given 3 such
    functions f, g, h:
        >>> F = compose((f, g, h))
        >>> F(open("infile.txt"), open("outfile.txt", "w"))
    def composition(infile, outfile):
        fh = infile
        for islast, func in last_marked(file_transforms):
            temp = outfile if islast else StringIO()
            func(fh, temp)
            fh, temp = temp, None
            if not islast:
    return composition