This snippet composes functions of the form
func(infile, outfile) which read an input file and transform its content to write an output file. A function
compose((f, g, h,...)) is defined which returns a new such file transformation function which successively applies f, g, h, using in-memory temporary files.
# Title: Compose file transformation functions in python # Date: 2011 Oct 27 # Author: Gribouillis for the python forum at www.daniweb.com # License: public domain # Use this code freely in your programs. try: from io import StringIO # python 3 except ImportError: from cStringIO import StringIO # python 2 def last_marked(iterable): """Generate pairs (islast, item) with the iterable's items The boolean value 'islast' is False unless the item is the last in the sequence. """ it = iter(iterable) try: prev = next(it) except StopIteration: raise for item in it: yield False, prev prev = item yield True, prev def compose(file_transforms): """Compose a sequence of file transformation functions to create a new one. Here a file transformation function has the signature def func(infile, outfile): ... where infile is a file opened for reading and outfile is a file opened for writing. compose() takes a sequence of file transformation functions and returns a new file transformation function which runs each function one after the other, using StringIO (memory files) as temporary files. For example, given 3 such functions f, g, h: >>> F = compose((f, g, h)) >>> F(open("infile.txt"), open("outfile.txt", "w")) """ def composition(infile, outfile): fh = infile for islast, func in last_marked(file_transforms): temp = outfile if islast else StringIO() func(fh, temp) fh, temp = temp, None if not islast: fh.seek(0) return composition
About the Author