Hi,

In my current project I'm trying to compose some objects at runtime based on a string containing the object-configuration. Based on the "Favour composition, not inheritance" pattern, but done runtime.

My usecase:
I have a downloader that downloads web-pages. The downloader is configured with a set of request-handlers and response-handlers.
First it construct a request and lets every request-handler do something with the request. Then it passes the request to the downloadStrategy which constructs a response. The response is then passed to the response-handlers before it's returned to the caller.

With this configuration I can add and remove handlers to get different behaviours from the downloader. This is implemented and works like a charm.

What I want to do is to configure the downloader at runtime based on a configuration-string stored in a database. That way I don't have to do any codechanges to alter the behaviour of my downloader. I'm almost done coding this, but I'm quite stuck at the moment. My configurer automatically imports packages and instantiates classes, but it's not as versatile as I want it to be.

Current configuration-string syntax:
moduleName#className?kwarg1=value1&kwarg2=value2
Example:
remote.download#EqualityChecker?threshold=5

This makes the configurer import the remote.download module and constructs a EqualityChecker(threshold=5)

I can also reference already constructed objects using absolute notation:
remote.download#CheckerRunner?checker=|EqualityChecker?threshold=5|
The EqualityChecker object in both examples will be the same instance of the class (singleton), based on the keyword arguments.

My problem is that I can't find a good way to represent a list of references in my configurationstring. say I want the checker keyword argument for the CheckerRunner to be a list of other objects already constructed.

Here's the code I have so far. Hope someone can take a look and give me some ideas how to represent the list of references in the configuration-string.

Sorry for the long post and my inability to clearly state what I'm having trouble with :P

import re
class ObjectConfigurer(object):
    """
    Class for configuring objects at runtime based on a configuration-string.
    """
    nameExp = re.compile(r"(?P<module>(?:[^#]*#)?)(?P<class>[^?]*)(?P<arguments>.*)")
    argsExp = re.compile(r"[?&](?P<key>[^=]*)=(?P<value>(?:[|][^|]*[|])|(?:[^&]*))")
    
    def __init__(self):
        self.modules = {}
        self.objects = {}
        self.defaultModule = None
    
    def configure(self, configurationString, defaultModule=None):
        """
        Creates a list of objects based on their respective configuration-string
        """
        if defaultModule: self.defaultModule = defaultModule.strip()
        return self.parseString(configurationString)
        
    
    def parseString(self, s):
        """
        Split the comma-separated string and run parseObject for each value
        """
        stack = [self.parseObject(obj) for obj in s.split(",") if not len(obj.strip()) == 0]
        return stack

    def parseObject(self, obj):
        """
        Parse the string into module, classname and optional arguments.
        Returns an instance of the class, created with the given arguments as keyword 
        arguments to the constructor. Only one instance of a class will be constructed
        for each set of constructor arguments. Different ordering of arguments will
        result in a different object being created.
        
        Format for the string
        [moduleName#]className[?key=value[&key2=value2 ...]]
        ModuleName is optional
        Arguments are optional
        If moduleName is missing, the specified defaultModule will be used.
        Self-reference is allowed by pre- and post-fixing a value with |
        
        Example:
        remote.download#GzipContentHandler     
        - GzipContentHandler class is imported from remote.download
        - Object is constructed with no arguments
        
        remote.download#Downloader?contentHandler=|remote.download#GzipContentHandler|
        - Downloader class is imported from remote.download
        - GzipContentHandler object is retrieved or created from the object-pool
        - Downloader objects is constructed with the GzipContentHandler object as the 'contentHandler' keyword
        """
        match = self.nameExp.match(obj)
        module = match.group('module').strip()[: - 1] # Remove the '#'
        argString = match.group('arguments')
        
        if len(module) == 0: module = self.defaultModule
        className = match.group('class').strip()
        arguments = {}
        for match in self.argsExp.finditer(argString):
            key = match.group('key').strip()
            value = match.group('value').strip()
            if value.startswith('|') and value.endswith('|'):
                arguments[key] = self.parseObject(value[1: - 1])
            else:
                arguments[key] = value
        
        qualifiedName = "{0}#{1}{2}".format(module, className, argString)
        return self.getObject(qualifiedName, module, className, arguments)
    
    def getModule(self, moduleName):
        """
        Retreives already imported modules by name. 
        Will import the module if it's not yet imported.
        """
        if not moduleName in self.modules:
            #fromlist can't be empty if I want the whole module returned and not just the package
            self.modules[moduleName] = __import__(moduleName.strip(), fromlist=['test']) 
        return self.modules[moduleName]
    
    def getObject(self, qualifiedName, moduleName, objectName, arguments):
        """
        Using the qualifiedName as an identifier for the created object,
        the object is fetched from the object-pool. It it's not yet created
        it is imported from the module and constructed with the given
        arguments.
        """
        if not qualifiedName in self.objects:
            m = self.getModule(moduleName)
            for name in dir(m):
                if name == objectName:
                    clazz = getattr(m, name)
                    self.objects[qualifiedName] = clazz(**arguments)
        if qualifiedName not in self.objects or self.objects[qualifiedName] == None:
            raise ConfigureError("{0} does not exists.".format(qualifiedName))
        return self.objects[qualifiedName]

if __name__ == '__main__':
    module = "remote.download"
    request = "UrlProcessor, remote.download#CrawlIdHandler?crawlId=1, RobotIdentitiferHandler, GzipContentHandler, orm.models#Constraint?name=|CrawlIdHandler?crawlId=1|&urlProcessor=|UrlProcessor|"
    
    configurer = ObjectConfigurer()
    req = configurer.configure(request, defaultModule=module)
    print(req)

Something like this: remote.download#CheckerRunner?checker=|EqualityChecker?threshold=5|+|AnotherChecker?threshold=6| Or are you looking for help with the code?

Ooo.. That might work :) I'll poke around in my code and see if I can get it to work. Thanks :)

Worked like a charm :D

Changed the regex to use negative lookbehind and lookahead to match the +, and a new regex to match the list of references.

Thanks for your input :)

I'll post the updated code in case anyone's interested.

Updated regex:

argsExp = re.compile(r"[?&](?P<key>[^=]*)=(?P<value>(?:[|].*?(?<![+])[|](?![+]))|(?:[^&]*))")
listExp = re.compile(r"[|](.*?)[|][+]?")

And changed parseObject to:

def parseObject(self, obj):
    match = self.nameExp.match(obj)
    module = match.group('module').strip()[: - 1] # Remove the '#'
    argString = match.group('arguments')
    
    if len(module) == 0: module = self.defaultModule
    className = match.group('class').strip()
    arguments = {}
    for match in self.argsExp.finditer(argString):
        key = match.group('key').strip()
        value = match.group('value').strip()
        matches = self.listExp.findall(value)
        if matches:
            arguments[key] = [self.parseObject(reference.strip()) for reference in matches if len(reference.strip()) > 0]
            continue
        if value.startswith('|') and value.endswith('|'):
            arguments[key] = self.parseObject(value[1: - 1])
        else:
            arguments[key] = value
    
    qualifiedName = "{0}#{1}{2}".format(module, className, argString)
    return self.getObject(qualifiedName, module, className, arguments)
This question has already been answered. Start a new discussion instead.