I'm stuck with a certain problem. I'm using URLLIB2 to get the end url of a list of links. This was pretty straightforward. Some of the links I'm probing pass through 1 or more other urls before landing the user at the end destination.

For example, the start url might be 1, but then you only end up at 3 after a quick and probably unnoticeable redirect to 2:

1. http://click.someurl.net/script?query=somedata
2. http://bridge.url.net/redirect?url=http://thenewurl.com
3. http://thenewurl.com

In my code url 1 is passed to my function, and url 3 is returned, but how do I capture/return any URL in between the start and end points?

Thanks in advance for any help!


Your post does not provide many details, but I think what you are looking for is a generator function., Instead of returning an URL, yield each URL, like this:

def generateURLs(startingURL):
    url = startingURL
    while url:
        yield url
        url = getNextURL(url)
for url in generateURLs(http://click.someurl.net/script?query=somedata):
    print url