Python

Python RSS

HELP! Return URLs in redirect chain

•

Join Date: Nov 2009

Posts: 1

Reputation: LB22 is an unknown quantity at this point

Solved Threads: 0

LB22

Offline

Newbie Poster

HELP! Return URLs in redirect chain

23 Days Ago

Hi,

I'm stuck with a certain problem. I'm using URLLIB2 to get the end url of a list of links. This was pretty straightforward. Some of the links I'm probing pass through 1 or more other urls before landing the user at the end destination.

For example, the start url might be 1, but then you only end up at 3 after a quick and probably unnoticeable redirect to 2:

 Help with Code Tags 
 Python Syntax (Toggle Plain Text) 
http://click.someurl.net/script?query=somedata
http://bridge.url.net/redirect?url=http://thenewurl.com
http://thenewurl.com
http://click.someurl.net/script?query=somedata
http://bridge.url.net/redirect?url=http://thenewurl.com
http://thenewurl.com

In my code url 1 is passed to my function, and url 3 is returned, but how do I capture/return any URL in between the start and end points?

Thanks in advance for any help!

LB

•

Join Date: Nov 2009

Posts: 79

Reputation: pythopian is an unknown quantity at this point

Solved Threads: 21

pythopian

Offline

Junior Poster in Training

Use a generator function/method

17 Days Ago

Your post does not provide many details, but I think what you are looking for is a generator function., Instead of returning an URL, yield each URL, like this:

 Help with Code Tags 
 Python Syntax (Toggle Plain Text) 
def generateURLs(startingURL):
    url = startingURL
    ...
    while url:
        yield url
        url = getNextURL(url)
...
for url in generateURLs(http://click.someurl.net/script?query=somedata):
    print url
def generateURLs(startingURL):
    url = startingURL
    ...
    while url:
        yield url
        url = getNextURL(url)
...
for url in generateURLs(http://click.someurl.net/script?query=somedata):
    print url

"I have come here to chew bubblegum and kick ass... and I'm all out of bubblegum."
------------
This posts may be redistributed under the Creative Commons BY-SA License.