Text slice and split made easy.

By Afonso Salgado on Aug 31st, 2010 10:50 pm

Two litle functions to help on text slice and spliting.

The code comments say it all.

# Slicer takes as arguments a tuple containing the string before,
# the string after and the string to truncate. It returns the string
# between the two given strings
Slicer = lambda((b, a, t)): t.partition(b)[2].partition(a)[0]

# Spliter takes as arguments a tuple containing the string before,
# the string after and the string to truncate. It returns a tuple
# containing the string before, the string between and the string
# after the given strings
Spliter = lambda((b, a, t)): ((t.partition(b)[0]),) + t.partition(b)[2].partition(a)[0::2]

Cheers and Happy coding

import urllib

sock = urllib.urlopen('http://www.columbia.edu/')
htmlSource = sock.read()
sock.close()

# Slicer takes as arguments a tuple containing the string before,
# the string after and the string to truncate. It returns the string
# between the two given strings
Slicer = lambda((b, a, t)): t.partition(b)[2].partition(a)[0]

tag_before = '<!-- BEGIN COLUMBIA NEWS -->'
tag_after = '<!-- END COLUMBIA NEWS -->'

newsSource = Slicer((tag_before, tag_after, htmlSource)).split('\r')

stripwhite = lambda x: x.strip()

newsSource = [stripwhite(item.strip('\n')) for item in newsSource if stripwhite(item.strip('\n')) and stripwhite(item.strip('\n')) != '<br /><br />']

# Spliter takes as arguments a tuple containing the string before,
# the string after and the string to truncate. It returns a tuple
# containing the string before, the string between and the string
# after the given strings
Spliter = lambda((b, a, t)): ((t.partition(b)[0]),) + t.partition(b)[2].partition(a)[0::2]

tag_before = '<a href='
tag_after = '>'

for line in newsSource:
    news_title, link, rest = Spliter((tag_before, tag_after, line))
    print 'News:', news_title
    print 'Link:', link

1 Year Ago

I did little clean up for your code, looks like you have compressed my between function . The lambdas looked little out of place and I changed them to normal defs to be more understandable for people without Lisp or similar experience, hope I did not break anything:

import urllib

sock = urllib.urlopen('http://www.columbia.edu/')
htmlSource = sock.read()
sock.close()

# Slicer takes as arguments a tuple containing the string before,
# the string after and the string to truncate. It returns the string
# between the two given strings
def slicer((before, after, text)):
    return (text.partition(before)[2].partition(after)[0])

# Spliter takes as arguments a tuple containing the string before,
# the string after and the string to truncate. It returns a tuple
# containing the string before, the string between and the string
# after the given strings
def spliter((b, a, t)):
    return ((t.partition(b)[0]),) + t.partition(b)[2].partition(a)[0::2]

def stripwhite(x):
    return x.strip()

tag_before = '<!-- BEGIN COLUMBIA NEWS -->'
tag_after = '<!-- END COLUMBIA NEWS -->'

newsSource = slicer((tag_before, tag_after, htmlSource)).split('\r')
newsSource = [stripwhite(item)
              for item in newsSource
              if '' != stripwhite(item) != '' ]

tag_before = '<a href='
tag_after = '>'

for line in newsSource:
    news_title, link, rest = spliter((tag_before, tag_after, line))
    print 'News:', news_title
    print 'Link:', link

pyTony

pyMod

Moderator

5,359 posts since Apr 2010

Reputation Points: 782

Solved Threads: 852

1 Year Ago

Yes, it follows the same principle of multi partition calls to slice and split an string.

I always use similar methods, and decided to make one liners of my current versions for easying the things out.

Cheers and Happy coding

Beat_Slayer

Posting Pro in Training

405 posts since Jun 2010

Reputation Points: 30

Solved Threads: 105

This article has been dead for over three months

You