Python

Python RSS

Python and Unicode

Please support our Python advertiser: Programming Forums - DaniWeb Sister Site

•

Join Date: Oct 2008

Posts: 1

Reputation: suresh_iyengar is an unknown quantity at this point

Solved Threads: 0

suresh_iyengar suresh_iyengar is offline

Offline

Newbie Poster

Python and Unicode

Oct 27th, 2008

Hello,

I want to fetch a web page and parse links in that. I am using the foll. code

 Help with Code Tags 
 python Syntax (Toggle Plain Text) 
file =urllib.urlopen("file:///home/suresh/html_parser/Category:Sports.html")
content = file.read()
# Process the page.
file =urllib.urlopen("file:///home/suresh/html_parser/Category:Sports.html")
content = file.read()
# Process the page.

But since the page contains UTF-8 content, its not able to parse it properly. But, if I save the page locally, then its able to parse. How to handle this problem,

•

Join Date: Oct 2008

Posts: 45

Reputation: tyincali is an unknown quantity at this point

Solved Threads: 6

tyincali

Offline

Light Poster

Re: Python and Unicode

Oct 28th, 2008

Are you getting an error when you try to parse it?

what about:

 Help with Code Tags 
 Python Syntax (Toggle Plain Text) 
content = file.read().encode('ascii', 'ignore')
content = file.read().encode('ascii', 'ignore')