Web Spider Help

Please support our Python advertiser: Programming Forums - DaniWeb Sister Site
Reply

Join Date: May 2009
Posts: 28
Reputation: poeticinsanity is an unknown quantity at this point 
Solved Threads: 1
poeticinsanity poeticinsanity is offline Offline
Light Poster

Web Spider Help

 
0
  #1
May 28th, 2009
I am coding a web spider for research purposes and have run into an error I am uncertain about. I am fairly new to web programming and need a bit of guidance. I use http.client to get a connection, request a site, get the response, and read the resonse into a variable. Then, using HTMLparser, I attempt to read() the variable, but am given this error:

Traceback (most recent call last):
File "C:\Users\snorris4\Desktop\FLOSSmoleSpiderSavannah\src\SavannahSpider.py", line 45, in <module>
main()
File "C:\Users\snorris4\Desktop\FLOSSmoleSpiderSavannah\src\SavannahSpider.py", line 41, in main
spider.feed(page)
File "C:\Python31\lib\html\parser.py", line 107, in feed
self.rawdata = self.rawdata + data
TypeError: Can't convert 'bytes' object to str implicitly

Any help would be very much appreciated. Thank you.
Reply With Quote Quick reply to this message  
Join Date: Jun 2008
Posts: 128
Reputation: slate is an unknown quantity at this point 
Solved Threads: 31
slate slate is offline Offline
Junior Poster

Re: Web Spider Help

 
0
  #2
May 29th, 2009
A classic error in python3. You are concatenating a byte array (rawdata) to a string(data).

Try converting the data to bytes with encoding or do not convert it to string.

Try googling the error code to get the notion.
Reply With Quote Quick reply to this message  
Join Date: May 2009
Posts: 28
Reputation: poeticinsanity is an unknown quantity at this point 
Solved Threads: 1
poeticinsanity poeticinsanity is offline Offline
Light Poster

Re: Web Spider Help

 
0
  #3
May 29th, 2009
Thank you for the help!
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:



Similar Threads
Other Threads in the Python Forum


Views: 361 | Replies: 2
Thread Tools Search this Thread



Tag cloud for Python
About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC