943,752 Members | Top Members by Rank

Ad:
  • Python Discussion Thread
  • Unsolved
  • Views: 497
  • Python RSS
May 28th, 2009
0

Web Spider Help

Expand Post »
I am coding a web spider for research purposes and have run into an error I am uncertain about. I am fairly new to web programming and need a bit of guidance. I use http.client to get a connection, request a site, get the response, and read the resonse into a variable. Then, using HTMLparser, I attempt to read() the variable, but am given this error:

Traceback (most recent call last):
File "C:\Users\snorris4\Desktop\FLOSSmoleSpiderSavannah\src\SavannahSpider.py", line 45, in <module>
main()
File "C:\Users\snorris4\Desktop\FLOSSmoleSpiderSavannah\src\SavannahSpider.py", line 41, in main
spider.feed(page)
File "C:\Python31\lib\html\parser.py", line 107, in feed
self.rawdata = self.rawdata + data
TypeError: Can't convert 'bytes' object to str implicitly

Any help would be very much appreciated. Thank you.
Similar Threads
Reputation Points: 12
Solved Threads: 1
Light Poster
poeticinsanity is offline Offline
28 posts
since May 2009
May 29th, 2009
0

Re: Web Spider Help

A classic error in python3. You are concatenating a byte array (rawdata) to a string(data).

Try converting the data to bytes with encoding or do not convert it to string.

Try googling the error code to get the notion.
Reputation Points: 56
Solved Threads: 65
Posting Whiz in Training
slate is offline Offline
242 posts
since Jun 2008
May 29th, 2009
0

Re: Web Spider Help

Thank you for the help!
Reputation Points: 12
Solved Threads: 1
Light Poster
poeticinsanity is offline Offline
28 posts
since May 2009

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Python Forum Timeline: Socket programming with a service
Next Thread in Python Forum Timeline: Newbie With Function Trouble





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC