954,541 Members — Technology Publication meets Social Media
Username:
Password:
Lost login information?
Have something to say? Contribute New Article Reply to this Article

Web Spider Help

I am coding a web spider for research purposes and have run into an error I am uncertain about. I am fairly new to web programming and need a bit of guidance. I use http.client to get a connection, request a site, get the response, and read the resonse into a variable. Then, using HTMLparser, I attempt to read() the variable, but am given this error:

Traceback (most recent call last):
File "C:\Users\snorris4\Desktop\FLOSSmoleSpiderSavannah\src\SavannahSpider.py", line 45, in
main()
File "C:\Users\snorris4\Desktop\FLOSSmoleSpiderSavannah\src\SavannahSpider.py", line 41, in main
spider.feed(page)
File "C:\Python31\lib\html\parser.py", line 107, in feed
self.rawdata = self.rawdata + data
TypeError: Can't convert 'bytes' object to str implicitly

Any help would be very much appreciated. Thank you.

poeticinsanity
Light Poster
28 posts since May 2009
Reputation Points: 12
Solved Threads: 1
 

A classic error in python3. You are concatenating a byte array (rawdata) to a string(data).

Try converting the data to bytes with encoding or do not convert it to string.

Try googling the error code to get the notion.

slate
Posting Whiz in Training
252 posts since Jun 2008
Reputation Points: 72
Solved Threads: 66
 

Thank you for the help!

poeticinsanity
Light Poster
28 posts since May 2009
Reputation Points: 12
Solved Threads: 1
 

This article has been dead for over three months

Post: Markdown Syntax: Formatting Help
You