okay, it's been a while since I've asked any dumb simple questions you all know the answers to.
when a site is using https instead of http, can spiders still crawl that site?
My understanding is those pages load slightly slower and are just more secure. But I'm starting to suspect they cause all sorts of problems with spiders.
Thoughts?
Track
0
Newbie Poster
Recommended Answers
Jump to PostIn general https pages are not crawled or indexed by search engines. If you want a page to be crawled and indexed it is wise not encrypt its contents.
Jump to PostThe only pages that need to be secure are your order pages and any other page that captures and transmits sensitive data. Otherwise you are just spinning your wheels and making your server work hard encrypting pages that don't need to be encrypted.
All 6 Replies
Watt
0
Newbie Poster
stymiee
111
He's No Good To Me Dead
Team Colleague
Valhalla
0
Newbie Poster
stymiee
111
He's No Good To Me Dead
Team Colleague
Candy_ME
0
Junior Poster in Training
nytrokiss
0
Light Poster
DanceInstructor
commented:
It can be done via robots.txt, url rewriting, and a scripting language.
+0
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.