Hi All -

Forgive this naieve questions but I am not sure of the answer and know someone can help on Daniweb!

If I choose to ENCRYPT my HTML pages (for whole page) will the encryption scrample the text that the robots or crawlers will find?

That is - obviously encryption takes a whole language (text wise) on it's own and I want to be sure that I won't shy away the bots because there is no text visible in the coding.....

Hopefully this makes sense!!

Does anyoe have any experience with this or have any thoughts?

Thanks SO much!

-John

Recommended Answers

All 4 Replies

The important thing is how (and even more importantly when) you decrypt your page. If your page content is decrypted using javascript or other client-side means; then, because most search engines don't process client side scripts; your page wil not be indexed with relevant content.

If you decrypt during a server side operation, or use an encrypted transfer protocol (like SSL); and basically providing your page ends up as decrypted source being delivered as a result of a request; then search spiders won't know the difference.

If you're encrypting page content for any reason other than 'hiding the source code', then why do you want the pages to be indexed in search engines?

The important thing is how (and even more importantly when) you decrypt your page. If your page content is decrypted using javascript or other client-side means; then, because most search engines don't process client side scripts; your page wil not be indexed with relevant content.

If you decrypt during a server side operation, or use an encrypted transfer protocol (like SSL); and basically providing your page ends up as decrypted source being delivered as a result of a request; then search spiders won't know the difference.

If you're encrypting page content for any reason other than 'hiding the source code', then why do you want the pages to be indexed in search engines?

It's actually because I am building a site that will be used for download of a musicians music... so I just wish to hide the links for downloads, etc...

Though I am using paypal for the actual links for the download and will encrypt those... I am using other links to PLAY the music online and do not wish people to be able to see those links...

Make sense?

Sort of; but if you don't wish people to see the links, you probably should not put them on the Internet... If you want to restrict certain people from accessing the links; encrypting page content isn't the best way to go about it.

Do you want spiders to not see the links; or people to not to see the links?

Search spiders are like very fast people on very low-end browsers capable of following every link on your site, probably simultaneuosly. If you don't link to things; spiders can't see them. If you make your links difficult for a person to follow; spiders will find them difficult to follow. There's ways to stop spiders following links; and it's alot easier to stop (amicable) spiders from following links than it is to stop people following them.

Providing that enough of your page can be interpretted by a person on a browser without client-side-scripts; (even better, if it can be interpretted by looking at the source of the page as it is from the 'View Source' button); and providing that content is considered acceptable to the search engine's rules; then your pages will be indexed in some way.

If your page when viewed from the 'View Source' button is an unintelligable mix of encrypted HTML code; search spiders wouldn't understand it.

To answer your question (very generally); if you (effectively) stop users from seeing something, you stop spiders from seeing it to. If you stop spiders from seeing something (using many available means) you don't neccessarily have to stop users from seeing it. If you want to stop both; well that's easy.

You are forgetting that, once a user finds a web page, he can find any other file in the same folder using simple methods. He can also find subfolders.

I have a page which is not linked to from anywhere else, except that my browser starts with it (to save download MBs for real users). Yet I found it indexed on a search engine last week.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.