hi, i am newbie. i have question how make search engine will crawl/spider www.host.com/file.php for www.host.com

Thank You

Recommended Answers

All 6 Replies

If the file can be navigated to, then it will most likely get indexed. To point a robot to the file, you can always use a sitemap.

If the file can be navigated to, then it will most likely get indexed. To point a robot to the file, you can always use a sitemap.

before thank you, but i mean not for indexed, exmp: if i check metatag , which the data output for http://host.com from http://host.com/otherpage.php

I do not understand the question.

sorry my english so bad. let me explain.

i was checked http://www.rumahraya.com/ website through http://www.seocentro.com/tools/search-engines/metatag-analyzer.html but result is different for metatag also the content.

if i check the metatag http://www.rumahraya.com/
<meta name="ROBOTS" CONTENT="NOINDEX, NOFOLLOW" />

but the results check metatag

<meta name="ROBOTS" CONTENT="INDEX,FOLLOW" />
also for all the content.

thank you

hello buddy i don't understanding your question.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.