Please support our Search Engine Optimization advertiser: Affiliate Marketing
Views: 963 | Replies: 4
![]() |
•
•
Join Date: May 2006
Location: New Jersey
Posts: 1,422
Reputation:
Rep Power: 5
Solved Threads: 34
It depends on your site. Sites that are crawled more frequently will see results faster then sites crawled less frequently. If you want pages removed from the SERPs quickly robots.txt is not your best option. Use a removal request instead.
John Conde
Brainyminds | Merchant Account Services | I Love Code
IT'S HERE: Merchant Accounts 101 Everything you need to know about merchant accounts!
Brainyminds | Merchant Account Services | I Love Code
IT'S HERE: Merchant Accounts 101 Everything you need to know about merchant accounts!
•
•
Join Date: Jun 2006
Posts: 47
Reputation:
Rep Power: 0
Solved Threads: 3
when the seach engine crawls your site the first file it hits is robots.txt and if it sees to disallow any files it won't index them which means they should go off there index!
when the bot crawls thats the time when it'll see what is allowed and whats not allowed and make sure u use opening slash / and closing slash /
/images/
something like that when not allowing the bot to crawl with the right syntax =)
/images/
something like that when not allowing the bot to crawl with the right syntax =)
•
•
Join Date: Mar 2007
Posts: 65
Reputation:
Rep Power: 2
Solved Threads: 0
Yes, I agree with webmasterran. I too use robot,txt for my sites to say the search engines not to crawl my page.
![]() |
•
•
•
•
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)






Linear Mode