I am SEO executive. But I don't know how to crate robots ? so please suggest me.
starwebmaker09
1
Newbie Poster
usman_24
commented:
robots.txt must be an UTF-8 encoded text file (which includes ASCII). Using other character sets is not possible. the robots exclusion protocol or sta
+0
10bestgloves
commented:
A robots. txt is a text file that resides in the root directory of your website and gives search engines crawlers instructions as to which pages they
+0
kliffcareerhub
commented:
A robots. txt file tells search engine crawlers which URLs the crawler can access on your site. This is used mainly to avoid overloading your site wit
+0
andro_3
commented:
spiders. programs that collect the data from your website
+0
Recommended Answers
Jump to PostRobots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website.
Google specifically follows instructions given in this robots.txt file
Jump to PostRobot.txt is a file which control search engine To visit website.We can made it easily. We should upload the robot.txt file in domain root folder.
Jump to PostYou're an SEO Executive... but you don't know what robots are?
Jump to PostHello everyone,
Can anyone tell me, May i use this robots file for sitemap like
User-agent: *
Disallow: /fade/
Sitemap:http://www.xxxxxxxxxxxxxxx.com/sitemap.xml
Jump to PostRobots.txt is a file through which you can guide search engines to crawl or not to crawl certain sections of your website.
All 53 Replies
kapildeven89
-1
Light Poster
cchstore
0
Newbie Poster
Ruby Spark
0
Newbie Poster
bhav123
0
Junior Poster in Training
brahmalreja
0
Newbie Poster
ozstylo
-3
Junior Poster in Training
Geek-harjeet
0
Newbie Poster
Shail164
0
Light Poster
arundlk
-3
Newbie Poster
Myronz
0
Light Poster
thomasbruce
0
Newbie Poster
steve787
0
Newbie Poster
arunmarvel5
0
Light Poster
Ravi Rajput
2
Junior Poster
bradly.spicer
47
Posting Whiz in Training
oliversmith
0
Newbie Poster
arunmarvel5
0
Light Poster
bhavdeeps
0
Light Poster
johnkennedi
0
Newbie Poster
stefandeny
0
Newbie Poster
denniscordell1
0
Newbie Poster
carmen100
0
Newbie Poster
rohit03catalyst
0
Newbie Poster
Raunikkapoor
-3
Newbie Poster
rproffitt
commented:
6
-3
shivya bhargava
-1
Google Certified
rproffitt
commented:
Look up. You are 6 years late and didn't answer how to "crate robots."
-3
j.khurram
0
Newbie Poster
usman_24
0
yescustomboxes
Dani
3,632
The Queen of DaniWeb
Administrator
Featured Poster
Premium Member
Antonio_7
0
Newbie Poster
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.