Menu
Menu
DaniWeb
Log In
Sign Up
Read
Contribute
Meet
Search
Search
About 481 results for
googlebot
- Page 1
Re: Do backlinks to disallowed URLs help the domain overall?
Digital Media
Digital Marketing
Search Engine Strategies
1 Month Ago
by Dani
… matter. If a URL is disallowed via robots.txt, then
Googlebot
never crawls the page to discover what is or is…
Googlebot crawling AMP pages
Digital Media
Digital Marketing
Search Engine Strategies
3 Years Ago
by Dani
Googlebot
is crawling my AMP pages more than they are crawling …
Googlebot??
Digital Media
Digital Marketing
Search Engine Strategies
17 Years Ago
by iTaChi
i checked my site's statistics..and i viewed the bots who visited my site.. then there was
googlebot
..40 hits..last may 9...but when i checked my index status in google..they said that my site is not yet indexed... the problem is..since my site was crawled by
googlebot
...why is it not yet indexed by google??? thank you...
Googlebot: Valid meta robots content values
Digital Media
Digital Marketing
Search Engine Strategies
17 Years Ago
by Lampard33
Valid meta robots content values
Googlebot
interprets the following robots meta tag values: * …from being included in the index. * NOFOLLOW - prevents
Googlebot
from following any links on the page. (Note that… different from the link-level NOFOLLOW attribute, which prevents
Googlebot
from following an individual link.) * NOARCHIVE - prevents …
Googlebot gets to grip with Flash
Digital Media
Digital Marketing
Search Engine Strategies
15 Years Ago
by happygeek
… Flash indexing is heading for Google. In fact, if the
Googlebot
engineers posting to that blog are to be believed, the…. There are some other limitations, according to the
Googlebot
engineers, namely: [LIST=1] [*]
Googlebot
does not execute some types of JavaScript. So…
Googlebot ignores robots.txt
Digital Media
Digital Marketing
Search Engine Strategies
3 Years Ago
by Dani
… not respecting my robots.txt. I'm seeing
Googlebot
's user agent crawling pages that have been in my … dates indicated as recent as yesterday. Additionally, I'm seeing
Googlebot
crawl my robots.txt file a few times a day…
Re: Googlebot ignores robots.txt
Digital Media
Digital Marketing
Search Engine Strategies
3 Years Ago
by rproffitt
I'm hearing more about fake
googlebot
requests. Maybe that? PS. Google those 3 words to find out more. Also: https://support.google.com/webmasters/answer/80553
Re: Googlebot ignores robots.txt
Digital Media
Digital Marketing
Search Engine Strategies
3 Years Ago
by Dani
Per the OP post: > Additionally, I'm seeing
Googlebot
crawl my robots.txt file a few times a day, **and the URLs are definitely blocked per the Google robots.txt tester**.
Re: Googlebot ignores robots.txt
Digital Media
Digital Marketing
Search Engine Strategies
3 Years Ago
by rproffitt
Time for Google's own to give up ideas. Given the Google Seach Console (GSC) gives it a passing grade tells me that that likelihood of fake
googlebot
requests just went up even if the useragent is legitimate. Sorry for not defining GSC first. I'll work harder on that.
Re: Googlebot ignores robots.txt
Digital Media
Digital Marketing
Search Engine Strategies
3 Years Ago
by Dani
… rather large, and blocks a significant chunk of our pages,
Googlebot
may think that we made a mistake and didn't…
Googlebot found an extremely high number of URLs on your site
Digital Media
Digital Marketing
Search Engine Strategies
12 Years Ago
by Dani
… a message in my Google Webmaster Tools saying that "
Googlebot
found an extremely high number of URLs on your site…
Re: Googlebot found an extremely high number of URLs on your site
Digital Media
Digital Marketing
Search Engine Strategies
12 Years Ago
by zabalex
… for the pages or directories as per the requirement for
googlebot
to follow the instructions. regards zabalex
Googlebot uses cookies
Digital Media
Digital Marketing
Search Engine Strategies
2 Years Ago
by Dani
Google has always said that all of their bots do not use cookies. I caught a screenshot on Google PageSpeed Insights that displayed a modal that I only have implemented for users with a specific cookie already set. So there you have it,
Googlebot
uses cookies.
Re: Googlebot uses cookies
Digital Media
Digital Marketing
Search Engine Strategies
2 Years Ago
by AussieWebmaster
From 2014 https://www.tomanthony.co.uk/blog/
googlebot
-accepting-cookies/
Re: Googlebot help.
Digital Media
Digital Marketing
Search Engine Strategies
14 Years Ago
by Chuck's
I have something new to me
Googlebot
-o, it has consumed a day yesterday and is still there today! Will it leave my stat counter soon? Thnak You:) Chuck
Re: Can you tell me something about googlebot?
Digital Media
Digital Marketing
Search Engine Strategies
7 Years Ago
by Nicoli
Googlebot
is the name of google's search engine crawler/spider. It crawls to find new and updated webpage to index to Google.
Re: Googlebot??
Digital Media
Digital Marketing
Search Engine Strategies
17 Years Ago
by uggur
Google has fresh bot and deep bot. Fresh bot coming to your site everyday. (Its change pagerank or other google criteria) Deep bot is coming to your site every month and its indexing your site to google database so you should wait a period.
Re: Googlebot??
Digital Media
Digital Marketing
Search Engine Strategies
16 Years Ago
by cooluks
[QUOTE=uggur;365963]Google has fresh bot and deep bot. Fresh bot coming to your site everyday. (Its change pagerank or other google criteria) Deep bot is coming to your site every month and its indexing your site to google database so you should wait a period.[/QUOTE] Excellent explanation.. Thanks for this informative post...
Re: Googlebot??
Digital Media
Digital Marketing
Search Engine Strategies
16 Years Ago
by x3mario
Thanks for the info. It's a great explanation.
Re: Googlebot crawling AMP pages
Digital Media
Digital Marketing
Search Engine Strategies
3 Years Ago
by daxclark52
Best and beneficial information. Thank you very much to share this information. I was very worry about amp. Thanks again to solve my problem
Re: Need mod_security rules to prevent Googlebot from crawling one file
Hardware and Software
Linux and Unix
12 Years Ago
by vectro
… rule for the main Apache configuration which will keep
Googlebot
off of 1 particular file. This only applies if…;/file.php"> SecRule REQUEST_HEADERS:User-Agent "@pm
Googlebot
" "deny,status:403" </LocationMatch>… you want to protect. The part that says "
Googlebot
" can also be changed to any user-agent.…
How to get Googlebot stay longer on my website.
Digital Media
Digital Marketing
Search Engine Strategies
16 Years Ago
by devesh9392
… to check the google crawler frequency, i found that the
googlebot
visits store.xyz.com every 3-4 days, but only… as well as by crawler. can anybody help me knowing
googlebot
's behaviour. thnx.
Need mod_security rules to prevent Googlebot from crawling one file
Hardware and Software
Linux and Unix
12 Years Ago
by vectro
I need mod_security rules to prevent
Googlebot
from indexing any file named browse.php anywhere on the server, while still allowing
Googlebot
to access anything else. I figured mod_security will do the trick because it can recognize user-agents and set rules accordingly. Any ideas?
Re: Can you tell me something about googlebot?
Digital Media
Digital Marketing
Search Engine Strategies
7 Years Ago
by Cristy_1
Hello,
Googlebot
is the search bot software used by Google to index a webpage. Crawling, Caching and indexing of a webpage happen through this
Googlebot
by collecting details from that webpage.
Re: It has been over 45 days since googlebot visited my site, what shoudl I do?
Digital Media
Digital Marketing
Search Engine Strategies
15 Years Ago
by abhicho
i have a problem.
googlebot
visited my home page on the 18th of this month. but has not indexed a single page. not even the home page. can you tell me how will it be indexed. thanks in advance
Does googlebot like chat logs?
Digital Media
Digital Marketing
Search Engine Strategies
10 Years Ago
by Dani
… from a live chat room. What are the chances that
Googlebot
would eat our chat archive right up?! I think that…
Can you tell me something about googlebot?
Digital Media
Digital Marketing
Search Engine Strategies
7 Years Ago
by life assosication
Can you tell me something about
googlebot
?
Re: Googlebot: Valid meta robots content values
Digital Media
Digital Marketing
Search Engine Strategies
17 Years Ago
by Dexter515
Don't think I use any of them..
Re: Googlebot: Valid meta robots content values
Digital Media
Digital Marketing
Search Engine Strategies
17 Years Ago
by stymiee
Thank you for copying what Google has in the blog and posting it here.
Re: Googlebot gets to grip with Flash
Digital Media
Digital Marketing
Search Engine Strategies
15 Years Ago
by kelly johnson
Yeh thats great news......! Actually my site [url]http://www.emailappenders.com[/url] ..here in the home page i have used lots of flash files....i guess that was one of the Fact on my site emailappenders ranking...Now Google can read Flash files .. i have a hope, like this is defenately going to help my sites ranking in the search engines....!
1
2
3
9
Next
Last
Search
Search
Forum Categories
Hardware/Software
Recommended Topics
Programming
Recommended Topics
Digital Media
Recommended Topics
Community Center
Recommended Topics
Latest Content
Newest Topics
Latest Topics
Latest Posts
Latest Comments
Top Tags
Topics Feed
Social
Forums
Top Members
Meet People
Community Functions
DaniWeb Premium
Newsletter Archive
Markdown Syntax
Community Rules
Developer APIs
Connect API
Forum API Docs
Tools
SEO Backlink Checker
Legal
Terms of Service
Privacy Policy
FAQ
About Us
Advertise
Contact Us
© 2024 DaniWeb® LLC