13 Discussion / Question Topics
Remove Filter
|
|
|
The new robots.txt tester that is now built into Google Search Console is finding errors with the last two rows of my robots.txt file: Sitemap: https://www.daniweb.com/latest-sitemap.xml Sitemap: https://www.daniweb.com/sitemap.xml The error is: *Invalid sitemap URL detected; syntax not understood* Any idea what could be wrong here?
|
|
|
I am trying to remove an entire folder of thin content from Google to help me recover from a Panda/EAT-related penalty. I want to keep the content on the site for the benefit of users, but not waste crawl budget or have Google think that we have so many pages … 
|
|
|
Most Google searches for common words usually result in a high number of hits. As an example, I just typed in "love", and Google returned, "About 11,840,000,000 results (0.77 seconds)". My bet is most end users see probably the first few pages at the most. That is, they would see …
|
|
|
I'm noticing Googlebot is not respecting my robots.txt. I'm seeing Googlebot's user agent crawling pages that have been in my robots.txt file for many months. Some of them are showing up in GSC as "Indexed, though blocked by robots.txt" with Last crawled dates indicated as recent as yesterday. Additionally, I'm …
|
|
I want to disallow certain query strings with robots.txt and I want to check I am doing this correctly... I am using: Disallow: /browse.asp?cat=*-* I want to check that this rule will allow these urls to be indexed: *browse.asp?cat=123/1234/1234-1 browse.asp?cat=123/1234-1* While disallowing these urls: *browse.asp?cat=1234-1 browse.asp?cat=1234-2* Will this rule work? …
|
|
|
Hifriends! I've just started working on SEO for my sites, but I have problems in my first attempt. I signed int to Google Webmaster tools...added a site and verified it successfully. My site even started appearing on Google Search and within a very short time, it was on the first …
|
|
|
Hi, I am going to submit a new website in Google (deveoped in PHP) today. I have 'include' folder which has header.php, footer.php and navigation.php files. I have included them in each web page of my site using <?php include ('./includes/header.php'); ?>. I do not want these php files to …
|
|
|
We have the valid URLs: www.daniweb.com/foo www.daniweb.com/foo/ www.daniweb.com/foo/1 www.daniweb.com/foo/2 www.daniweb.com/foo/3 If I want to disallow them all in robots.txt, are *both* of these valid and will they do the same thing? Disallow: /foo Disallow: /foo/ Will the latter also block the URL www.daniweb.com/foo or will that be interpreted as a … 
|
|
My URL is: http://www.mildaspergers.com/sitemap.txt and for robots: http://www.mildaspergers.com/robots.txt Hello, When I started, I only submitted one site to Google (which went fine). Then I tried adding more pages, and it would not go through. I have added a phpbb script, tried editing it, to no avail. I tired messing around …
|
|
|
I would like to know that which things of website, an expert SEO must put into robots.txt file for disallow? Which pages is better to not show to Search Engines?
|
|
|
I have created a sub domain and put robots file As follows User-agent: * DisAllow: / Will it affect my main domain? I have robots file in my main domain as follows User-agent: * Allow: /
|
|
|
I have page1.html that is being 302 redirected (temporary redirect) to page2.html page2.html is disallowed in my robots.txt file Under normal circumstances, when googlebot encounters a 301 redirect from page1.html to page2.html, it will index page2.html, and when googlebot encounters a 302 redirect from page1.html to page2.html, it will index …
|
|
|
I want to report deadlinks on my site, my current script does work but it allows search engine bots to click the dead link report which makes it hard for me to determine which reports are form visitors and which are from bots. What am I doing wrong here? As …
|
The End.