Member Avatar for JenniferNavi

I am building out an e-commerce site Billy.com but I am having difficulty getting all of my pages indexed. I throw lots of links at it and use sitemaps but it's taking forever!! I'm really perplexed, please help.

Check out our movers page, I think we are well formatted but nothing, not full indexation like we would like!

Edited by JenniferNavi because: n/a
  • 4 Contributors
  • 3 Replies
  • 183 Views
  • 3 Months Discussion Span
  • Latest Post Latest Post by Andy Mathew

Dani AI

Generated

Quick checklist and where to start: since you already have sitemaps and external links, the fastest way to learn "why" pages are excluded is Search Console + server logs. Use the Coverage/Indexing report and URL Inspection to see the specific reason (noindex, canonicalized, redirect, 4xx/5xx, blocked by robots.txt, or simply "discovered — currently not indexed"). Check raw server logs for Googlebot activity and HTTP status codes — that will show crawl failures and redirect chains that an index report alone can miss. ()

Common technical causes to rule out next:

  • robots.txt or x-robots-tag accidentally blocking categories or parameter folders.
  • meta robots set to noindex on templates (cart, filters, or old staging pages).
  • canonical tags pointing to a different URL or to homepage.
  • parameterized/faceted URLs creating duplicate or low-value pages (filters, sort, session IDs).
  • large redirect chains, soft 404s, or frequent 5xx errors.
    Moving inline CSS/JS to external files (good tip from ) helps page weight and render time, but won’t fix pages that are explicitly blocked or canonicalized away.

Mini examples to verify quick fixes:

# robots.txt (example)
User-agent: *
Disallow: /checkout/
Allow: /

<link rel="canonical" href="https://www.example.com/product/sku123" />
<meta name="robots" content="index,follow">

E-commerce-specific priorities: remove thin/duplicate product copy — give each product a unique description and at least some user-generated content (reviews). Noindex internal search results, faceted combinations, and any tag pages that add no unique value. Put only canonical, final product/category URLs in your XML sitemap(s); split/compress sitemaps if the site is large. Be cautious about link schemes: the quick PR5 backlink idea from can give short-term visibility but risks penalties if it’s low-quality or manipulative. Fix technical blockers first, then focus on natural linking and content quality.

Next steps: run a full crawl with a site crawler, check Search Console Coverage for the most common exclusion reasons, fix a representative page, then request reindexing. Document changes and re-check logs for Googlebot behavior — repeat until the excluded-category count falls.

Recommended Answers

All 3 Replies

Member Avatar for xtremeux

I am building out an e-commerce site Billy.com but I am having difficulty getting all of my pages indexed. I throw lots of links at it and use sitemaps but it's taking forever!! I'm really perplexed, please help.

Check out our movers page, I think we are well formatted but nothing, not full indexation like we would like!

As on date 04-Dec-10, 900+ pages are indexed in Google. Sitemap.xml is also available and assume that your sitemap is submitted in search engines.

Not full indexation...
I reviewed your example page "mover" and noticed that page has lot of in-line CSS validations and Java scripting. Use common CSS file and replace with in-line CSS validations.
Create external Java Include files (.js file), call them externally and replace with current in-line Java scripting.

Edited by xtremeux because: n/a
Member Avatar for harris123

what is the best way to index your page in google?

Member Avatar for Andy Mathew

If you can add your site in any PR5+ theme related site (Do Follow Link) then Google will index your site with in 48 hours. But in that case your site should not have any broken link and should be running properly. You can also submit your site into webmaster tools and after adding a Sitemap Google will index your site.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.