IP based re-direct filter confusing GoogleBot

[Chris_TDR]

Our website is split into 13 host countries, each with language data and (eventually) individual prices on products, to stop people viewing other country's prices we use the ip2nation database to determine the user's IP location (or proxy location) then a server-side filter redirects them to the relevant homepage 'shop' for their country & only that country's data is shown to the client.
Our problem is that this filter is confusing the Google Spider Bot and our page rankings have dropped dramatically.
We need this functionality and cannot think of a way to achieve it without this script & filter. I imagine other websites use it or variations of it without losing Google rank and I wonder how they do it or if there are other ways I don't know about?

[sknake]

You're probably better off asking questions like this in the SEO forums. My *personal* opinion is that you should not alter the site based on the client IP or referring URL as these will likely get you penalized, as you have seen.

I'm guessing you don't want to purchase a domain for each respective country? This is the recommended route as google can best index the content this way for each language. However if you have not already purchased the domains then they are probably taken plus domain age is a big factor in SEO. Having a TLD extension that jives with the "META HTTP-EQUIV=Content-Language CONTENT" will help.

Going back to your original question you are probably better off in letting a user select the language but making all languages available. Some people physically located in the USA are not natively english speaking and may prefer other languages. Likewise this will give the google bot access to all of your content which should help your rankings.

[edit]I missed the comment about prices on the first go round. I see why this is more important now.
Can't you make all language versions of the site available but substitute the prices based on the ip2location database? ie show american USD prices for all languages if the IP is determined to be an american IP.
[/edit]

[Chris_TDR]

Thanks sknake, i'm new to forums wasn't sure where to post.

In an ideal world all our EU distributors would have an in-line pricing structure, but they dont so I need to restrict a French user viewing a German price and so on.

I guess I need to know what it is about the code we have that confuses the Google Spider Bot, but then Google doesnt let on much about the way ot works!

Here is the .vb code for the redirect - it contains 5 exceptions where the user is directed to another website, this is temporary:
<code>
Sub IPLookupRedirect()

If Session("DoneIpRedirect") IsNot Nothing AndAlso Session("DoneIpRedirect") Then
Exit Sub
End If

Dim SQLStmt As String = "SELECT TOP 1 Ip2nations.Ip2nationCountryCode " & _
"FROM Ip2nations " & _
"WHERE Ip2nations.Ip2nationIp < dbo.rtf_INET_ATON('" & URL.ServerVariables.REMOTE_ADDR & "') " & _
"ORDER BY Ip2nations.Ip2nationIp DESC;"

'Perform redirect
Dim CountryCode As New xString

CountryCode.Value = DBAccess.ExecuteGetData(New DBStatusClass, SQLStmt, "Ip2nationCountryCode", AccessLevel.Standard)

'Dg.V(SQLStmt)
'Dg.V(CountryCode.Value)

Select Case CountryCode.Text.ToLower

Case "ie"
PageRedirect("http://www.totaldiscrepair.ie")
Exit Sub

Case "de"
PageRedirect("http://www.cdr-online.de")
Exit Sub

Case "cz"
PageRedirect("http://www.tdr.cz")
Exit Sub

Case "fi"
PageRedirect("http://www.pv-supa.fi/pages/index.php?id=16&pid=1")
Exit Sub

Case "gb"
Session("DoneIpRedirect") = True
Exit Sub

End Select

SQLStmt = "SELECT TOP 1 DistributorCountries.DistributorCountryShopCountryId " & _
"FROM DistributorCountries INNER JOIN ShopCountries ON DistributorCountries.DistributorCountryShopCountryId = ShopCountries.ShopCountryId " & _
"WHERE ShopCountryIsoCode2 = " & CountryCode.Sql & ";"

Dim CountryId As New xInt32

CountryId.Value = DBAccess.ExecuteGetData(New DBStatusClass, SQLStmt, "DistributorCountryShopCountryId", AccessLevel.Standard)

'Dg.V(CountryId.Value)

Session("DoneIpRedirect") = True

PageRedirect("/PMContent/home.aspx?fdCountryId=" & CountryId.Url)

End Sub
</code>

[sknake]

One more daniweb lesson :) Please use code tags when posting code:

[code=vb.net] ...code here...

[/code]

Go ahead and hit "Edit Post" on your last post and you should be able to fix it.

For Google -- They don't want you to know how it works so you can't circumvent the logic.

Take this scenario:
I have a site jam-packed with keywords that looks horrible but it has a lot of keyword rich content. I only want the search engines bots to see it so they rank me higher, but I want to show users another page. (This happened to www.bmw.de -- see http://abcnews.go.com/Technology/story?id=8252259&page=1 )

Now consider your scenario.

You are both redirecting based on IP which not recommended by anyone practicing "whitehat" SEO or google. You should serve up the same content based on IP and referring URL for the most part. You could alter something smaller such as a price, but not change the language on your page which makes it look 100% different to a computer. The only thing that you should really change is based on the user agent for browser compatibility.

Google will hand-audit sites (see Matt Cutt's blog) for sites that are suspected of serving up different content and make judgement calls in some cases, in other cases it is purely automated.

I would recommend you:
* Allow all ips to view all languages
* Substitute price and currency notation based on IP. Only a small part of the page will change.

This should actually benefit you. Say you have 50 pages for each language. Now say the google bot was detected as a US ip. It would have access to 50*1 pages. If you allow all ips to view all languages you have 50*5, where 5 is the number of translations you have online. The increased number of pages will help increase your PageRank.

[sknake]

Ah -- I didn't notice that you were also redirecting to other domains. This is a double whammy. Domains that forward should use an HTTP-301 "Content Moved Permanently" redirect to notify google that the domain has basically shut down (you may be doing this, you didn't post your redirect code). You should also make this update in your Google Webmasters Tools (not in your case, since that isn't the intended effect).

If google can access your .fi site with its finland bot, then it can't with its' US bot its going to smack your rankings for "gateway domains" I believe they are called. Basically it thinks you have a domain that is being advertised to gain page rank and you're passing the traffic off to yet another domain. It doesn't follow their rules of linking so they smack you for it to discourage the practice.

Bottom line -- Never transfer to another domain based on IP. Google bots will pick up on this behavior as they have ips all over the world and assume you're trying to beat the system