I have had some local webmasters tell me that wealthy marketers can hurt the little advertiser by driving up the bid amount on AdWords that they might want to use.
If that were true, then there would be no point in trying to bid on any words that the big guys might use.
Anyone know if this might be the case?
Jump to PostI believe you can use the meta tag for your desired functionality:
<meta name="robots" content="noarchive" />but I am not 100% whether this doesn't cache your pages but still indexes them.
Adding disallow: googlebot to your robots.txt file makes Google completely ignore you.
Jump to PostYou're hopeless.
It's impossible to hide the affection with which you say that.
Jump to PostYou know I love ya, tgreer. That isn't to say that I don't tend to disagree with you more times than not, but ... eh, what the hell!
Jump to PostI still don't get the whole Google is evil thing. I'm a fan of the AdSense / AdWords system.
Jump to PostThey aren't necessarily "evil". They are a business. Specifically, they are in the business of scraping/gathering (but not authoring) content, and then finding ways to sell advertising against it. I think that is unethical, but "evil"? Not quite. However, that is their business model, so it's a given that everything …
Of course it's the case. The entire Adwords/Adsense system is profoundly flawed. It's like Las Vegas, where the casinos always win.
Google scrapes the web for content, caches it locally on their servers without permission, and then sells keywords against that content (which they didn't generate).
The initial idea of PageRank was very good: that a site's popularity could be measured through backlinks, which can be considered as "votes". Back when the web was new, that was valid.
As soon as they started selling keywords, monetizing content, and leaking little tidbits about their practices to fuel the SEO craze, they lost all credibility, in my opinion.
Sorry, I fail to see the unethical aspect to search engine indexing and crawling. The way that I see it, Google provides you with a service. They send you free traffic. You have the option of not receiving this free traffic by using a robots.txt file. However, if you enjoy the service Google offers you, which is sending visitors to your site, then you shouldn't complain about them indexing your content.
Sorry, it was just the way you said Google "caches [your content] locally on their servers without permission."
There is a difference between indexing, and caching. There is also a difference between behaving as a legitimate search engine, vs. scraping the entire web for content, and then re-packaging that content as sales goods.
Google is like a virus that co-opts some mechanism in the organism to reproduce. Because of Adsense/Adwords, the web "content" these days is turning into pre-packaged, easy-to-digest, monetized content for Google. Hardly anyone puts out a site these days without excessive hand-wringing about SEO/Google: what keywords to pick, will using SiteMap or Analytics make them better indexed, and similar nonsense.
There was even a question on the forum here, about changing servers: "Will having a new IP address hurt my rankings?"
This is the new state of the web.
And yet ... you don't use a robots.txt file to ban googlebot on your site.
I certainly use robots.txt. I don't prevent googlebot, you're right, because it isn't clear to me that Google differentiates between caching and indexing. Their FAQ on the subject of caching contains instructions for removing a site from their "index". They are being purposely vague about the differences.
If there is an instruction I can add to robots.txt that will prevent caching but allow indexing, I'd certainly like to know what it is.
Indexing, according to Google, is storing a local version of your pages within their database, to be searched. Their search engine works by searching a local version of your pages. How else would you like them to be able to perform fast searches if the content being searched wasn't local to them? Confused where the ambiguity is.
A book, and a book's index, are separate things, Dani. If Google wants to index my site, and return my site's URL in response to a search on indexed topics (digital printing, PostScript, PDF, etc.) then you're right: this is a service to me and my site. Thanks.
Google is the only search engine that uses a complete, cached copy, of your pages, and then uses their local cache, to generate an index.
cache != index
Even that would be OK, but they then allow users to view the cache, rather than going to your page.
Moreover, they allow companies to bid on keywords, which completely bypasses the natural search engine mechanism.
The combination of caching content (from Web, from Gmail, from Print, and now with Google Apps, add in all your business documents, and with Google checkout, your user's buying patterns), plus monetizing that content, is the problem.
A book and book's index are separate things, yes. But what I am saying is that Google's index and Google's cached version of pages are synonymous.
Allowing users to view the cache ... ok, I'll give you that you may not like this "feature". Perhaps it shouldn't exist. Fine. But if you can appreciate the service Google offers you in sending free traffic, you have to take the good with the bad.
Google's variation of keyword bidding is 1000% better than what some other search engines, such as the old goto.com, used to do. Google, at least, clearly labels sponsored listings in a separate area than natural search results. Goto.com used to intermingle them all together, so every search was just a list of sponsored links.
But, like I said, if you aren't comfortable with Google's caching / indexing / whatever you call it, then don't use them. But if you are thankful for the service they provide to you, don't complain about the negatives, which are far outweighed by the amount of traffic they are able to give us website owners BECAUSE of the way they spider content and not in spite of it.
We definitely have evaluated the risk/reward ratio differently here. I see the RISK of privacy abuses by Google as a result of them caching everything they can get their hands on (not just the web, but emails, books, business documents, site traffic stats, buying habits, search terms), and then re-selling it for profit, far outweighs the rewards.
I don't know for certain (because they ain't saying), but I would be very surprised if what you say is true, that their cache IS their index. I've asked, and have had no reply. You're saying they search their entire cache in response to each and every web query? I really doubt it.
I'll ask again, does anyone know? If I add a 'no-cache' directive to my robots.txt, will my site still be indexed? If not, you're absolutely right: I'll have to decide if it's worth it to me, or not, to be indexed by Google.
I believe you can use the meta tag for your desired functionality:
<meta name="robots" content="noarchive" />but I am not 100% whether this doesn't cache your pages but still indexes them.
Adding disallow: googlebot to your robots.txt file makes Google completely ignore you.
We'll see. Added to my forum. Thanks.
Despite the fact that big advertisers can drive up the cost some, Google is still by far the most democratic when it comes to sharing their ad space where the sponsored links are concerned. Even if you spend twice what everyone else does you're not guaranteed that first spot everytime. Better than Yahoo where you can see who's outbidding you and exactly how much they're spending per click.
Here is the gist of most of the replies I have received on this subject:
"The minimum bid, also known as the minimum CPC, is the least that one can pay to have an ad appear for a particular keyword in a particular account. It is very important to know, however, that one's minimum bid is entirely unrelated to how many other advertisers are using the same keyword. Instead, since August of 2005, the minimum bid has been quality based. To put it simply, the higher the Quality Score of a keyword, the lower one's minimum bid will be for that keyword."
I have to admit, it's Greek to me...
It's called "double-talk". If you can fill normal sentences/paragraphs with terms that you never define and that no one really understands, you can:
1. Satisfy the public's demand for answers without really providing any and,
2. Provide gainful employment for people who claim to understand the terms and for a small fee will put them into practice for you.
In all cases where you encounter double-talk, the proper interpretation is "this is something that makes us obscene amounts of money, and so we simply aren't going to explain it to you."
Ahh, the joys of George Orwell. As I understand it, your position in AdWords is based on a formula which combines both the amount of competition, the bids of competing ads, and the CTR of your ad.
Orwell, definitely! I'm also struck by certain passages in Plato's Republic.
"If, though unjust, I acquire the reputation of justice, a heavenly life is promised to me. Since then, as philosophers prove, appearance tyrannizes over truth and is lord of happiness, to appearance I must devote myself. I will describe around me a picture and shadow of virtue to be the vestibule and exterior of my house; behind I will trail the subtle and crafty fox."
He even talks about SEOs and Matt Cutts:
"With a view to concealment we will establish secret brotherhoods and political clubs. And there are professors of rhetoric who teach the art of persuading courts and assemblies; and so, partly by persuasion and partly by force, I shall make unlawful gains and not be punished."
Someone at Google clearly knows the classics!
You're hopeless.
You're hopeless.
It's impossible to hide the affection with which you say that.
You know I love ya, tgreer. That isn't to say that I don't tend to disagree with you more times than not, but ... eh, what the hell!
Google scrapes the web for content, caches it locally on their servers without permission, and then sells keywords against that content (which they didn't generate).
LOL - I just got a private gmail from a friend who likes cats. The text of her email did not mention a single word about cats, but the paid ads on the right were mostly for cat stuff.
I suppose Google knows what brand of toothpaste all my friends use, too.
And - Tgreer...
It's called "double-talk". If you can fill normal sentences/paragraphs with terms that you never define and that no one really understands, you can:
1. Satisfy the public's demand for answers without really providing any
I have to admit that the first time I heard Google's explanation of PPC, that was exactly my theory - in fact, I just KNEW it in my bones. I said "Duh - Google is making all this up - all the way to the bank!" I worked for a large electronics store where they did the same thing while supposedly "Fixing" customers' computers. (For obscene amounts of money)
I also spent my $5 initiation fee for adwords, and decided on a budget of $200 a month. But I was trying to expose my blog about "xxxxxxxxx", (which Googles very well....)
But the single key words I chose were very expensive - since I was in competition with the enitire NAR. After over a week I still hadn't had one single click. So I suspended it.
I tend to be pretty suspicious of just about everyone lately. I suppose that "Dont' be evil" probably went out last season sometime....
:eek: Uh-oh. I Googled up this forum - I hope they don't read it and try to get back at me by taking it out on my friend's cats!
I still don't get the whole Google is evil thing. I'm a fan of the AdSense / AdWords system.
They aren't necessarily "evil". They are a business. Specifically, they are in the business of scraping/gathering (but not authoring) content, and then finding ways to sell advertising against it. I think that is unethical, but "evil"? Not quite. However, that is their business model, so it's a given that everything they do is to further their own business interests. The proliferation of junk sites that are essentially "made for Adsense" is their fault. Such sites dilute the overall value/quality of the web. Does Google care? Of course not, since they make huge amounts of money off of these sites. So the overall effect of Google and AdSense on the web has been overwhelmingly negative.
I view them in much the same light as environmentalists view timber companies. Whatever short-term benefit they bring has to be compared to the long-term effects on the environment. Google's environment is the web, and they are polluting it for their own financial gain.
(Not to mention the immense opportunity and temptation for privacy violations, since they not only scrape massive amounts of content, cache it essentially forever, but also maintain search history, personal emails, site traffic patterns, etc.)
I know...be careful not to lose you temper!
I only used gmail because "Adelphia" wouldn't fit on my business cards. But Adephia was just bought by Warner Cable, so maybe I can switch to RR.net. And try not to move or change.
They are polluting the web, and invading my privacy. Although I was just able to find an old, lost web page that I thought had been lost forever, and I was pretty glad for that.
Where on earth do they keep this stuff? Is it backed up on tape - on disk - on 16,000,000,000,000 little thumb drives?
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.