We've all seen them -- a user survey, often performed by Respected Independent Firm, but promoted by Vendor Y -- which just happens to come up with the result that users want the products, or the features, that vendor Y provides. Amazing how that works.

Here's the six telltale signs to take such a survey with a grain of salt.

1. Lack of detail about how the participants came to be included. For survey results to be statistically significant, the people need to be randomly chosen. Even if they're all high-powered executives, you don't want them all to be in the same geographic area or industry. (You may even find that all the users just happen to be customers of Vendor Y.) Also, to make sure there's a representative sample, a reputable survey will typically be calling or mailing the user. Surveys that are performed by having users decide to go to a website are what's called self-selected; the people who participate are either doing so because they're really happy -- or they have an ax to grind.

2. Look for the line "Respected independent firm X was commissioned by Vendor Y." That means Vendor Y paid for it. How likely is it that the results are going to be something Vendor Y doesn't like?

3. Meaningless graphs. Look for a graph measuring something impossible to quantify, such as "User satisfaction," typically with no units on the graph, and with the graph forming a perfect line or curve.

4. Lack of detail about the survey methodology. Are the questions listed? Is there a figure for margin of error, confidence level, or standard deviation? (Don't worry about what the terms mean. The important thing is that they mean the survey is statistically valid.)

5. Vagueness in terminology. Look for terms that don't really mean anything, terms the vendor or the survey has invented, multiple terms for similar concepts without a clear explanation of how they differ, and so on.

6. The "Duh" factor. Look for Mom-and-apple-pie issues that of *course* any rational person would support, but that the survey presents in breathless terms.

Once you've read through this, use your favorite search engine to find press releases about surveys, see how many you find -- and see how many publications and websites picked them up. Play it with your friends!