It sounds pretty damning. Two recent surveys suggest that Google Shopping isn’t leading searchers to the best prices on products. But the surveys weren’t well documented, nor did they include competitors like Bing, Shopzilla, PriceGrabber and Nextag, which have similar issues. So, Search Engine Land will be running its own fully documented tests. Every few days, we’ll search for an item and show what we found, starting with today’s test, a search for a toaster.How Much Is That Toaster In The Shopping Search Window?
This is a long story. It’s designed to be that way so that people can understand exactly what was tested and the reasoning for selecting prices to compare, which is lacking in the two other surveys. This type of details is important, especially when lobbing around accusations to the US Federal Trade Commission about potential consumer misleading, as Consumer Watchdog has done.
Take the time to read the full story if you want to understand just how very complicated it is to conduct a survey of this nature. But for those who want a quick summary now, the first test shows that Google actually ranked best in featuring a price that matched the lowest price that could be found from a major merchant:
The survey also found that all the shopping search engines have issues in terms of disclosing to consumers why they rank results in the way they do. Several have some serious issues with pricing inaccuracies.
Remember, a single test isn’t enough to draw a conclusion across the entire industry. I’m not sure that running six tests as the Financial Times did is enough, nor the 14 that Consumer Watchdog did is, either. But far more important, the how you test and what you count can produce skewing that can mess up even a large sample.
How Much Is That Toaster In The Shopping Search Window?
Why look for a toaster? That’s one of several items where Consumer Watchdog said its survey found a $20 price gap between what was listed on Google and what was listed as the lowest price for the same item at Shopzilla, PriceGrabber or Nextag, when it looked for “Cuisinart Classic 2P Slice Toaster.”
Bing wasn’t surveyed, even though it has shopping search ads exactly like Google. Nor did Consumer Watchdog explain the exact query was used or list a model number for the toaster. Nothing was documented. We were only given the end result, and when it comes to comparing shopping search engines, as you’ll see, it’s essential to show your work.
We can’t search for the exact same toaster as Consumer Watchdog did, because we don’t know the model. Plus, even if we did know it, looking for it might cause some to assume Google’s now “fixed” the toaster price problem for that particular model. So, for our test, we’re seeking a Hamilton Beach 2-Slice Metal Toaster, model 22504. Why? It’s one of the best sellingtoasters at Amazon, so it seems a popular model worth testing.
Which Price Do You Compare To?
At Google, the search began by entering “Hamilton Beach 2-Slice Metal Toaster” into the main search box. That brought back a special area of the search page with shopping search results, as the arrow points to below:
0 Comments