Is Spam Killing Google?

Brad DeLong offers Vivek Wadhwa’s argument that we desperately need a successor to Google’s core web search business:

The problem is that content on the internet is growing exponentially and the vast majority of this content is spam. This is created by unscrupulous companies that know how to manipulate Google’s page-ranking systems to get their websites listed at the top of your search results. When you visit these sites, they take you to the websites of other companies that want to sell you their goods. (The spammers get paid for every click.) This is exactly what blogger Paul Kedrosky found when trying to buy a dishwasher. He wrote about how he began Googleing for information…and Googleing…and Googleing. He couldn’t make head or tail of the results. Paul concluded that the “the entire web is spam when it comes to major appliance reviews”. […]

Content creation is big business, and there are big players involved. For example, Associated Content, which produces 10,000 new articles per month, was purchased by Yahoo! for $100 million, in 2010. Demand Media has 8,000 writers who produce 180,000 new articles each month. It generated more than $200 million in revenue in 2009 and planning an initial public offering valued at about $1.5 billion. This content is what ends up as the landfill in the garbage websites that you find all over the web. And these are the first links that show up in your Google search results.

In my experience, this is quite true but also a more narrow problem than Wadhwa makes it out to be. For almost everything, searching the web with Google works great. For product reviews, it doesn’t work very well except as a way to look up specific individuals’ reviews of things.

But developing a whole new search engine seems like an inordinately difficult response. For starters, the reason Google’s so spammed-up is precisely because it’s so popular. Any very popular search site will become a target for similar manipulations. The business innovation that’s needed isn’t so much a better way to search for product reviews, but instead firms that specialize in providing reliable product reviews. It’s a big problem, but a pretty specific one. Personally, I find there’s a lot to like about the Consumer Reports website.