Fixing Google

Claire Cain Miller has an article in The New York Times about issues with Google:

In a tacit admission that web publishers are flooding its search engine with low-quality pages, Google has revised its methods to improve the usefulness of its results. Google said the change would raise the rankings of high-quality Web sites and reduce those of lesser sites, affecting twelve percent of search queries.
Sites known as content farms, which churn out sometimes mindless articles based on what people are searching for, have recently worked their way to the top of search results, frustrating some Google users. High rankings in search results are crucial because they allow websites to get more traffic and bring in more business, either through sales of goods and services or through advertising.
“I haven’t seen as much negative attention on Google’s results as I have in the last month or two; it’s been fairly unprecedented,” said Danny Sullivan, editor of Search Engine Land and an industry expert.
Persuading users that it has the best results is crucial for Google, whose reputation and status as the front door to the Internet depend on them. Though there were many search engines before Google, it became the dominant player because its technology produced better results for users. If people begin to doubt the quality of its results, Google risks losing them to competitors.
While so-called content farms can provide useful information, many of their articles are of questionable value but achieve high rankings in searches. For example, an eHow article on making friends in college includes tips like “consider joining a sorority or fraternity” and “remember to have a good time, smile, and laugh.”
Google makes about five hundred changes a year to the algorithm, or formula, that runs its search engine, most of them minor. Amit Singhal, a Google Fellow who worked on the latest change, said in an interview that users were likely to quickly notice this one, which was announced late Thursday. “We haven’t done a change where we have impacted low-quality sites at this level in years,” Mr. Singhal said. “It’s a clear evolution of the algorithm as the Web is evolving, the content on the Web is evolving, the user expectation is evolving.”
Google still dominates the web search market, with a 66 percent share in the United States and a larger one in many other countries, according to comScore, a Web analytics company. But it faces ambitious competitors, most notably Microsoft’s Bing.
Hitwise, an analytics firm, measures how happy users are with their searches by looking at how many are successful, meaning the user stays at the first site they click on. At Bing, 82 percent of searches are deemed successful. At Google, the figure is 66 percent.
“This change is about more than just cleaning up content farms,” said Chris Copeland, chief executive of GroupM Search, a search marketing firm that is part of the advertising company WPP Group. “Google has a relevancy problem, and they are trying to do something about it.”
Google made the change after technology bloggers, industry analysts, and everyday users complained that its search results had useless pages. The response may help Google’s reputation, Mr. Sullivan said. “The change may not necessarily improve the results, though hopefully it will, but it will definitely improve the perception of Google,” he said.
The new algorithm change does not address the full scope of techniques that sites use to manipulate Google. It is a constant cat-and-mouse game: as soon as Google makes a change, web developers figure out a way around it.
When Google’s search engine was first introduced, in 1998, its primary advantage was that it considered the number of times other sites linked to a certain page, weighing those links as though they were endorsements. But as people quickly learned how to manipulate those links, Google’s search began focusing more heavily on other factors, too. Google has punished e-commerce sites, including J. C. Penney, for inflating its rankings by paying for links from unrelated sites.
“Our algorithm clearly gets attacked by these techniques every day,” Mr. Singhal said. “However, with the amount of information that we have, we are pretty far ahead in the game.”
Though Google’s announcement did not explicitly mention content farms, and the company declined to say which sites were appearing lower in results, Matt Cutts, who leads Google’s spam-fighting team, has spoken in recent weeks about content farms and said Google was working on ways to deal with them. “There are some content farms that I think it would be fair to call spam, in the sense that the quality is so low-quality that people complain,” Mr. Cutts said in a recent interview.
Sites that are frequently given the “content farm” label include Yahoo’s Associated Content, AOL’s Seed, and Demand Media’s eHow and Answerbag. Demand Media, for example, uses software to track what people are searching for on Google and other sites, generates headlines based on those searches, and pays small amounts to freelancers to write the articles.
Criticism of these sites has been on Google’s radar, and the company said it had worked on addressing these problems for more than a year. This winter, Demand Media went public, and its shares jumped 33 percent on its first day of trading; it is now worth $1.9 billion. Around that time, technology bloggers began writing posts like one that complained that a Google search for new dishwashers had produced useless results.
Of course, the quality of a particular site is subjective. To determine quality, Google does things like track “boomerang” searches, when people click on a link and promptly click back to the results, and ask people to compare search results.
Demand Media, which relies on traffic from Google for its livelihood, said in a blog post that it applauded the changes, but that it was too early to determine the long-term effect on Demand’s sites. This week, Richard Rosenblatt, Demand’s chief executive, said it was working to bring in readers from sites other than Google, and introduced a site that discusses Demand’s quality controls. The company’s stock opened sharply lower Friday but closed 1.6 percent higher at $22.96.
Some consultants who help websites to improve their search rankings said sites like Demand’s might not feel the brunt of the change. They said Google’s real target was the hundreds of no-profile companies that post duplicate copies of the same text on hundreds of websites.
Many of the sites will figure out a new way to climb back up Google’s rankings, said Mr. Copeland of GroupM Search: “This is a group of people who will analyze this change, come back with a new strategy on Tuesday and be ranking by Thursday,” he said. “It’s kind of like what happens when drug dealers get busted. They don’t find new jobs. They switch corners.”