Google logo
Google is overhauling the way it indexes web pages - not only to improve its search results, but also to compete effectively with the large social networking sites that users are increasingly turning to for information.
The world's most popular search engine announced Wednesday it is launching a new system, dubbed Caffeine, to index web pages.
Recently, Google has been under pressure to compete with sites such as Twitter and Facebook, which let users search for information posted by other users in real time. Facebook is quickly challenging the search engine for the title of the world's most-visited site, and Google has previously implemented real-time search specifically as a way to incorporate Tweets into its results page.
The move to a new indexing system is squarely aimed at keeping Google's traditional search engine relevant in a world of real-time information - from Tweets to status updates.
"Content on the web is blossoming. It's growing not just in size and numbers but with the advent of video, images, news and real-time updates, the average webpage is richer and more complex," Google software engineer Carrie Grimes wrote on the company blog. "In addition, people's expectations for search are higher than they used to be. Searchers want to find the latest relevant content and publishers expect to be found the instant they publish."
When a user enters a search query, Google doesn't search the Web directly, but instead searches its own index of information about the Web. Everything else being equal, the more closely a search engine's index mirrors the Internet, the better the search results.
Caffeine is designed to update Google's index more quickly than the previous system, something the company must do in order to provide relevant search results about items such as breaking news events.
The new indexing method is also likely to have an impact on thousands of small businesses that depend on traffic from Google. An entire industry - "search engine optimization" - exists to help companies tweak their websites in order to appear earlier in certain search engine query results. Google's move to a faster indexing system will likely force a similar change in the SEO industry.
Google isn't the only search engine responding to the rise of real-time information. Microsoft's Bing search engine also announced this week that it will incorporate Twitter and Facebook posts in its interface. Specifically, Bing will have a page dedicated to such media, which will keep track of people and events making waves in social networks.