NEEDED: A SEARCH ENGINE FOR INDIA

The time has come for Indians to seriously think about having a search engine of our own. Google, Bing, and Yahoo (powered by Bing) are very powerful search engines. But that doesn’t mean there’s no room for others. Homegrown Search Engines play a meaningful role in many big world markets, including Russia, Ukraine, Kazakhstan, Belarus, China, South Korea, and the Czech Republic.

Yandex has around 60% of the search market in Russia, 43% in Belarus and a third (around 33%) of the market in Ukraine and Kazakhstan. Baidu has around 56% of the Chinese search market and Naver leads the South Korean market with around 70% market share. There’s also a search engine with a sizeable market share in the Czech Republic, known as Seznam. All of them compete with Google in their respective markets, and are no pushovers. Baidu and Yandex are Nasdaq listed companies. So those interested in researching more about these companies can look up their SEC filings, some of which are quite elaborate.

Yandex apparently has an edge in the Russian and neighboring markets because of the Search Engine’s ability to recognize the Russian inflection in search queries. Baidu has an impressive pedigree, with co-founder Robin Li boasting of sound search credentials, besides being one of the richest individuals in China with a net worth of more than $12 billion.
LI, A SEARCH PIONEER
Most Western narratives give one a misleading picture that Baidu's success is because it kowtows to Beijing and allows its search results to be censored. But it's a fact that internet giants like Google in the US were more than willing to hand over treasure troves of user data to the US government when their spy agency came calling, often voluntarily giving more than what was asked for. It's difficult to believe that working for the interests of the US government is benign, and heeding requests from Beijing is evil.

And here's a surprising nugget: Baidu's co-founder Robin Li is one of the pioneers of search engine technology, and we cannot outright dismiss the possibility that Baidu is successful because it returns better search in Chinese.

Even when they touch upon Robin Li's background, Western news media skirt around the fact that he's an American educated tech wizard, who patented search technology ahead of Brin and Page. He was one among a handful who arrived early at the conclusion that inbound links from other Websites are a crucial pointer towards the quality of a Website. Just as Larry Page had devised the PageRank algorithm named after himself, Li —an alumnus of SUNY, Buffalo — had worked out his own system, known as RankDex.

In his highly regarded book on Google called In the Plex, legendary tech writer Steven Levy had this to say about Robin Li:

“One day in April 1996 he was at an academic conference. Bored by the presentation, he began to ponder how search engines could be improved. He realized that the Science Citation Index phenomenon could be applied to the Internet. The hypertext link could be regarded as a citation! ‘When I returned home, I started to write this down and realized it was revolutionary,’ he says. He devised a search approach that calculated relevance from both the frequency of links, and the content of anchor text. He called his system RankDex.” 

He didn’t sit quiet with this insight though, notes Levy. Li first asked his company Dow Jones to file a patent, and when that didn’t happen, he bought a self-help book on patent applications and filed it on his own in June 1996. “But when he told his boss, Dow Jones re-asserted itself, and hired a lawyer to review the patent, which it re-filed in February 1997,” Levy writes. Remember, this was two years before Stanford filed for the PageRank algorithm in 1998. And Li’s insight on the centrality of anchor text way back in 1996 is simply amazing, considering the entire industry of search engine optimization (SEO) draws its competitive advantage from it.

When Dow Jones failed to monetize the patent, Li quit to join an internet company by name Infoseek. Eventually, he left the United States and found his fortune in the Chinese market. So the Baidu founder is someone with serious Search Engine engineering chops, and we should not blindly accept Western speculation that Baidu got where it's now simply due to government patronage.

Similarly, Yandex has a research lab in California’s Bay Area. So these homegrown Search Engines have good pedigree, are financially sound, and have market leadership which they won’t be ceding any time to multinational competitors like Google.

GURUJI, INDIA’S LOST SEARCH ENGINE
This begs the question: What about India? A random search found that there apparently existed a Search Engine in India called Guruji.com. Without research, and without speaking to its founders, I won’t be in a position to comment on what went wrong, but apparently it shut shop sometime in 2012. I could capture this screenshot of how Guruji.com looked like just before it shut down, from the Wayback Machine. Google, Bing, Yahoo, and their upstart competitors like Blekko and DuckDuckGo are formidable in what they do, but it would be foolish to simply abandon the market to these companies, considering none of them are still very competitive in searching in Indian languages. The field is open for a Search Engine powerhouse in Hindi, or major Indian languages like Bengali, Tamil, and Telugu.

Search Engine technology is not rocket science, and if India can develop the knowhow to commercially launch satellites for others, providing state funding for any fledgling start-up in the domestic Search Engine space until it finds its footing would be a wise investment for the future. There’s no guarantee how multinational Search Engines would behave in times of conflict which pits Western interests against India's. Therefore, it would be prudent that the country makes this a priority.