Need Underground Google?

Computerworld - There has never been a search engine that accurately reflects the Internet.

In the 1990s and 2000s, the limitation was technical. The so-called "deep web" and "dark Internet" -- which sound shady and mysterious, but simply refer to web sites inaccessible by conventional means -- have always existed.

Many parts of the Internet are hard to index, or are blocked from being indexed by their owners.

Companies like Google have worked hard to surface and bring light to the "deep, dark" recesses of the global web on a technical level.

But in the past few years, a disturbing trend has emerged where governments -- either through law or technical means or by the control of the companies that provide access -- have forced inaccuracy, omissions and misleading results on the world's major search engines.

The censorship
Until recently, search engine censorship was not on the list of first-world problems. But in the last few years, governments in the United States, Europe and elsewhere in the industrialized world have discovered that, although they're prevented by free-speech laws from actually blocking or banning content where it lives, censoring search engine results is a kind of "loophole" they can get away with. In an increasingly digitized, search-engine discoverable world of content, censoring search results is a way to censor without technically violating free speech protections.

Starting in 2011, companies like Google started reporting a disturbing rise in government requests for search engine results to lie -- to essentially tell users that existing pages and content on the Internet do not exist when in fact they do. Requests for such removals by the U.S. government, for example, rose 718% from the first half of 2011 to the last half. And they've continued to rise since.

And such requests weren't just coming from the U.S., but from "Western democracies not typically associated with censorship," according to the Google policy analyst who reported the trend on behalf of the company and talked about Google's Transparency Report.

The reasons for these requests vary, and often sound reasonable -- national security, law and order, national pride, religious sensitivity, social order, suppression of hate speech, privacy, protection of children -- you name it. But when you add them up and allow them to grow in number over time, the cumulative effect is that increasingly, search results don't reflect the real Internet.

Many of these cases start out with the best intentions but result in serious problems. Let's start with a disturbing recent case in Canada.

A Supreme Court of British Columbia ruling on an intellectual property dispute between two small industrial equipment companies ordered Google to not only delete all search results referring to one of the companies, but all future such results as well -- not only in Canada, but worldwide. (Yet another unsavory dimension to the case was that the ruling applied only to Google. Bing and other search engines were not required to comply.)

The particulars of the case are irrelevant and the data involved unimportant. The precedent that a government in one country could censor information in other countries has bad implications if allowed to stand. Imagine if China were allowed to censor information about the Dalai Lama within the US, or if Pakistan were allowed to censor images offensive to Muslims in Denmark.

Even more recently, the European Court of Justice brought into existence Europe's "right to be forgotten" ruling. In a nutshell, Europe wanted to protect citizens from the fact that the Internet never forgets.

Why we need an underground Google
Governments are forcing search engines to show wrong results. It's time for search engines to go rogue so they can be right.

The particular case heard by the court involved a Spanish man who was in the press for serious debt problems, but who later climbed out of debt. Rather than ruling that the actual information about his money problems be removed or censored, the court invoked the search engine loophole for censorship and ordered Google, Bing and other search engines to remove his name as a search query that returned the outdated information about his finances.

Worse, the ruling required search engines to offer a process by which any European could request similar treatment, and ordered Google, Microsoft and other search engine companies to judge whether those requests were valid and to take action on the valid ones.

At last count, Google had received some 70,000 requests for changes to search results under the ruling in the past month. Microsoft only this week launched its process for censoring results.