By 1996, the web had a discovery problem. Millions of pages were coming online, and the search engines of the day—AltaVista, Lycos, Excite—mostly worked like frantic librarians who could read but not judge. Type a query and they would hand back every page that happened to contain your words, ranked by little more than how many times those words appeared. It was easy to game, easy to flood, and increasingly useless. The more the web grew, the worse search got.
Larry Page, then a PhD student at Stanford, was circling a different question entirely. Casting about for a dissertation topic in 1995, he became fascinated not by the content of web pages but by the links between them—the mathematical structure of the web itself. What if the connections mattered more than the words?
Borrowing an idea from the library
The insight Page reached for was older than the web, and it came from academia. For decades, scholars had measured the influence of a research paper by counting its citations—a technique pioneered by Eugene Garfield in the 1950s. A paper cited by hundreds of others was probably important. A paper cited by a single landmark study might be more important still. Citations were a currency of trust, accumulated one reference at a time.
Page realized the web already worked the same way. Every hyperlink was, in effect, a citation—one page vouching for another. Nobody had thought to read the web backwards: instead of asking "what does this page link to?", ask "who links to this page?" Those inbound links, or backlinks, were a vast, untapped map of what the web itself considered worth pointing at.
He gave the project a fittingly cheeky name: BackRub. In March 1996, Page pointed a homemade web crawler at a single seed URL—his own Stanford homepage—and let it wander outward, following links and recording who pointed where. Sergey Brin, the mathematician with an NSF fellowship and a taste for hard problems, joined soon after. The engineering muscle came from Scott Hassan, who wrote much of the early crawler code. Together they began pulling the web's link graph into a Stanford basement.
The math that ranked the web
Counting backlinks alone wasn't enough—spammers could always manufacture a thousand junk links. Page and Brin's real breakthrough was recognizing that not all votes should count equally. A link from a page that was itself important should carry more weight than a link from an obscure one. And the importance of that linking page depended, in turn, on who linked to it.
That circular definition sounds like it should collapse into nonsense, but it's exactly the kind of self-referential system mathematics knows how to solve. The result was an algorithm they named PageRank—a pun that worked on two levels, referring both to web pages and to Larry Page himself. Importance flowed through the web like water finding its level, each page inheriting a share of the trust held by the pages that pointed to it.
The elegance was that the web scored itself. No editor, no directory of hand-picked sites, no human deciding what mattered—just the collective judgment of everyone who had ever bothered to make a link. A single endorsement from a genuinely trusted source could outweigh a hundred desperate ones, which made the whole scheme remarkably hard to fool.
A tool that quietly outclassed everyone
The difference showed instantly. Where rival engines returned bloated lists padded with keyword-stuffed junk, BackRub surfaced the pages people actually wanted—often at the very top. It felt smarter, because in a real sense it was borrowing the accumulated intelligence of the entire web.
By August 1996 the system was running on Stanford's network, indexing on the order of tens of millions of pages and gobbling up close to half the university's bandwidth—an early hint that this "research project" had ambitions no dorm-room server could contain. It ran on a scavenged mix of Sun and Intel machines, held together with academic budgets and stubbornness.
The idea proved valuable enough to patent. Stanford filed for it, was granted the PageRank patent in 2001, and licensed it exclusively to the company its two students would go on to found. In exchange the university took shares—a stake it later sold for roughly $336 million. Not a bad return on a dissertation topic.
In 1998, Page and Brin would lay the whole thing out in a now-famous paper, The Anatomy of a Large-Scale Hypertextual Web Search Engine. But long before the academic write-up, they had a problem of a very different kind: BackRub was outgrowing Stanford, outgrowing its name, and outgrowing its founders' plans to ever finish their degrees.
Next time: the awkward name that came from a spelling mistake, a garage in Menlo Park, and a $100,000 check written to a company that didn't legally exist yet.
Comments
Post a Comment