In Part 8, the world's scientists quietly discovered that a graphics card could do their heavy lifting — folding proteins, colliding galaxies, pricing derivatives — while the box hummed under a desk like the gaming rig it secretly was. NVIDIA had spent years arguing that a GPU was really a machine for doing the same sum a million times over. In September 2012, three researchers in Toronto proved it in a way no marketing slide ever could. They did not set out to change NVIDIA's fortunes. They set out to win a contest about pictures.
The contest nobody thought a graphics card could win
The contest was the ImageNet Large Scale Visual Recognition Challenge, an annual test built on a dataset of over a million labelled photographs sorted into 1,000 categories — a thousand ways to be a dog, a mushroom, a container ship. Each year, teams submitted software that tried to name what was in an unseen set of images, and each year the results improved by grinding, incremental fractions. The reigning approaches were elaborate hand-engineered pipelines, decades of computer-vision craft distilled into features a human had designed by hand. In 2010 the winner missed roughly 28 percent of the time. In 2011, 26 percent. Progress looked like sanding down a very hard wall.
Then a team calling itself SuperVision submitted an entry on September 30, 2012. Its three members — Alex Krizhevsky, Ilya Sutskever, and their advisor Geoffrey Hinton at the University of Toronto — had ignored the hand-crafted playbook entirely. Their system, later christened AlexNet, was a deep convolutional neural network: eight layers of artificial neurons, sixty million parameters, that learned to see purely by being shown examples. No human told it what an edge or a whisker was. It figured that out itself.
The result did not sand the wall. It knocked it over. AlexNet finished with a top-5 error rate of 15.3 percent, more than ten percentage points ahead of the runner-up sitting near 26 percent. In a field where a single point was a good year's work, a newcomer had won by a landslide. The room understood immediately that something had shifted. The old approach had not been beaten so much as retired.
Two GTX 580s and a bedroom-scale budget
Here is the part that made NVIDIA's leadership sit up. The team did not train AlexNet on a supercomputer or a data-centre cluster. They trained it on two NVIDIA GeForce GTX 580 cards — consumer gaming hardware, the kind of thing a teenager would buy to run a shooter at higher frame rates. Each card had just three gigabytes of memory, so little that the network literally did not fit on one. Krizhevsky split the model in half and ran a piece on each GPU, a workaround born of a hardware constraint that happened to work beautifully.
The code that drove them was written in CUDA, the very C-like language NVIDIA had shipped in 2006 and spent years insisting people would eventually need. Training took five to six days across ninety passes over the 1.2 million images. What had been theoretically possible for decades — deep neural networks were an old idea — had been stuck behind a wall of raw arithmetic. GPUs, and CUDA, quietly removed the wall. The intelligence was in the algorithm; the fuel was NVIDIA silicon.
AlexNet's tricks read now like the founding grammar of a field: ReLU activations that let deep networks train without stalling, dropout to keep them from simply memorising, and the sheer nerve to make the network deep in the first place. But the deepest lesson was economic. If two gaming cards costing a few hundred dollars each could topple the best of hand-built computer vision, then the bottleneck on artificial intelligence was no longer clever engineering. It was compute — and compute was exactly what NVIDIA sold.
The spark that lit the fuse
The aftermath moved with startling speed. Within months, the entire computer-vision community pivoted to deep learning; by the next year's ImageNet, nearly every serious entry was a neural network running on GPUs. The three researchers formed a company, DNNresearch, and in early 2013 Google acquired it, folding Hinton, Sutskever, and Krizhevsky into its ranks. Sutskever would go on to help found OpenAI two years later. The people in that Toronto lab did not just win a contest; they seeded a decade.
For NVIDIA, the AlexNet moment was the vindication of a decade-long bet nobody had asked it to make. Jensen Huang's company had built a general-purpose parallel computer, wrapped it in a games business to pay the bills, and waited for the world to find a use for it. Now the world had, and the use was the most consequential technology of the century. Demand for GPUs to train neural networks began a climb that has not stopped since. The graphics company that had almost died three times over was about to become the arms dealer of the AI age — and, crucially, it now knew it.
Recognising the moment was one thing. Betting the company on it was another. Next: NVIDIA goes all in on AI — cuDNN, the DGX-1, and a supercomputer hand-delivered to a startup called OpenAI.
Comments
Post a Comment