In Part 9, three researchers in Toronto used two gaming cards to win an image-recognition contest by a landslide, and NVIDIA's leadership understood the message: the same silicon they sold to teenagers was the fastest engine on Earth for training neural networks. Recognizing a wave is one thing. Betting the company on it is another. Between 2014 and 2016, NVIDIA stopped treating deep learning as a happy accident and started building an entire stack for it — a software library, a purpose-built machine, and, eventually, a supercomputer carried by hand to a startup's front door.
cuDNN: teaching the GPU to think in neurons
The first move was quiet and technical. In September 2014, NVIDIA released cuDNN — the CUDA Deep Neural Network library. It was not a product you could see or a chip you could hold. It was a set of hand-tuned building blocks: the convolutions, pooling, and activation routines that every neural network runs millions of times during training. Researchers had been writing these by hand in CUDA, each lab reinventing the same wheel with varying skill. cuDNN did it once, exquisitely, at the level of NVIDIA's own hardware engineers.
The genius was in where it sat. Rather than force every scientist to become a GPU-optimization expert, cuDNN slid underneath the popular frameworks — Caffe, Torch, Theano, and later TensorFlow — so a researcher could write clean, high-level code and get low-level speed for free. Drop in the library, and a model simply ran faster, on every future NVIDIA card, without changing a line. It was the same strategy that made CUDA itself durable: own the layer everyone builds on top of, and you own the platform. By the time deep learning caught fire, NVIDIA had already paved the road it would travel on.
The DGX-1: a supercomputer in a box
Software was half the bet. The other half was iron. On April 6, 2016, at its GTC conference in San Jose, NVIDIA unveiled the DGX-1 and called it, without much modesty, “an AI supercomputer in a box.” The name was earned. Inside a single 3U chassis sat eight Tesla P100 GPUs — the debut of NVIDIA's new Pascal architecture — wired together not through the ordinary bottleneck of the motherboard but through NVLink, a custom high-speed mesh that let the eight chips trade data almost as if they were one.
The numbers were startling for a machine you could roll under a desk: 128 GB of blazing HBM2 memory, 170 teraflops of half-precision throughput, dual Intel Xeon server CPUs, and a 3,200-watt appetite for power. The list price was $129,000. NVIDIA's pitch was that the DGX-1 collapsed what used to be a room full of servers — and weeks of finicky assembly and tuning — into a single turnkey appliance that arrived ready to train models on day one. For AI labs racing each other, the scarcest resource was not money but time, and the DGX-1 sold time.
The box carried to OpenAI's door
Every product needs a story, and NVIDIA got one it could not have scripted better. OpenAI had been founded only months earlier, in December 2015, as a non-profit AI lab backed by Silicon Valley figures including Elon Musk, with more than a billion dollars pledged to its mission of building AI “to benefit humanity as a whole.” A young lab with grand ambitions and a hunger for compute was exactly the customer NVIDIA wanted the world to picture.
So on August 15, 2016, NVIDIA's CEO Jen-Hsun “Jensen” Huang did not ship the first production DGX-1. He carried it himself to OpenAI's San Francisco office and handed it over in person. Photos from the day show Huang and Musk standing over the machine; before it powered on, Huang signed the chassis with a dedication to the future of computing and humanity. It was equal parts gift, marketing, and prophecy. Ilya Sutskever — the same researcher who had co-built AlexNet four years earlier, now OpenAI's chief scientist — called the machine “a huge advance,” saying it would let the lab attempt experiments that had simply required too much computing power to try before.
The symbolism was almost too neat. The person who had proven neural networks on two consumer gaming cards was now unboxing a purpose-built AI supercomputer, hand-delivered by the company whose chips had made his breakthrough possible. NVIDIA was no longer a graphics company that happened to be useful for AI. With cuDNN underneath and the DGX-1 on top, it had declared itself the infrastructure of a new industry — and delivered that declaration, literally, to the doorstep of the lab that would come to define it.
The bet on AI was placed. But before that future paid off, an entirely different crowd discovered NVIDIA's cards and started buying them by the pallet — not to train neural networks, but to mint digital money. Next in Part 11: Dig for Gold, Sell the Shovels — the crypto mining boom, the bust, and the hard lesson NVIDIA learned about riding a gold rush.
Comments
Post a Comment