With the rapid development of artificial intelligence (AI), high-performance computing (HPC), and deep learning, computing systems are experiencing an increasing demand for high-speed interconnect technology. NVIDIA's NVLink technology has emerged as a significant solution in this field, evolving through several stages since its debut, driving continuous breakthroughs in computing power. This article will comprehensively review the development history of NVLink, exploring its technological iterations and application advancements.

The Origin and Background of NVLink

Before the advent of NVLink, PCIe (Peripheral Component Interconnect Express) was the primary communication interface between GPUs, CPUs, and other devices. However, as GPU computing performance rapidly improved, PCIe's bandwidth gradually became a bottleneck limiting overall system performance. To break this limitation, NVIDIA first proposed the concept of NVLink in 2014, aiming to provide a high-bandwidth, low-latency solution for communication between GPUs, and between GPUs and CPUs.

First-Generation NVLink: Groundbreaking Technological Innovation

NVLink 1.0 debuted in 2016 with NVIDIA's Pascal architecture-based Tesla P100 GPU. This version marked a revolutionary breakthrough in high-performance interconnect technology, featuring:

• Bi-directional bandwidth of a single NVLink reaching 20 GB/s, 5 times faster than PCIe 3.0.
• Support for direct interconnection between multiple GPUs, enabling efficient point-to-point communication.
• First collaboration with IBM to integrate NVLink into IBM POWER8 CPUs, providing high-speed interconnects between GPUs and CPUs.

The introduction of this generation of NVLink brought significant performance improvements to the fields of deep learning and scientific computing.

Second-Generation NVLink: Further Optimizing Bandwidth and Efficiency

In 2017, with the release of the Volta architecture, NVIDIA introduced NVLink 2.0. This version improved upon the shortcomings of the first-generation NVLink while significantly enhancing performance metrics:

• Bi-directional bandwidth of a single NVLink increased to 50 GB/s.
• Expanded the range of supported devices, including not only communication between GPUs but also complex topologies of multiple CPUs and multiple GPUs.
• More efficient power management, making it more competitive in high-performance computing clusters.

The advent of NVLink 2.0 enabled revolutionary progress for NVIDIA's Tesla V100 GPUs in AI training and inference tasks.

Third-Generation NVLink: Laying the Foundation for AI Supercomputing

In 2020, the release of the Ampere architecture was accompanied by the introduction of NVLink 3.0. This version took performance and compatibility to the next level:

• Bi-directional bandwidth of a single NVLink further increased to 100 GB/s.
• Supported more flexible topologies, such as fully connected and partially connected, adapting to different computing needs.
• Enabled efficient connection of up to 16 GPUs through NVSwitch technology, forming a unified computing pool.

NVLink 3.0 is widely used in NVIDIA DGX systems and AI supercomputers, providing powerful support for large-scale deep learning models (like GPT-3) and scientific simulation tasks.

Fourth-Generation NVLink: Future-Oriented Interconnect Standard

In 2022, with the release of the Hopper architecture, NVLink 4.0 officially debuted, representing the new generation of interconnect technology:

• Bi-directional bandwidth of a single NVLink increased to 200 GB/s, double that of the previous generation.
• Integrated new NVLink-C2C (Chip-to-Chip) technology, supporting direct interconnection between different chips (such as CPU and GPU), further reducing latency.
• Optimized multi-node connection capabilities for large-scale computing tasks, making it more practical in hyperscale data centers.

NVLink 4.0 extends NVLink's application scope from single servers to multi-server clusters, driving the development of HPC and AI technologies.

NVLink's Application Expansion and Technology Trends

With the continuous development of NVLink, its application scope is also gradually expanding. Initially, NVLink was primarily used for AI training and scientific computing tasks, but now it has been integrated into more areas:

1. AI Supercomputers

NVLink's application in NVIDIA DGX series systems has made it a core technology for training ultra-large-scale models.

2. Enterprise Data Centers

Thanks to its efficient power-to-performance ratio, NVLink has become the preferred interconnect technology for building AI workloads in enterprise data centers.

3. Real-time Graphics Processing

In high-performance graphics processing tasks requiring multiple GPUs, NVLink provides excellent multi-GPU communication capabilities.

Looking to the future, the development trends of NVLink primarily focus on the following aspects:

• Higher bandwidth and lower latency to further break through current technical bottlenecks.
• Cross-platform interconnection, enabling seamless connectivity between GPUs, CPUs, and other specialized chips through NVLink-C2C.
• Expansion towards quantum computing to explore application possibilities in new computing architectures.

From NVLink 1.0 to NVLink 4.0, NVIDIA has driven the innovation of high-performance interconnect technology through continuous iterations. The development of NVLink has not only solved the bottleneck problems of traditional interconnect interfaces but also provided a solid foundation for the rapid development of AI, HPC, and cloud computing. As computing demands continue to grow, NVLink is bound to play an even more crucial role in the future, offering infinite possibilities for technological advancement.