40 Gigabit Ethernet long-haul optics

The demonstration leveraged an existing 1650-km production-quality SURFnet link, connecting an experimental high-performance computer cluster equipped with a Mellanox ConnectX-2 EN 40GbE NIC at the University of Amsterdam to a remote data processing unit with a corresponding interface at the GLIF meeting venue. The demonstration pushed 26 Gbps (the practical limit of the PCIe bus) from the processor in Amsterdam to the processor at CERN through a single optical wavelength. The network infrastructure was based on Ciena’s Optical Multiservice Edge (OME) 6500 equipped with 40GbE interfaces. Ciena says the OME 6500 was seamlessly upgraded from a 10-Gbps optical wavelength to a 40-Gbps wavelength with no added signal regeneration or modifications to the existing infrastructure.

Ciena did not reveal whether the interface complied to the specifications expected to be included in the upcoming IEEE 802.3bg standard.

“We are excited that we are able to push the envelope further to 40-Gbps long-haul transmission for research and education,” said Erik-Jan Bos, chief technology officer at SURFnet. “This 40G demo showed that we now can do single-stream end-to-end transport well above today's common boundary of 10 Gbps.”

“The novelty of this work is the new unobstructed 40-Gbps single-channel bandwidth between compute nodes implemented directly on a lambda network. This marks the next step in the growth of long-haul communication capacity for distributed data processing,” said Cees de Laat, professor in system and network engineering at the University of Amsterdam. “These capacities are essential not only for data-intensive e-science but also, for example, in high-resolution 3D digital cinema and movie processing.”

The demonstration included a variety of Internet transport protocols optimized for the network, as well as a highly parallel model checker (DiVinE) that was optimized for distributed execution by the group of Professor Henri Bal at the Vrije Universiteit Amsterdam.