One trend in the data centre that has shown consistent progress is the convergence of switching fabrics. Servers in data centres are typically connected to three different network areas: Storage, TCP/IP and Cluster networking. Due to these areas having different requirements, different switching fabrics are typically used: Fibre Channel, Ethernet and InfiniBand.
An important aspect of Ethernet’s value proposition is the possibility of using it in all network areas to provide servers with a single connection. Consolidating to an Ethernet fabric greatly reduces the amount of equipment required in the data centre and drastically lowers both power consumption and the overall cost of administration.
IP-based Storage solutions
IP-based storage connectivity encapsulates storage messages into an IP packet and uses the underlying Ethernet technology to transport this between servers and storage devices. Obviously, higher Ethernet speeds mean the information shared between the servers and the storage devices is exchanged even faster. One can safely assume that 10 Gigabit Ethernet is ten times faster than Gigabit Ethernet.
File access protocols such as NFS and CIFs typically always run over Ethernet today and these protocols are already seeing performance improvements with 10G Ethernet. For customers who use file access for their storage, Ethernet is already the ideal converged fabric.
One of the main ways to encapsulate block storage information over IP packets is by using the iSCSI protocol. In a recent Gartner report , Robert E. Passmore provides a detailed review of iSCSI and concludes that the technology is ready to be deployed with servers in data centres. He estimates that iSCSI deployments can offer cost savings up to $3,500 per server versus connecting these same servers with Fibre Channel.
An analyst from the Forrester Group, Andrew Reichman, further confirmed this trend. In his report he concludes with significant hard and soft costs saving of using iSCSI versus Fibre Channel. By reviewing the work of these two analysts, it is clear that iSCSI offers a great value proposition for deployment in the data centre.
The IDC Group reported a 61.6% revenue growth year-of-year in 2008 for iSCSI technology. Analyst Natalya Yezhkova concluded the difficult economic situation and price sensitive customers contributed to iSCSI-increased growth. In the same report, Liz Connor, Research Analyst, Disk Storage System Tracker notes that Fibre Channel SANs saw a downturn, falling 3.2% year over year.
“FC SAN systems fulfill many high-end storage needs, but usually at a higher average price. However, iSCSI and NAS storage solution alternatives offer increased enterprise-level features at lower costs, and compel vendors to consider these technologies. Continued end user education, growing confidence in IP-based storage, increasing product sophistication, as well as a typically lower price point, result in increased adoption of iSCSI and NAS by many budget conscious end users.”
IP-based storage solutions are easy to configure and administer and are using the same networking infrastructure (Ethernet), by reducing the data centre complexity and cost of operation. Most IP-based storage deployments currently use Gigabit Ethernet and will benefit from the higher speed of 10 Gigabit Ethernet to reach performance levels beyond that of Fibre Channel.
Data Centre Bridging
Ethernet was previously a best-effort network that may drop packets or deliver packets out of order when the network is busy, resulting in retransmissions and time-outs. The iSCSI protocol allows for retransmissions and time-outs since iSCSI leverages the IP standards. However, reducing possible network losses improves iSCSI performance. New standards have created a new, more capable Loss-Less family of Ethernet protocols. These standards are referred to collectively as Data Centre Bridging (DCB).
- Priority Flow Control (IEEE 802.1Qbb)
- Congestion notification – (IEEE 802.1Qau)
- Enhanced Transmission Selection (802.1Qaz)
- Data Centre Bridging eXchange protocol (DCBX)
Loss-Less Ethernet enables a 10 Gigabit Ethernet connection to support multiple traffic types simultaneously, preserving the respective traffic properties. With these extensions, the same Ethernet link will enhance iSCSI performance and robustness as well enabling Fibre Channel storage traffic with FCOE by offering a no-drop capable consolidated I/O fabric. With the introduction of products that support Data Centre Bridging, iSCSI (and NFS) will benefit from improved performance and reliability from a loss-less Ethernet fabric.
Fibre Channel over Ethernet (FCoE)
It was mentioned earlier that storage area networks are either built using IP-based (using Ethernet) or Fibre Channel Solutions. The storage industry is now experiencing a strong support for a new technology called Fibre Channel over Ethernet or FCoE. FCoE is described by many as a combination of the best from both worlds. It offers investment protection and application backward compatibility for Fibre Channel environments while using the ubiquitous, widely deployed and cost effective Ethernet technology for transportation in lieu of a Fibre Channel switching fabric.
This is accomplished by encapsulating Fibre Channel messages into Ethernet frames and using the same Ethernet infrastructure, present in every data centre, to transport the information back and forth from the servers and storage devices. FCoE requires a loss-less DCB Ethernet infrastructure as the switching fabric. Bob Laliberte, analyst at the ESG research firm, describes FCoE as one of the drivers accelerating the industry acceptance of 10 Gigabit Ethernet as the main switching fabric in data centres and predicts significant FCoE roll out between 2010-2012 . FCoE is expected to offer improved performances over Fibre Channel based solutions.
The Ethernet switching fabric is undergoing a serious transformation: enhancing its capability to transport mission-critical information such as storage. Already, storage networks are increasingly moving away from Fibre Channel to Ethernet-based storage solutions using 10 Gigabit Ethernet, and this trend is accelerating now with Loss-Less DCB Ethernet products.
High performance computing (HPC) applications are sensitive to performance and price. With the DCB enhancements and the proposed RDMA over Converged Ethernet (RoCE) standard, Ethernet will also be able to provide the low latency IPC connectivity required for HPC clusters. With the adoption of the new DataCentre Bridging (DCB) Ethernet protocols, Ethernet will offer all the right technical features to become the only switching fabric needed in the data center for networking, storage and clustering.