FCoE, what is it?

In my last post (Converged Infrastructure) I mentioned FCoE as one of the elements used to bring convergence to the data center. In this post I decided to explain a little as to what FCoE is and why it has a role in bringing convergence to the data center.

What is FCoE?

FCoE is Fibre Channel Over Ethernet - this is the encapsulating of the fibre Channel frames inside ethernet frames so fibre channel traffic and LAN traffic is sent over the same cables instead of over separate fibre and LAN cabling. Although this technically can be done over 1GigE network, vendors are only providing 10GigE devices. This means a certain amount of disruption (more on this below) to the existing network architecture.

There is also a requirement for what is called DCB - Data Center Bridging as Ethernet is ‘lossy’ (packets will get lost/retransmitted), whilst fibre is ’lossless’ (no packets dropped) as you don’t want to lose/drop data being transmitted to your SAN!!.

In order for FC to run over Ethernet we need to ensure that losslessness of fibre is retained when run over ethernet. So various standards are presently being worked on by the Data Center Bridging group in order to enable a low latency and lossless ethernet network, that allows FCoE frames to be transmitted on the same bits of wire as LAN traffic.

Why is FCoE Useful?

Consider a typical data center as depicted in the diagram below



What FCoE gives you is convergence at the adapter level (CNA - Converged Network Adapter) within the server (a single card for both SAN/LAN connectivity), so a reduction in HBA’s which then requires less power utilised by the server. In addition a reduction in switches as SAN and LAN switches are no longer separate. So the future converged infrastructure will look something like this (not the best diagram, it is simply to show the reduction in switches/cables).



There is potential for a 50% reduction in the data center for switches/cables and less power (Green I.T) required due to the reduction both in terms of physical equipment and the amount of power used in servers (with fewer cards).

The convergence of the physical infrastructure takes many shapes (some mentioned here) including for example server virtualisation and/or storage virtualisation. FCoE is another (complementary) method, the other methods might bring just as much benefit, but together they can bring greater convergence to the data center.

The FCoE standard was adopted in June 2009 and details can be found here, here and here, this last entry is the pdf of standard. There is still work to be done around DCB however and that is mentioned in the second entry just provided.

Many of the leading companies (for example Cisco) have FCoE at the center of their Data Center strategy and so it is critical for them that FCoE is adopted widely. It is the means by which Cisco see the convergence of the data center and it is a core part of their Unified Computing System (UCS).

We mentioned earlier that FCoE/DCB will have some disruption on the data center due to the requirement for 10GigE (it may be that your core network is presently not 10GigE as an example) and DCB standards being adopted meaning new hardware - CNA’s and FCoE switches to take advantage of FCoE/DCB. That is why the full power of FCoE/DCB will be gradual (rather that what some have termed) rip-and-replace strategy. This will start at the Access layer (server edge) and over time move through the Aggregation layer into the Network Core.

If you look at Cisco’s UCS blade infrastructure that is an example of how this can be achieved NOW - in the blade chassis FCoE is run over 10GigE ports (on Fabric Extenders) that connect to Fabric Interconnect (6120 or 6140) allowing Fibre and IP traffic to run over the same cable in the chassis upto the access switch (Fabric Interconnect).

The Fabric Interconnect has 10GigE ports and can also take a module that has Fibre Channel ports. The fibre channel ports then uplink to the normal SAN switches using fibre cables, whilst the Ethernet ports uplink to the Core Switches using lan cables. So you have 10GigE FCoE at the server (chassis) edge over the same cable that branches to separate legacy cables as you approach the network core

I was on the Data Center Of The Future event in which Cisco presented their view on the subject. This included a Q&A session, during the session I asked the question as to whether FCoE was ready and the response was "yes". So then I adjusted my question and asked if I could use it from the server to the core and then I was told that "some standards still needed to be ratified for that to be achieved, but FCoE had been ratified". So there still needs to be some work done before FCoE becomes all embracing from the server edge to the network core in the Data Center.

Nigel Poulton has a really good series of posts (deep dives) on FCoE here if you want to know more. Also Dave Convery has a very good post here on FCoE. There is also a very good post here that details the savings from deploying FCoE for a hospital including space and power.