Fibre Channel

Fibre Channel
Layer 4. Protocol mapping
LUN masking
Layer 3. Common services
Layer 2. Network
Fibre Channel fabric Fibre Channel zoning Registered State Change Notification
Layer 1. Data link
Fibre Channel 8B/10B encoding
Layer 0. Physical

Fibre Channel, or FC, is a gigabit-speed network technology primarily used for storage networking.^[1]^[2] Fibre Channel is standardized in the T11 Technical Committee of the InterNational Committee for Information Technology Standards (INCITS), an American National Standards Institute (ANSI)–accredited standards committee. Fibre Channel was primarily used in the supercomputer field, but now, has become the standard connection type for storage area networks (SAN) in enterprise storage. Despite its name, Fibre Channel signaling can run on both twisted pair copper wire and fiber-optic cables.^[1]^[2]

Fibre Channel Protocol (FCP) is a transport protocol (similar to TCP used in IP networks) which predominantly transports SCSI commands over Fibre Channel networks.^[1]^[2]

1 History
2 Fibre Channel topologies
3 Layers
4 Ports
5 Optical carrier medium variants
6 Fibre Channel infrastructure
7 Fibre Channel Host Bus Adapters
8 Development tools
9 See also
10 References
- 10.1 INCITS Fibre Channel standards
11 Sources
12 Further reading
13 External links

History

Fibre Channel started in 1988, with ANSI standard approval in 1994, as a way to simplify the HIPPI system then in use for similar roles. HIPPI used a massive 50-pair cable with bulky connectors, and had limited cable lengths. When Fibre Channel started to compete for the mass storage market its primary competitor was IBM's proprietary Serial Storage Architecture (SSA) interface. Eventually the market chose Fibre Channel over SSA, rather than give IBM control over the next generation of mid- to high-end storage technology. Fibre Channel was primarily concerned with simplifying the connections and increasing distances, as opposed to increasing speeds. Later, designers added the goals of connecting SCSI disk storage, providing higher speeds and far greater numbers of connected devices.

It also added support for any number of "upper layer" protocols, including ATM, and IP, with SCSI being the predominant usage.

The following table shows Fibre Channel speed variants:^[3]

Fibre Channel Variants
NAME	Line-Rate (GBaud)	Throughput (Fullduplex) (MBps)*	Availability
1GFC	1.0625	200	1997
2GFC	2.125	400	2001
4GFC	4.25	800	2005
8GFC	8.5	1600	2008
10GFC Serial	10.52	2550	2004
10GFC Parallel	12.75
16GFC	14.025	3200	2011
20GFC	21.04	5100	20??

* - Throughput for duplex connections

Fibre Channel topologies

There are three major Fibre Channel topologies, describing how a number of ports are connected together. A port in Fibre Channel terminology is any entity that actively communicates over the network, not necessarily a hardware port. This port is usually implemented in a device such as disk storage, an HBA on a server or a Fibre Channel switch.^[1]

Point-to-Point (FC-P2P). Two devices are connected directly to each other. This is the simplest topology, with limited connectivity.^[1]

Arbitrated loop (FC-AL). In this design, all devices are in a loop or ring, similar to token ring networking. Adding or removing a device from the loop causes all activity on the loop to be interrupted. The failure of one device causes a break in the ring. Fibre Channel hubs exist to connect multiple devices together and may bypass failed ports. A loop may also be made by cabling each port to the next in a ring.
- A minimal loop containing only two ports, while appearing to be similar to FC-P2P, differs considerably in terms of the protocol.
- Only one pair of ports can communicate concurrently on a loop.
- Maximum speed of 8GFC.

Switched fabric (FC-SW). All devices or loops of devices are connected to Fibre Channel switches, similar conceptually to modern Ethernet implementations. Advantages of this topology over FC-P2P or FC-AL include:
- The switches manage the state of the fabric, providing optimized interconnections.
- The traffic between two ports flows through the switches only, it is not transmitted to any other port.
- Failure of a port is isolated and should not affect operation of other ports.
- Multiple pairs of ports may communicate simultaneously in a fabric.

Attribute	Point-to-Point	Arbitrated loop	Switched fabric
Max ports	2	127	~16777216 (2²⁴)
Address size	N/A	8-bit ALPA	24-bit port ID
Side effect of port failure	Link fails	Loop fails (until port bypassed)	N/A
Mixing different link rates	No	No	Yes
Frame delivery	In order	In order	Not guaranteed
Access to medium	Dedicated	Arbitrated	Dedicated

Layers

Fibre Channel does not follow the OSI model layering, but is split similarly into 5 layers, namely:

FC4 — Protocol Mapping layer, in which application protocols, such as SCSI or IP, are encapsulated into a PDU for delivery to FC2.
FC3 — Common Services layer, a thin layer that could eventually implement functions like encryption or RAID redundancy algorithms;
FC2 — Network layer, defined by the FC-PI-2 standard, consists of the core of Fibre Channel, and defines the main protocols;
FC1 — Data Link layer, which implements line coding of signals;
FC0 — PHY, includes cabling, connectors etc.;

Layers FC0 through FC2 are also known as FC-PH, the physical layers of Fibre Channel.

Fibre Channel routers operate up to FC4 level (i.e. they may operate as SCSI routers), switches up to FC2, and hubs on FC0 only.

Fibre Channel products are available at 1, 2, 4, 8, 10, 16 and 20 Gbit/s; these protocol flavors are called accordingly 1GFC, 2GFC, 4GFC, 8GFC, 10GFC, 16GFC, or 20GFC. The 16GFC standard was approved by the INCITS T11 committee in 2010, and those products are expected to become available in 2011. Products based on the 1GFC, 2GFC, 4GFC, 8GFC and 16GFC standards should be interoperable and backward compatible. The 1GFC, 2GFC, 4GFC, 8GFC designs all use 8b/10b encoding, while the 16GFC standard uses 64b/66b encoding. Unlike the 10GFC and 20GFC standards, 16GFC provides backward compatibility with 4GFC and 8GFC.

The 10 Gbit/s standard and its 20 Gbit/s derivative, however, are not backward compatible with any of the slower speed devices, as they differ considerably on FC1 level in using 64b/66b encoding instead of 8b/10b encoding, and are primarily used as inter-switch links.

Ports

The following types of ports are defined by Fibre Channel:

node ports
- N_port is a port on the node (e.g. host or storage device) used with both FC-P2P or FC-SW topologies. Also known as Node port.
- NL_port is a port on the node used with an FC-AL topology. Also known as Node Loop port.
- F_port is a port on the switch that connects to a node point-to-point (i.e. connects to an N_port). Also known as Fabric port. An F_port is not loop capable.
- FL_port is a port on the switch that connects to a FC-AL loop (i.e. to NL_ports). Also known as Fabric Loop port.
- E_port is the connection between two fibre channel switches. Also known as an Expansion port. When E_ports between two switches form a link, that link is referred to as an inter-switch link (ISL).
- D_port is a diagnostic port, used solely for the purpose of running link-level diagnostics between two switches and to isolate link level fault on the port, in the SFP, or in the cable.
- EX_port is the connection between a fibre channel router and a fibre channel switch. On the side of the switch it looks like a normal E_port, but on the side of the router it is a EX_port.
- TE_port * a Cisco addition to Fibre Channel, now adopted as a standard. It is an extended ISL or EISL. The TE_port provides not only standard E_port functions but allows for routing of multiple VSANs (Virtual SANs). This is accomplished by modifying the standard Fibre Channel frame (vsan tagging) upon ingress/egress of the VSAN environment. Also known as Trunking E_port.
- VE_Port an INCITS T11 addition, FCIP interconnected E-Port/ISL, i.e. fabrics will merge.
- VEX_Port a INCITS T11 addition, is a FCIP interconnected EX-Port, routing needed via lsan zoning to connect initiator to a target.

general (catch-all) types
- Auto or auto-sensing port found in Cisco switches, can automatically become an E_, TE_, F_, or FL_port as needed.
- Fx_port a generic port that can become a F_port (when connected to a N_port) or a FL_port (when connected to a NL_port). Found only on Cisco devices where oversubscription is a factor.
- GL_port on a switch can operate as an E_port, FL_port, or F_port. Found on QLogic switches.
- G_port or generic port on a switch can operate as an E_port or F_port. Found on Brocade, McData, and QLogic switches.
- L_port is the loose term used for any arbitrated loop port, NL_port or FL_port. Also known as Loop port.
- U_port is the loose term used for any arbitrated port. Also known as Universal port. Found only on Brocade switches.....

(*Note: The term "trunking" is not a standard Fibre Channel term and is used by vendors interchangeably. For example: A trunk (an aggregation of ISLs) in a Brocade device is referred to as a Port Channel by Cisco. Whereas Cisco refers to trunking as an EISL.)

Optical carrier medium variants

Fiber modality	Speed (MByte/s)	Transmitter^[4]	Medium variant	Distance
Single-Mode Fiber	1600	1310 nm longwave light^{[ITS 1]}	1600-SM-LC-L^{[ITS 2]}	0.5 m - 10 km
	1600	1490 nm longwave light^{[ITS 1]}	1600-SM-LZ-I^{[ITS 2]}	0.5 m - 2 km
	800	1310 nm longwave light^{[ITS 3]}	800-SM-LC-L^{[ITS 4]}	2 m - 10 km
	800	1310 nm longwave light^{[ITS 3]}	800-SM-LC-I^{[ITS 4]}	2 m - 1.4 km
	400	1310 nm longwave light^{[ITS 3]}^{[ITS 5]}	400-SM-LC-L^{[ITS 6]}	2 m - 10 km
			400-SM-LC-M^{[ITS 4]}	2 m - 4 km
			400-SM-LL-I^{[ITS 7]}	2 m - 2 km
	200	1550 nm longwave light^{[ITS 8]}	200-SM-LL-V^{[ITS 8]}	2 m - 50 km
		1310 nm longwave light^{[ITS 5]}^{[ITS 3]}	200-SM-LC-L^{[ITS 6]}	2 m - 10 km
		1310 nm longwave light^{[ITS 5]}^{[ITS 3]}	200-SM-LL-I^{[ITS 7]}	2 m - 2 km
	100	1550 nm longwave light^{[ITS 8]}	100-SM-LL-V^{[ITS 8]}	2 m - 50 km
		1310 nm longwave light^{[ITS 9]}^{[ITS 3]}	100-SM-LL-L^{[ITS 10]} 100-SM-LC-L^{[ITS 6]}	2 m - 10 km
		1310 nm longwave light^{[ITS 9]}^{[ITS 3]}	100-SM-LL-I^{[ITS 10]}	2 m - 2 km
Multimode Fiber	1600	850 nm shortwave light^{[ITS 11]}^{[ITS 12]}^{[ITS 13]}	1600-M5F-SN-I^{[ITS 14]}	0.5 m - 125 m
			1600-M5E-SN-I^{[ITS 14]}	0.5 m - 100 m
			1600-M5-SN-S^{[ITS 14]}	0.5 m - 35 m
			1600-M6-SN-S^{[ITS 15]}	0.5 m - 15 m
	800		800-M5F-SN-I^{[ITS 14]}	0.5 m - 190 m
			800-M5E-SN-I^{[ITS 16]}	0.5 m - 150 m
			800-M5-SN-S^{[ITS 16]}	0.5 m - 50 m
			800-M6-SN-S^{[ITS 16]}	0.5 m - 21 m
	400		400-M5F-SN-I^{[ITS 14]}	0.5 m - 400 m
			400-M5E-SN-I^{[ITS 16]}	0.5 m - 380 m
			400-M5-SN-I^{[ITS 17]}	0.5 m - 150 m
			400-M6-SN-I^{[ITS 17]}	0.5 m - 70 m
	200		200-M5E-SN-I^{[ITS 16]}	0.5 m - 500 m
			200-M5-SN-I^{[ITS 17]}	0.5 m - 300 m
			200-M6-SN-I^{[ITS 17]}	0.5 m - 150 m
	100		100-M5E-SN-I^{[ITS 18]}	0.5 m - 860 m
			100-M5-SN-I^{[ITS 19]}	0.5 m - 500 m
			100-M6-SN-I^{[ITS 20]}	0.5 m - 300 m
			100-M5-SL-I^{[ITS 20]}	2 m - 500 m
			100-M6-SL-I^{[ITS 21]}	2 m - 175 m

Multimode fiber	Fiber Diameter	FC media designation
OM1	62.5 µm	M6
OM2	50 µm	M5
OM3	50 µm	M5E
OM4	50 µm	M5F

Modern Fibre Channel devices support SFP transceiver, mainly with LC fiber connector. Older 1GFC devices used GBIC transceiver, mainly with SC fiber connector.

Fibre Channel infrastructure

Fibre Channel switches can be divided into two classes. These classes are not part of the standard, and the classification of every switch is a marketing decision of the manufacturer:

directors offer a high port-count in a modular (slot-based) chassis with no single point of failure (high availability).

switches are typically smaller, fixed-configuration (sometimes semi-modular), less redundant devices.

A fabric consisting entirely of one vendor is considered to be homogeneous. This is often referred to as operating in its "native mode" and allows the vendor to add proprietary features which may not be compliant with the Fibre Channel standard.

If multiple switch vendors are used within the same fabric it is heterogeneous, the switches may only achieve adjacency if all switches are placed into their interoperability modes. This is called the "open fabric" mode as each vendor's switch may have to disable its proprietary features to comply with the Fibre Channel standard.

Some switch manufacturers offer a variety of interoperability modes above and beyond the "native" and "open fabric" states. These "native interoperability" modes allow switches to operate in the native mode of another vendor and still maintain some of the proprietary behaviors of both. However, running in native interoperability mode may still disable some proprietary features and can produce fabrics of questionable stability.

Fibre Channel Host Bus Adapters

Fibre Channel HBAs are available for all major open systems, computer architectures, and buses, including PCI and SBus. Some are OS dependent. Each HBA has a unique World Wide Name (WWN), which is similar to an Ethernet MAC address in that it uses an Organizationally Unique Identifier (OUI) assigned by the IEEE. However, WWNs are longer (8 bytes). There are two types of WWNs on a HBA; a node WWN (WWNN), which can be shared by some or all ports of a device, and a port WWN (WWPN), which is necessarily unique to each port.

Development tools

When developing and/or troubleshooting the Fibre Channel bus, examination of hardware signals can be very important to find problems. Logic analyzers and bus analyzers are tools which collect, analyze, decode, store signals so people can view the high-speed waveforms at their leisure.

References

^ ^a ^b ^c ^d ^e Preston, W. Curtis (2002). "Fibre Channel Architecture". Using SANs and NAS. Sebastopol, CA: O'Reilly Media. pp. 19–39. ISBN 9780596001537. OCLC 472853124.
^ ^a ^b ^c Riabov, Vladmir V. (2004). "Storage Area Networks (SANs)". In Bidgoli, Hossein. The Internet Encyclopedia. Volume 3, P-Z. Hoboken, NJ: John Wiley & Sons. pp. 329–338. ISBN 9780471689973. OCLC 55610291.
^ "Roadmaps". Fibre Channel Industry Association. http://www.fibrechannel.org/roadmaps. Retrieved 2009-11-26.
^ Transmitter values listed are the currently specified values for the variant listed. Some older versions of the FC standards listed slightly different values (however, the values listed here fall within the +/- variance allowed). Individual variations for each specification are listed in the references associated with those entries in this table. FC-PH = X3T11 Project 755D; FC-PH-2 = X3T11 Project 901D; FC-PI-4 = INCITS Project 1647-D; FC-PI-5 = INCITS Project 2118D. Copies are availabe from INCITS.

INCITS Fibre Channel standards

^ ^a ^b FC-PI-5 Clause 6.3
^ ^a ^b FC-PI-5 Clause 8.1
^ ^a ^b ^c ^d FC-PI-4 Clause 6.3
^ ^a ^b ^c FC-PI-4 Clause 8.1
^ ^a ^b FC-PH-2 lists 1300nm (see clause 6.1 and 8.1)
^ ^a ^b ^c FC-PI clause 8.1
^ ^a ^b FC-PH-2 clause 8.1
^ ^a ^b ^c ^d FC-PI-4 Clause 11
^ FC-PH lists 1300nm (see clause 6.1 and 8.1)
^ ^a ^b FC-PH Clause 8.1
^ FC-PI-5 Clause 6.4
^ FC-PI-4 Clause 6.4
^ The older FC-PH and FC-PH-2 list 850nm (for 62.5µm cables) and 780nm (for 50µm cables)(see clause 6.2, 8.2, and 8.3)
^ ^a ^b ^c ^d ^e FC-PI-5 Clause 8.2
^ FC-PI-5 Annex A
^ ^a ^b ^c ^d ^e FC-PI-4 Clause 8.2
^ ^a ^b ^c ^d FC-PI Clause 8.2
^ PC-PI-4 Clause 8.2
^ PC-PI Clause 8.2
^ ^a ^b PC-PI Clause 8.2
^ FC-PH Annex C and Annex E

Sources

Clark, T. Designing Storage Area Networks, Addison-Wesley, 1999. ISBN 0-201-61584-3

External links

Computer bus official and de facto standards (wired)

General	System bus Front-side bus Back-side bus Daisy chain Control bus Address bus Bus contention Plug and play List of bus bandwidths

Standards	S-100 bus Unibus VAXBI MBus STD Bus SMBus Q-Bus ISA Zorro II Zorro III CAMAC FASTBUS LPC HP Precision Bus EISA VME VXI VXS NuBus TURBOchannel MCA SBus VLB PCI PXI HP GSC bus CoreConnect InfiniBand UPA PCI-X AGP PCI Express Intel QuickPath Interconnect HyperTransport

Portable	PC Card ExpressCard

Embedded	Multidrop bus AMBA Wishbone

Storage	ST-506 ESDI SMD Parallel ATA (PATA) DMA SSA HIPPI USB MSC FireWire (1394) Serial ATA (SATA) eSATA eSATAp SCSI Parallel SCSI Serial Attached SCSI (SAS) Fibre Channel (FC) iSCSI ATAoE

Peripheral	Apple Desktop Bus HIL MIDI Multibus RS-232 (serial port) DMX512-A IEEE-488 (GPIB) EIA/RS-422 IEEE-1284 (parallel port) UNI/O ACCESS.bus 1-Wire I²C SPI EIA/RS-485 Parallel SCSI Profibus USB FireWire (1394) Fibre Channel Camera Link External PCI Express x16 Thunderbolt

Note: interfaces are listed in speed ascending order (roughly), the interface at the end of each section should be the fastest Category