SAN Fundamentals: Fabrics, Zoning, and LUN Masking

A storage area network presents raw block devices to servers over a dedicated network. The host sees what looks like a local disk, formats it with its own filesystem, and never knows the blocks live in an array elsewhere. That single distinction, block instead of file, drives almost every design decision that follows.

Block storage is not shared storage

Because the host owns the filesystem, two hosts writing to the same block device without a cluster aware filesystem will corrupt it. NAS protocols like NFS and SMB arbitrate access on the server side, so concurrent clients are safe by design. A SAN does not do that for you. If more than one host must see the same LUN, you need either a clustered filesystem or a layer above that coordinates ownership.

Choosing a transport

Fibre Channel runs block traffic over a purpose built lossless fabric with its own switches and host bus adapters. It is expensive and it is predictable, which is why it persists in environments that cannot tolerate variance.

iSCSI carries SCSI commands over ordinary TCP/IP. The hardware is cheap and familiar, but the network stops being someone else’s problem. Put storage traffic on isolated VLANs or physically separate switching, and be deliberate about flow control and frame size. Sharing a congested general purpose network with storage is a frequent cause of unexplained latency complaints.

NVMe over Fabrics carries the NVMe command set across a network instead of SCSI. It removes translation overhead that made sense for spinning disks and makes little sense for flash. It can run over Fibre Channel, RDMA capable Ethernet, or plain TCP, and the fabric choice determines how much of the benefit you actually see.

Picking between the three is a decision about operational cost and failure behaviour more than about bandwidth. Fibre Channel vs iSCSI vs NVMe-oF sets them side by side on latency, cost and who ends up owning the incident. If iSCSI is already the answer, iSCSI SAN setup: targets, initiators, multipath is the build order.

Names you have to get right

Every SAN control in this article keys off a name, so the naming scheme is not clerical detail. It is the identifier that zoning, masking and audit logs all match on, and a typo in it is indistinguishable from a permissions failure.

Fibre Channel identifies endpoints by World Wide Name. Each adapter carries a node name and each of its ports carries a port name, both 64-bit values burned in by the manufacturer and written as eight colon-separated hex bytes. Because they are assigned by the vendor rather than by the site, two ports never collide, but they also carry no meaning: nothing in a WWPN tells you which server it belongs to. Keep an external map from WWPN to host and slot, and update it when an adapter is replaced, because a swapped HBA changes the name that every zone references.

iSCSI takes the opposite approach and lets the site mint its own names. RFC 7143 defines the iSCSI Qualified Name as iqn. followed by a year-month date code, then the naming authority’s domain in reverse order, then an optional colon-separated suffix. The date is the point at which the naming authority held the domain, not the date the LUN was created, which is the detail most often got wrong. The suffix is free text, and it is the only part that carries operational meaning, so spend the effort there: a suffix naming the host and its role survives a rebuild in a way that a serial number does not.

NVMe over Fabrics uses NVMe Qualified Names built on the same reverse-domain pattern, and presents namespaces rather than LUNs. The vocabulary differs but the control model does not: a namespace is still a block device that exactly one host should own unless something above it coordinates ownership.

How a host actually finds a LUN

Discovery is a separate step from access, and conflating the two is behind a lot of “the LUN is not there” tickets.

On a Fibre Channel fabric, a host port logs into the switch, registers itself with the fabric name server, and then asks that name server what else it is allowed to see. Zoning is enforced at that query, so an unzoned initiator gets an empty answer rather than an error. That is why a freshly cabled host with a working link light and no visible targets is almost always a zoning problem rather than a cabling one.

On iSCSI, the initiator either has target addresses configured statically or performs a discovery session against a portal that returns the target list. Either way, discovery only tells the host that a target exists. A normal login still has to succeed afterwards, and that login is where CHAP authentication and the array’s access controls apply.

NVMe over Fabrics formalises the same split with a dedicated discovery controller that returns the log page of subsystems a host may connect to. In all three cases the sequence is the same: find out what exists, then be permitted to use it, then be permitted to see specific volumes on it. The three checks live in three different places, and troubleshooting goes faster when you know which one you are testing.

Zoning and masking are different controls

Zoning happens in the fabric. It decides which initiators and targets can see each other at all, and it is what stops an unrelated host from discovering an array port. Prefer single initiator zones, each containing one host port and the target ports it needs. Large zones full of initiators create noise and make change control harder than it needs to be.

LUN masking happens on the array. It decides which specific volumes a permitted initiator is allowed to use. You want both. Zoning alone still exposes every masked volume on that target to anyone in the zone, and masking alone leaves the fabric wide open.

The failure that hurts is presenting a LUN already owned by another host. The new host sees an unformatted disk, an administrator formats it, and the original data is gone. Verify ownership before presenting anything.

Zoning itself comes in two flavours, and the difference matters when someone re-patches a rack. Soft zoning filters what the fabric name server is willing to tell an initiator, so it is enforced at discovery and follows the WWPN wherever it is plugged in. Hard zoning is enforced in the switch’s forwarding path by port, so it survives a host that ignores the name server but breaks the moment a cable moves to a different port. Modern fabrics generally combine the two, filtering name server responses by WWPN while also policing frames in hardware. Choose WWPN-based zone members if you expect cables to be re-dressed, and port-based members if you need the physical position itself to be part of the policy, but do not mix the two schemes inside a single zone set: a half-migrated zone set is much harder to reason about during an incident than either scheme on its own.

Keep the zone database under the same change control as the array. A fabric configuration is a live security control, not switch scratch space, and an exported copy of the active zone set taken before every change is the fastest route back when a change goes wrong.

Multipathing is mandatory, not optional

Every host should reach every LUN through more than one path: two HBAs or NICs, two fabrics, two array controllers. Multipath software on the host collapses those duplicate device nodes into one and handles failover. Skipping it means a single cable, optic or switch reboot takes an application down.

Two things go wrong here. First, multipathing is configured but never tested, so nobody discovers the path failover settings are wrong until an outage. Second, the paths are not genuinely independent, because both run through the same switch or the same physical route. Draw the topology and confirm no single device appears on every path.

A third failure is quieter than either: the paths all work, but the path grouping sends production traffic down the array’s non-optimised route, so everything is permanently slower and nothing ever errors. That check and the rest of the triage order are in SAN latency troubleshooting.

That quiet failure has a name. Most dual-controller arrays are asymmetric: a given volume is owned by one controller at a time, and reaching it through the other controller means the request is forwarded internally before it is served. The SCSI command set standardised by the T10 committee lets the array advertise which paths are the optimised ones, and multipath software is supposed to group paths accordingly and prefer the optimised group. When that advertisement is ignored, either because the host is using a generic device profile or because someone pinned a path group by hand, every read takes the long way through the array’s interconnect. Nothing errors, no alert fires, and the only symptom is latency that has always been slightly worse than the hardware should deliver.

Three settings decide how a path failure feels to the application, and all three are worth setting deliberately rather than inheriting. The path grouping policy decides whether traffic spreads across an entire group or pins to one path until it dies. The retry behaviour decides what happens when every path is gone: fail the I/O immediately and let a cluster manager react, or queue indefinitely and hope the fabric comes back, which keeps the data safe but can hang a host until it does. The failback behaviour decides whether a recovered path is used again automatically, which is usually what you want, or held back until an operator confirms the path is genuinely stable, which is what you want when a marginal optic is flapping. The device-mapper multipath documentation covers the per-array defaults, and a device with no matching entry falls back to generic behaviour that is safe but rarely optimal.

Test the failover rather than the configuration. Pulling one optic at a time while a synthetic write load runs, and confirming both that the paths drop out and that they come back into the right group, is the only check that distinguishes multipathing that works from multipathing that is merely configured.

Capacity planning

Provision for the workload profile, not just the total size. Thin provisioning is useful and it is also a way to oversubscribe an array into an outage, so monitor actual pool consumption rather than allocated capacity. Watch queue depth and latency together, since throughput numbers alone hide the moment a fabric starts queuing.

For a first pass at the numbers, the SAN LUN and NVMe-oF sizer converts raw capacity, RAID level, port speed and initiator count into usable capacity and aggregate fabric throughput, which is enough to check a proposed port count before it becomes a purchase order.

The order these controls should be built in

The four concepts above are not independent, and building them out of order is how a SAN acquires the failure modes it keeps for the rest of its life.

Transport first, because it fixes the hardware budget and decides who owns an incident: a Fibre Channel fabric is a separate estate with its own switches and skills, while iSCSI hands storage traffic to the network team whether or not they were told. Naming second, before any zone exists, because renaming a host after fifty zones reference its WWPN is a change window nobody wants. Zoning third, then masking, in that order, so that at no point is a volume reachable by a host that has not yet been given explicit permission to use it. Multipathing last, and tested, because a path layout designed after the fabric is built usually inherits a single shared switch that nobody notices until it reboots.

The single highest-value habit is drawing the topology before touching a switch, then confirming that no device appears on every path from any host to any volume. That drawing catches the shared-switch problem, the shared-optic problem and the single-controller problem in one pass, and it takes minutes. Recovering from any of the three after the array is in production takes a maintenance window and a conversation about downtime.

Everything past this point is depth on one of these four decisions. The transport comparison lives in Fibre Channel vs iSCSI vs NVMe-oF, the build order for the most common transport in iSCSI SAN setup: targets, initiators, multipath, and the diagnostic order for when the fabric is up but slow in SAN latency troubleshooting.