NetApp FAS

This article has multiple issues. Please help or discuss these issues on the talk page. (Learn how and when to remove these messages) This article needs additional citations for verification. Please help by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "NetApp FAS" – news · newspapers · books · scholar · JSTOR (May 2019) (Learn how and when to remove this message) This article may contain an excessive amount of intricate detail that may interest only a particular audience. Please help by spinning off or relocating any relevant information, and removing excessive detail that may be against Wik.ipedia.Pro's inclusion policy. (May 2019) (Learn how and when to remove this message) This article relies largely or entirely on a single source. Relevant discussion may be found on the talk page. Please help improve this article by introducing citations to additional sources. Find sources: "NetApp FAS" – news · newspapers · books · scholar · JSTOR (June 2011) (Learn how and when to remove this message)

A NetApp FAS is a computer storage product by NetApp running the ONTAP operating system; the terms ONTAP, AFF, ASA, FAS are often used as synonyms. "Filer" is also used as a synonym although this is not an official name. There are three types of FAS systems: Hybrid, All-Flash, and All SAN Array:

1. NetApp proprietary custom-build hardware appliances with HDD or SSD drives called hybrid Fabric-Attached Storage (or simply FAS)^[1]
2. NetApp proprietary custom-build hardware appliances with only SSD drives and optimized ONTAP for low latency called ALL-Flash FAS (or simply AFF)
3. All SAN Array build on top of AFF platform, and provide only SAN-based data protocol connectivity.

ONTAP can serve storage over a network using file-based protocols such as NFS and SMB, also block-based protocols, such as the SCSI over the Fibre Channel Protocol on a Fibre Channel network, Fibre Channel over Ethernet (FCoE), iSCSI, and FC-NVMe transport layer. ONTAP-based systems that can serve both SAN and NAS protocols called Unified ONTAP, AFF systems with ASA identity called All-SAN.

NetApp storage systems running ONTAP implement their physical storage in large disk arrays.

While most large-storage systems are implemented with commodity computers with an operating system such as Microsoft Windows Server, VxWorks or tuned Linux, ONTAP-based hardware appliances use highly customized hardware and the proprietary Data ONTAP operating system with WAFL file system, all originally designed by NetApp founders David Hitz and James Lau specifically for storage-serving purposes. ONTAP is NetApp's internal operating system, specially optimized for storage functions at high and low levels. It boots from FreeBSD as a stand-alone kernel-space module and uses some functions of FreeBSD (command interpreter and drivers stack, for example).

All NetApp ONTAP-based hardware appliances have battery-backed non-volatile random access memory or NVDIMM, referred to as NVRAM or NVDIMM,^{[citation needed]} which allows them to commit writes to stable storage more quickly than traditional systems with only volatile memory. Early storage systems connected to external disk enclosures via parallel SCSI, while modern models (as of 2009^[update]) use fibre channel and SAS (Serial Attach SCSI) SCSI transport protocols. The disk enclosures (shelves) use fibre channel hard disk drives, as well as parallel ATA, serial ATA and Serial attached SCSI. Starting with AFF A800 NVRAM PCI card no longer used for NVLOGs, it was replaced with NVDIMM memory directly connected to the memory bus.

Implementers often organize two storage systems in a high-availability cluster with a private high-speed link, either Fibre Channel, InfiniBand, 10 Gigabit Ethernet, 40 Gigabit Ethernet or 100 Gigabit Ethernet. One can additionally group such clusters together under a single namespace when running in the "cluster mode" of the Data ONTAP 8 operating system.

This section does not cite any sources. Please help by adding citations to reliable sources. Unsourced material may be challenged and removed. (November 2017) (Learn how and when to remove this message)

Modern NetApp FAS, AFF or ASA system consist of customized computers with Intel processors using PCI. Each FAS, AFF or ASA system has non-volatile random access memory, called NVRAM, in the form of a proprietary PCI NVRAM adapter or NVDIMM-based memory, to log all writes for performance and to play the data log forward in the event of an unplanned shutdown. One can link two storage systems together as a cluster, which NetApp (as of 2009) refers to using the less ambiguous term "Active/Active".

Each storage system model comes with a set configuration of processor, RAM, and non-volatile memory, which users cannot expand after purchase. With the exception of some of the entry point storage controllers, the NetApp FAS, ASA, and AFF systems usually have at least one PCIe-based slot available for additional network, tape and/or disk connections. In June 2008 NetApp announced the Performance Acceleration Module (or PAM) to optimize the performance of workloads that carry out intensive random reads. This optional card goes into a PCIe slot and provides additional memory (or cache) between the disk and the storage system cache and system memory, thus improving performance.

All-Flash FAS, also known as AFF A-series. Usually, AFF systems based on the same hardware as FAS but first one optimized and works only with SSD drives on the back end while the second one can use HDD and SSD as a cache: for example, AFF A700 & FAS9000, A300 & FAS8200, A200 & FAS2600, A220 & FAS2700 use the same hardware, but AFF systems do not include Flash Cache cards. Also, AFF systems do not support FlexArray with third-party storage array virtualization functionality. AFF is a Unified system and can provide SAN & NAS data protocol connectivity, and in addition to traditional SAN & NAS protocols in FAS systems, AFF has block-based NVMe/FC protocol for systems with 32 Gbit/s FC ports. AFF & FAS use the same firmware image, and nearly all noticeable functionality for the end-user is the same for both storage systems. However, internally data is processed and handled differently in ONTAP. AFF systems, for example, use different Write Allocation algorithms as compared to FAS systems. Because AFF systems have faster underlying SSD drives, Inline data deduplication in ONTAP systems is nearly not noticeable (≈2% performance impact on low-end systems).^[2]

All SAN Array running ONTAP, and based on AFF platform thus inherits its features & functionalities, and data internally processed and handled the same as in AFF systems. All other ONTAP-based hardware and software platforms can be referred to as Unified ONTAP meaning they can provide unified access with SAN & NAS data protocols. ONTAP architecture in ASA systems is the same as in FAS & AFF, with no changes. ASA systems using the same firmware image as AFF & FAS systems. ASA is the same as AFF, and the only difference is in the access to the storage over the network with SAN protocols: ASA provides symmetric active/active access to the block devices (LUN or NVMe namespaces), while Unified ONTAP systems continue to use ALUA and ANA for the block protocols. The original ASA platform was first announced in 2019 and re-launched in May 2023.^[3]

In February 2023, NetApp introduced a new AFF product line code named the C-Series. This platform uses quad-level cell (QLC) NAND flash and is aimed at competing with the QLC-based products already in the market from Pure Storage (specifically FlashArray//C).^[4] The C-Series has higher latency than typical AFF systems, at around 2-4 milliseconds compared to 500 microseconds on AFF using triple-level cell (TLC) media. However, the aim of the platform is to provide a lower price point for customers that might otherwise not choose all-flash systems.

NetApp uses either SATA, Fibre Channel, SAS or SSD disk drives, which it groups into RAID (Redundant Array of Inexpensive Disks or Redundant Array of Independent Disks) groups of up to 28 (26 data disks plus 2 parity disks). NetApp FAS storage systems which contain only SSD drives with installed SSD-optimized ONTAP OS called All-Flash FAS (AFF).

FAS, ASA, and AFF systems are using enterprise-level HDD and SSD (i.e. NVMe SSD) drives with two ports, each port connected to each controller in an HA pair. HDD and SSD drives can only be bought from NetApp and installed in NetApp's Disk Shelves for FAS/AFF platform. Some shelves, such as the D4246, can be upgraded from a 6 Gbit/s Shelf into a 12 Gbit/s with an IOM-12 upgrade.^[5] Physical HDD and SSD drives, partitions on them, and LUNs imported from third-party arrays with FlexArray functionality considered in ONTAP as a Disk. In SDS systems like ONTAP Select & ONTAP Cloud, logical block storage like virtual disk or RDM inside ONTAP also considered as a Disk. Do not confuse the general term "disk drive" and "disk drive term used in ONTAP system" because, with ONTAP, it could be an entire physical HDD or SSD drive, a LUN, or a partition on a physical HDD or SSD drive. LUNs imported from third-party arrays with FlexArray functionality in HA pair configuration must be accessible from both nodes of the HA pair. Each disk has ownership on it to show which controller owns and serves the disk. An aggregate can include only disks owned by a single node, therefore each aggregate owned by a node and any objects on top of it, as FlexVol volumes, LUNs, File Shares are served with a single controller. Each controller can have its own disks and aggregates them where both nodes can be utilized simultaneously even though they not serving the same data.

Advanced Drive Partitioning (ADP) can be used in ONTAP-based systems depending on the platform and use-case. ADP can be used only with native disk drives from NetApp Disk shelves, FlexArray technology does not support ADP. ADP also supported with third-party drives in ONTAP Select. This technique mainly used to overcome some architectural requirements and reduce the number of disk drives in ONTAP-based systems. There are three types of ADP: Root-Data partitioning; Root-Data-Data partitioning (RD2 also known as ADPv2); Storage Pool. Root-Data partitioning can be used in FAS & AFF systems to create small root partitions on drives to use them to create system root aggregates and, therefore, not to spend entire three disk drives for that purpose. In contrast, the bigger portion of the disk drive will be used for data aggregate. Root-Data-Data partitioning is used in AFF systems only for the same reason as Root-Data partitioning with the only difference that bigger portion of the drive left after root partitioning divided equally by two additional partitions, usually, each partition assigned to one of the two controllers, therefore reducing the minimum number of drives required for an AFF system and reducing waste for expensive SSD space. Storage Pool partitioning technology used in FAS systems to equally divide each SSD drive by four pieces which later can be used for FlashPool cache acceleration, with Storage Pool only a few SSD drives can be divided by up to 4 data aggregates which will benefit from FlashCache caching technology reducing minimally required SSD drives for that technology.

In NetApp ONTAP systems, RAID and WAFL are tightly integrated. There are several RAID types available within ONTAP-based systems:

• RAID 4 with 1 dedicated parity disk allowing any 1 drive to fail in a RAID group.
• RAID-DP with 2 dedicated parity disks allowing any 2 drives to fail simultaneously in a RAID group.^[6]
• RAID-TEC US patent 7640484  with 3 dedicated parity drives, allows any 3 drives to fail simultaneously in a RAID group.^[7]

RAID-DP's double parity leads to a disk loss resiliency similar to that of RAID 6. NetApp overcomes the write performance penalty of traditional RAID-4 style dedicated parity disks via WAFL and a novel use of its nonvolatile memory (NVRAM) within each storage system.^[8] Each aggregate consist of one or two plexes, a plex consists of one or more RAID groups. Typical ONTAP-based storage system have only 1 plex in each aggregate, two plexes used in local SyncMirror or MetroCluster configurations. Each RAID group usually consists of disk drives of same type, speed, geometry and capacity. Though NetApp Support could allow a user to install a drive to a RAID group with same or bigger size and different type, speed and geometry for temporary basis. Ordinary data aggregates if containing more than one RAID group must have same RAID groups across the aggregate, same RAID group size is recommended, but NetApp allows to have exception in last RAID group and configure it as small as half of the RAID group size across aggregate. For example, such an aggregate might consists of 3 RAID groups: RG0:16+2, RG1:16+2, RG2:7+2. Within aggregates, ONTAP sets up flexible volumes (FlexVol) to store data that users can access.

Aggregates enabled as FlshPool and with both HDD and SSD drives called hybrid aggregates. In Flash Pool hybrid aggregates same rules applied to the hybrid aggregate as to ordinary aggregates but separately to HDD and SSD drives, thus it is allowed to have two different RAID types: only one RAID type for all HDD drives and only one RAID type for all SSD drives in a single hybrid aggregate. For example, SAS HDD with RAID-TEC (RG0:18+3, RG1:18+3) and SSD with RAID-DP (RG3:6+2). NetApp storage systems running ONTAP combine underlying RAID groups similarly to RAID 0. Also, in NetApp FAS systems with FlexArray feature, the third-party LUNs could be combined in a Plex similarly to RAID 0. NetApp storage systems running ONTAP can be deployed in MetroCluster and SyncMirror configurations, which are using technique comparably to RAID 1 with mirroring data between two plexes in an aggregate.

NetApp Flash Pool is a feature on hybrid NetApp FAS systems that allows creating hybrid aggregate with HDD drives and SSD drives in a single data aggregate. Both HDD and SSD drives form separate RAID groups. Since SSD also used to write operations, it requires RAID redundancy contrary to Flash Cache but allows to use of different RAID types for HDD and SSD; for example, it is possible to have 20 HDD 8 TB in RAID-TEC while 4 SSD in RAID-DP 960 GB in a single aggregate. SSD RAID used as cache and improved performance for read-write operations for FlexVol volumes on the aggregate where SSD added as the cache. Flash Pool cache similarly to Flash Cache has policies for reading operations but also includes write operations that could apply separately for each FlexVol volume located on the aggregate; therefore, it could be disabled on some volumes while others could benefit from SSD cache. Both FlashCache & FlashPool can be used simultaneously to cache data from a single FlexVol to enable an aggregate with Flash Pool technology minimum 4 SSD disks required (2 data, 1 parity, and 1 hot spare), it is also possible to use ADP technology to partition SSD into 4 pieces (Storage Pool) and distribute those pieces between two controllers so each controller will benefit from SSD cache when there is a small amount of SSD. Flash Pool is not available with FlexArray and is possible only with NetApp FAS native disk drives in NetApp's disk shelves.

FlexArray is NetApp FAS functionality allows to visualize third party storage systems and other NetApp storage systems over SAN protocols and use them instead of NetApp's disk shelves. With FlexArray functionality RAID protection must be done with third party storage array thus NetApp's RAID 4, RAID-DP and RAID-TEC not used in such configurations. One or many LUNs from third party arrays could be added to a single aggregate similarly to RAID 0. FlexArray is licensed feature.

NetApp Storage Encryption (NSE) is using specialized purpose build disks with low level Hardware-based full disk encryption (FDE/SED) feature and also supports FIPS-certified self-encrypted drives, compatible nearly with all NetApp ONTAP features and protocols but does not offer MetroCluster. NSE feature does overall nearly zero performance impact on the storage system. NSE feature similarly to NetApp Volume Encryption (NVE) in storage systems running ONTAP can store encryption key locally in Onboard Key Manager or on dedicated key manager systems using KMIP protocol like IBM Security Key Lifecycle Manager and SafeNet KeySecure. NSE is data at rest encryption which means it protects only from physical disks theft and does not give an additional level of data security protection in a normal operational and running system. NetApp has passed NIST Cryptographic Module Validation Program for its NetApp CryptoMod (TPM) with ONTAP 9.2.^[9]

MetroCluster (MC) is free functionality for FAS and AFF systems for metro high availability with synchronous replication between two sites, this configuration requires additional equipment. Available in both modes: 7-mode (old OS) and Cluster-Mode (or cDOT - a newer version of ONTAP OS). MetroCluster in Cluster-Mode known as MCC. MetroCluster uses RAID SyncMirror (RSM) and plex technique where on one site number of disks form one or more RAID groups aggregated in a plex, while on the second site have the same number of disks with the same type and RAID configuration along with Configuration Replication Service (CRS) and NVLog replication. One plex synchronously replicates to another in a compound with non-volatile memory. Two plexes form an aggregate where data stored and in case of disaster on one site second site provide read-write access to data. MetroCluster Support FlexArray technology. MetroCluster configurations are possible only with mid-range and high-end models which provide the ability to install additional network cards required to MC to function.

With MetroCluster it is possible to have one or more storage node per site to form a cluster or Clustered MetroCluster (MCC). Remote and local HA perter node must be the same model. MCC consists of two clusters each located on one of two sites. There may be only two sites. In MCC configuration each one remote and one local storage node form Metro HA or Disaster Recovery Pare (DR Pare) across two sites while two local nodes (if there is partner) form local HA pare, thus each node synchronously replicates data in non-volatile memory two nodes: one remote and one local (if there is one). It is possible to utilize only one storage node on each site (two single node clusters) configured as MCC. 8 node MCC consists of two clusters - 4 nodes each (2 HA pair), each storage node have only one remote partner and only one local HA partner, in such a configuration each site clusters can consist out of two different storage node models. For small distances, MetroCluster requires at least one FC-VI or newer iWARP card per node. FAS and AFF systems with ONTAP software versions 9.2 and older utilize FC-VI cards and for long distances require 4 dedicated Fibre Channel switches (2 on each site) and 2 FC-SAS bridges per each disk shelf stack, thus minimum 4 total for 2 sites and minimum 2 dark fiber ISL links with optional DWDMs for long distances. Data volumes, LUNs and LIFs could online migrate across storage nodes in the cluster only within a single site where data originated from: it is not possible to migrate individual volumes, LUNs or LIFs using cluster capabilities across sites unless MetroCluster switchover operation is used which disable entire half of the cluster on a site and transparently to its clients and applications switch access to all of the data to another site.

Starting with ONTAP 9.3 MetroCluster over IP (MCC-IP) was introduced with no need for a dedicated back-end Fibre Channel switches, FC-SAS bridges and dedicated dark fiber ISL which previously were needed for a MetroCluster configuration. Initially, only A700 & FAS9000 systems supported with MCC-IP. MCC-IP available only in 4-node configurations: 2-node Highly Available system on each site with two sites total. With ONTAP 9.4, MCC-IP supports A800 system and Advanced Drive Partitioning in form of Rood-Data-Data (RD2) partitioning, also known as ADPv2. ADPv2 supported only on all-flash systems. MCC-IP configurations support single disk shelf where SSD drives partitioned in ADPv2. MetroCluster over IP require Ethernet cluster switches with installed ISL and utilize iWARP cards in each storage controller for synchronous replication. Starting with ONTAP 9.5 MCC-IP supports distance up to 700 km and starts to support SVM-DR feature, AFF A300, and FAS8200 systems.

NetApp storage systems using proprietary OS called ONTAP (Previously Data ONTAP). The main purpose for an Operating System in a storage system is to serve data to clients in non-disruptive manner with the data protocols those clients require, and to provide additional value through features like High Availability, Disaster Recovery and data Backup. ONTAP OS provides enterprise level data management features like FlexClone, SnapMirror, SnapLock, MetroCluster etc., most of them snapshot-based WAFL File System capabilities.

WAFL, as a robust versioning filesystem in NetApp's proprietary OS ONTAP, it provides snapshots, which allow end-users to see earlier versions of files in the file system. Snapshots appear in a hidden directory: ~snapshot for Windows (SMB) or .snapshot for Unix (NFS). Up to 1024 snapshots can be made of any traditional or flexible volume. Snapshots are read-only, although ONTAP provides additional ability to make writable "virtual clones", based at "WAFL snapshots" technique, as "FlexClones".

ONTAP implements snapshots by tracking changes to disk-blocks between snapshot operations. It can set up snapshots in seconds because it only needs to take a copy of the root inode in the filesystem. This differs from the snapshots provided by some other storage vendors in which every block of storage has to be copied, which can take many hours.

Each NetApp FAS system running Data ONTAP 8 could switch between modes either 7-Mode or Cluster mode. In reality each mode was a separate OS with its own version of WAFL, both 7-mode and Cluster mode where shipped on a single firmware image for a FAS system till 8.3 where 7-mode was deprecated. SnapLock migration from 7-Mode to ONTAP 9 now supported with Transition Tool. It is possible to switch between modes on a FAS system but all the data on disks must be destroyed first since WAFL is not compatible and server-based application called 7MTT tool was introduced to migrate data from old 7-mode FAS system to new Cluster-Mode:

• With SnapMirror based replication called Copy-based transition which helped to migrate all the data with planned downtime using only storage vendor capabilities. Copy-based transition require new controllers and disks with space no less than on source system if all the data to be migrated. Both SAN and NAS data are possible.
• Starting with 7-mode 8.2.1 and Cluster-Mode 8.3.2 WAFL compatibility where introduced and new feature in 7MTT tool called Copy-free transition to replace old controllers running 7-mode with new controllers running Cluster-Mode and planned downtime, while new system require additional system disks with root aggregates for new controllers (it could be as less as 6 disks). Since with Copy-free transition no data copying required 7MTT tool helping only for new controllers reconfiguration. Both SAN and NAS data conversion supported.

Additional to 7MTT there are two other paths to migrate data based on protocol type:

• SAN data could be copied with foreign LUN import (FLI) functionality integrated in NetApp FAS system which can copy data over SAN protocol while new storage system running ONTAP placed as SAN proxy between hosts and old storage system which require host reconfiguration and minimum downtime. FLI available as for old 7-mode systems and for some models of storage systems of competitors.
• NAS data could be copied with NetApp XCP free host-based utility thus host-based copy process processed with the utility from any copying data from source server with SMB or NFS protocols to ONTAP system with minimal downtime for client systems reconfiguration for new NAS server.

Before the release of ONTAP 8, individual aggregate sizes were limited to a maximum of 2 TB for FAS250 models and 16 TB for all other models.

The limitation on aggregate size, coupled with increasing density of disk drives, served to limit the performance of the overall system. NetApp, like most storage vendors, increases overall system performance by parallelizing disk writes to many different spindles (disk drives). Large capacity drives, therefore limit the number of spindles that can be added to a single aggregate, and therefore limit the aggregate performance.

Each aggregate also incurs a storage capacity overhead of approximately 7-11%, depending on the disk type. On systems with many aggregates, this can result in lost storage capacity.

However, the overhead comes about due to additional block-checksumming on the disk level as well as usual file system overhead, similar to the overhead in file systems like NTFS or EXT3. Block checksumming helps to ensure that data errors at the disk drive level do not result in data loss.

Data ONTAP 8.0 uses a new 64-bit aggregate format, which increases the size limit of FlexVolume to approximately 100 TB (depending on storage platform) and also increases the size limit of aggregates to more than 100 TB on newer models (depending on storage platform) thus restoring the ability to configure large spindle counts to increase performance and storage efficiency.^[10]

AI Performance testings (image distortion disabled):

This list may omit some models. Information taken from spec.org, netapp.com and storageperformance.org

EOA = End of Availability

SPECsfs with "*" is clustered result. SPECsfs performed include SPECsfs93, SPECsfs97, SPECsfs97_R1 and SPECsfs2008. Results of different benchmark versions are not comparable.

• Network attached storage
• NetApp
• ONTAP Operation System, used in NetApp storage systems
• Write Anywhere File Layout (WAFL), used in NetApp storage systems

• SnapLock Technical Report