|
RAIDZ Levels
There are many different ways to organize data in a RAIDZ array.
These ways are called "RAIDZ levels".
Different RAIDZ levels have different speed and fault tolerance properties.
Striped pools are not fault tolerant. RAIDZ1, RAIDZ2 and RAIDZ3 are fault tolerant to a different degree
- should one of the hard drives in the array fail, the data is still reconstructed on the fly and no access interruption occurs.
There are some more complex layouts, which are combinations of basic RAIDZ levels.
Although you may use traditional names for such nested layout like RAID50 or RAID60,
"Z" names are not introduced for them and often the layouts are just named after the schemes used for their
creation like mirror + stripe or RAIDZ + mirror. These are however beyond the scope of this reference.
|
RAIDZ levels comparison chart
  |
Stripe |
Mirror |
RAIDZ |
RAIDZ2 |
RAIDZ3 |
Stripe+mirror |
Min number of disks |
1 |
2 |
2 |
4 |
5 |
4 |
Fault tolerance |
None |
(N-1) disk |
1 disk |
2 disks |
3 disk |
(N-1) disk in each N-disk mirror |
Disk space overhead |
None |
(N-1)/N |
1 disk |
2 disks |
3 disks |
(N-1)*P for P stipe over N-disk mirrors |
Read speed |
Fast |
Fast |
Slow, see below |
Fast |
Write speed |
Fast |
Fair |
Slow, See below |
Fair |
Hardware cost |
Cheap |
High to highest |
High |
Very high |
Very High (disks) |
High to highest |
|
Striping and blocks
Striping is a technique to store data on the disk array.
The contigous stream of data is divided into blocks, and blocks are written to multiple disks in a specific pattern.
Striping is used with all RAIDZ levels due to the specifics of ZFS filesystem which is used on the RAIDZ pools.
Block size is dynamically selected for each data row to be written to the ZFS pool.
|
Stipe pool (RAID0)
Stripe pool and RAID0 are almost equal to each other.
Although ZFS provides checksumming to avoid silent data corruption,
there is no redundant data (no parity nor a mirror) to rebuild data from in case of a disk failure.
Another difference is that ZFS stores metadata records in several copies regardless of the top-level RAIDZ layout.
However, user data is still stored in a single copy on a striped ZFS pool.
While the configuration is not recommended to be used solely,
it still can be considered as a part of redundant layouts like RAID10 (stripe plus mirror).
Like RAID0, a striped ZFS pool provides no fault tolerance at all.
Should any of the disks in the array fail, the entire pool fails and all the data is lost.
Striped ZFS solutions are cheap, and they use all the disk capacity minus disk space used for metadata redundancy which is the must
on the ZFS pool regardless of the layout.
If a striped ZFS pool fails logically but all the disks are present and healthy, you can do a
ZFS stripe pool recovery
relatively easy using ReclaiMe Pro ZFS recovery software.
However, you should keep in mind that if the disk failure happens, data is lost irreversibly.
Disk 1 |
Disk 2 |
Disk 3 |
1 |
5 |
3 |
6 |
2 |
7 |
4 |
8 |
9 |
|
Mirror pool (RAID1)
Requires minimum of two drives and generally is very similar to a RAID1.
Using in conjunction with the ZFS filesystem gives you certain advantages like automatic checksumming.
Although checksumming just allows detecting data corruption, it is still something, which most traditional RAID systems lack.
Another difference from a regular RAID1 is that with the help of ZFS you can create storage systems
involving multiple mirrors rather than just 2 or 3-way mirrors within the traditional RAID1 technology.
While a traditional RAID1 is recommended to use when you need a reliable storage of relatively small capacity,
a mirrored ZFS pool is recommended when you need an extremly reliable storage system.
A regular RAID1 stores two identical copies of data on two hard drives and doesn't use blocks and stripes.
A mirrored ZFS pools can store as many copies of data as you wish and does use blocks and stripes: copies are stored within
particular "rows".
ZFS mirror pool uses the capacity of (N-1) drives to maintain fault tolearnce. This amounts to (N-1) \ N capacity loss for the array.
For example, if you combine four 500GB drives in a ZFS mirror, you'd only get 500GB of usable disk space and 1.5 TB goes for redundancy.
Disk 1 |
Disk 2 |
1 |
2 |
2 |
1 |
3 |
3 |
|
RAIDZ (RAID5)
RAIDZ fits as large, reliable, relatively cheap storage.
RAIDZ (sometimes called RAIDZ1 pointing to a single parity) is most similar to a traditional RAID5 and requires minimum of 2 disks.
Like in a regular RAID5, every "row" along with the data blocks stores a parity function calculated
over the data blocks, which allows a system to survive a single disk failure.
The difference is in the row placement pattern which is no longer the same over the disk set;
another difference is block size which can vary over the rows.
Also, there are no longer distinguished right/left, synchronous/asynchronous RAID5 layouts in ZFS RAIDz pools.
If all drives are OK, read requests are distributed evenly across the drives, providing read speed similar to that of a striped pool.
Theoretically, for N-disk array, a striped pool provides N times faster reads and RAIDZ should provide
(N-1) times faster reads. However, the number of blocks in a particular row depends on the data block size to be written
as well as on the ZFS unit block size so we can only talk about (N-1) times read speed acceleration as the top limit.
If one of the drives has failed, the read speed degrades to that of a single drive,
because all blocks in a row are required to serve the request.
Write speed of a RAIDZ is limited by the parity updates.
For each written block, its corresponding parity block has to be read, updated, and then written back.
Thus, there is no significant write speed improvement on RAIDZ, if any at all.
The capacity of one member drive is used to maintain fault tolerance.
For example, if you have 10 drives 1TB each, the resulting RAIDZ capacity would be 9TB.
If a ZFS RAIDZ pool fails, you can still recover data from the pool with
RAIDZ recovery software.
Unlike a striped ZFS pool, RAIDZ is redundant and it can survive one member disk failure.
Disk 1 |
Disk 2 |
Disk 3 |
Disk 4 |
P1,2 |
1 |
2 |
|
P3,4,5 |
3 |
4 |
5 |
6 |
P6 |
|
|
7 |
P7,8 |
8 |
|
|
RAIDZ2 (RAID6)
RAIDZ2 is a large, highly reliable, relatively expensive storage, which requires a minimum of 3 disks.
RAIDZ2 is like RAID6 – allows to survive a double disk failure at the expense of storing two different parity functions.
All considerations about block placement pattern and block size are the same as for a RAIDZ.
Read speed of the N-disk RAIDZ2 is up to (N-2) times faster
(apply the same considerations as for RAIDZ above) than the speed of a single drive, similar to RAIDZ.
If one or two drives fail in RAIDZ2, the read speed degrades significantly because a reconstruction of missing blocks
requires an entire row to be read.
There is no significant write speed improvement in RAIDZ2 layout.
RAIDZ2 parity updates require even more processing than that in RAIDZ.
The capacity of two member drives is used to maintain fault tolerance.
For an array of 10 drives 1TB each, the resulting RAIDZ2 capacity would be 8TB.
The recovery of a RAIDZ2 pool in case of failure is fairly complicated but anyway you need
ZFS recovery software for this.
Disk 1 |
Disk 2 |
Disk 3 |
Disk 4 |
Disk 5 |
P1,2 |
Q1,2 |
1 |
2 |
|
P3,4,5 |
Q3,4,5 |
3 |
4 |
5 |
|
|
P7 |
Q7 |
7 |
P8,9 |
Q8,9 |
8 |
9 |
|
|
RAIDZ3
Triple parity RAID. Although back in 2009
people discussed that triple-parity RAID should replace
the traditional RAID5/6 and even proposed a name – RAID7,
only in the ZFS environment the triple-parity RAID idea has been developed and implemented in full.
With RAIDZ3, you can create a storage system, which allows to tolerate 3 disk failures before losing data.
To create a RAIDZ3 pool you need at least 4 disks.
Disk 1 |
Disk 2 |
Disk 3 |
Disk 4 |
Disk 5 |
Disk 6 |
P11,2,3 |
P21,2,3 |
P31,2,3 |
1 |
2 |
3 |
P14,5 |
P24,5 |
P34,5 |
4 |
5 |
|
P16,7,8,9 |
P26,7,8,9 |
P36,7,8,9 |
6 |
7 |
8 |
9 |
P110,11 |
P210,11 |
P310,11 |
10 |
11 |
P112 |
P212 |
P312 |
12 |
|
|
|
Striped mirror pool (RAID10)
Striped mirror pool is a large, fast, reliable, but expensive storage.
Striped mirror pool uses several (minumum two) mirrored disk sets to hold identical copies of the content.
The layout is much similar to RAID10 and traditionally requires minimum of four devices.
Assuming that mirrors in ZFS environment can be created on any number of disks (starting from 2 disks),
you can easily create a bunch of almost any possible mirrors (as to children disk number)
and then stripe data across them.
Disk 1 |
Disk 2 |
Disk 3 |
Disk 4 |
1 |
3 |
4 |
6 |
5 |
5 |
2 |
2 |
7 |
1 |
6 |
4 |
3 |
7 |
8 |
8 |
|
Credits
This RAIDz calculator was created by ReclaiMe Team of www.ReclaiMe-Pro.com.
Check out our other stuff if you are interested in
|