ZFS on FreeBSD

Overview and iSCSI Example

What is ZFS
128bit filesystem
Combined volume management and filesystem
Released by Sun Microsystems to Replace aging UFS in
2006
Can store 256 quadrillion(256x1015) zettabytes (1
zettabyte = 1 billion terabytes)
256,000,000,000,000,000,000,000,000,000,000,000,000 (256x1036)
bytes (give or take a few terabytes)

Uses Pooled Storage Model

Open Source GPL and CDDL

ZFS Features

Data Integrity
Copy on Write transaction model
Raid Support (Striped, Mirror, and RaidZ/Raid5) in Pools/Volumes
Quick and Easy Snapshots
Data Compression and Deduplication
Block level full and differential replication
Robust Cache Architecture
Easy administration with command line, only two commands zpool and
zfs
Almost eliminates /etc/fstab
One the fly setting modifications

ZFS Limitations
ZFS is only fully supported on Solaris, OpenSolaris,

Open Indiana, and FreeBSD

64bit OS recommended
2GB of Ram per 1TB of disk space (Used mostly for
cache).
Poor performance on slow CPUs

Data Integrity
Every data block that is written has a checksum in

metadata (raid volumes have a checksum for each

redundant block)
Data verified against a checksum on every read
raid1 and up volumes have automatic error correction,
all volumes have automatic error detection
Data Integrity can find hardware issues in controllers,
disks, and memory
Data integrity can be disabled, not recommended.

Copy on Write (COW) Transaction Model

Any time data is written a new block is created instead
of reusing existing blocks
COW allows for extremely fast and efficient snapshots
(zero performance penalty unlike LVM2)
Old blocks are eventually reused if they have no
references in the current file system or and snapshot

ZFS Architecture

ZFS is a uses pooled storage model

pools are not mounted directly instead datasets
are created in a pool and they are then mounted
to access the filesystem
mounting a zfs is handled by the zfs daemon not
the UNIX mount command. Mount points are
defined in the dataset not /etc/fstab

Raid on ZFS (Pools)

Supports spaned, mirrored, and raid5 pools
(zpool create)
Supports easily replacing disks in pool
(zpool replace)
Supports cache devices
Supports easily adding disks to a pool (zpool
attach or zpool add)

ZFS (Zpool) Volume Management

Commands

zpool create pool device

create a new pool
zpool add pool device
add a device to a existing pool
zpool remove pool device
remove a device from a pool
zpool replace pool device new_device
replace a faulty device in a pool
zpool destroy pool
delete a pool from the system
zpool status
get the status of all available pools

More information is available in the zpool man page.

Snapshots
command syntax:

zfs snapshot Volume@snapshot

Snapshots are differential

Snapshots are created very quickly around 2 sec or less
Snapshots supported on volumes and sub volumes
previous snapshot data is accessible via the .zfs
directory on each volume

Replication/Backup
command syntax:

zfs send volume@snapshot

zfs receive volume@snapshot

zfs can only send snapshots

zfs send and receive allows to send a snapshot of

filesystem to anything (disk, usb storage, tape, or

network service)
using ssh or netcat zfs can be replicated to a remote
machine
Differential replication is possible if the receive end
has a previous snapshot

Compression and Deduplication

command syntax:

zfs set dedup=on|off volume

zfs set compression=on|off volume

Deduplication uses a checksum or verify to find
redundant data to consolidate. Really useful for data
that doesn't change much (Disk Images and Backups).
Compression uses gzip or lzjb to compress data
Both Deduplication and Compression both happen at
the file system level so the operating system sees the
uncompress or undeduped file

FreeBSD ZFS Installer

The install ISOs at http://www.freebsd.org do not setup ZFS by default

ZFS installer ISOs are available at http://mfsbsd.vx.sk
ZFS installer will use entire disk
Installation steps:
1. Boot from CD
2. login as root/mfsroot
3. run destroygeom (optional)
4. mount CD on /cdrom
5. run zfsinstall script
6. chroot to /tank/root and set root password, hostname and
IP
7. Reboot

Exporting ZFS using Samba

Advantages:

easy to administer, exports are based on directories,

Windows clients can easily connect, user based
authentication, Design for multiple connections per export

Disadvantages: I/O throughput relatively low, really only useful for using
export as a file share

Samba is available through the samba35 port

Config file is in /usr/local/etc/smb.conf
Init script is /usr/local/etc/rc.d/samba

Exporting ZFS using NFS

Advantages:

Disadvantages:

easy to administer, fair I/O throughput,Managed export

config, exports are based on directories not files

I/O throughput not as high as iSCSI over ethernet,

access is difficult for not Linux/Unix clients

ZFS uses the the distribution (system) NFS server in FreeBSD

nfs_server_enable and rpcbind_enable must be set to YES in /etc/rc.conf
zfs datasets are exported via the zfs dataset property sharenfs
zfs exports are stored in /etc/zfs/exports (this is a managed file do not
edit)
to share the dataset tank/vol1you would issue the command zfs set
sharenfs=on tank/vol1
nfs options such as alldirs, maproot, network, mask can be set using a zfs
set sharenfs=<option> dataset command

Exporting ZFS using iSCSI over Ethernet

Advantages:

Great I/O throughput, Works great for Virtual Machines,

Easy to connect nonUnix clients

Disadvantages:

Difficult to administer, Deamon reload required after each

config change, Initial setup can be trick without examples

iSCSI is provided using the istgt port (third party package)

istgt has two config files auth.conf and istgt.conf. They are both located in
/usr/local/etc/istgt/
istgt can export volumized datasets or raw diskimages (diskimages are easier)
each diskimage or dataset are exported as a Logical Unit in /usr/local/etc/istgt.
conf
The Windows iSCSI client does not like " " or "_" in Logical Unit entries, otherwise
istgt will accept them.
init script located in /usr/local/etc/rc.d/istgt. Set istgt_enable to YES in /etc/rc.
conf to start at boot.

Istgt Auth.conf Example

[AuthGroup1]
Comment "Unit Controller Users"
Auth

"user2" "secret123456"

[AuthGroup2]
Comment "Initiator Users"
Auth

"user2" "secret123456"

Istgt istgt.conf example

[PortalGroup1]

[Global]
Comment

"Global section"

Comment

"Portal Group 1"

NodeBase

"zfs1.sebeka.k12.mn.us"

Portal

DA1 10.1.0.95:3260

PidFile

/var/run/istgt.pid

AuthFile

/usr/local/etc/istgt/auth.conf

MediaDirectory

/var/istgt

Comment

"Initiator Group 1"

LogFacility

"local7"

InitiatorName

"ALL"

Timeout

30

Netmask

10.1.0.0/23

NopInInterval

20

DiscoveryAuthMethod

Auto

MaxSessions

64

MaxConnections

16

FirstBurstLength

65536

MaxBurstLength

262144

MaxRecvDataSegmentLength 262144

[UnitControl]

Comment

"Unit Controller"

AuthMethod

CHAP

AuthGroup

AuthGroup1

Portal

UC1 127.0.0.1:3261

Netmask

127.0.0.1

[InitiatorGroup1]

[LogicalUnit1]
TargetName
Mapping
AuthMethod
AuthGroup
UnitType
UnitInquiry
LUN0

sambaserversys
PortalGroup1 InitiatorGroup1
Auto
AuthGroup2
Disk
"FreeBSD" "iSCSI Disk" "0123" "10000001"
Storage /vol1/images/sambaserver_sys.raw auto

Leveraged Uses of ZFS

Shared Hypervisor Storgage (Dedupe,
Snapshots, and Rreprication)
Backup Storage (Dedupe/Compression,
Snapshots)
Software Installation Rollbacks (Snapshots)
Compressed Logging volumes (Compression)
Samba/NFS Server (Snapshots,Compression)

ZFS Online Resources

FreeBSD Wiki Page for ZFS

http://wiki.freebsd.org/ZFS

FreeBSD Handbook Chapter on ZFS

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/filesystems-zfs.html

Official ZFS Admin Guide

http://docs.oracle.com/cd/E19253-01/819-5461/