ZFS

What is zfs

Wikipedia: "ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include data integrity verification against data corruption modes, support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. ZFS is implemented as open-source software, licensed under the Common Development and Distribution License (CDDL)."

Typical uses

Operating systems

Solaris
OpenSolaris and derivatives
Mac OS/X
*BSD
Linux (FUSE)
Linux (Native)
Linux (Arm devices)
Windows - read only

Simple to use

$ zpool create pool_name device_name

$ zfs create pool_name/fdion

$ zfs snapshot pool_name/fdion@cloudcamp

TO THE cloud

$ zfs send pool_name/fdion@cloudcamp > /s3mount

Replace s3mount with dropbox folder etc.

For incremental, use -i

$ zfs send -i ...

Or use send on one server, piped to ssh or mbuffer, then receive on the cloud with zfs receive

but what about encryption?

PIPING

Compression is native on all ZFS, some OS have more choices. There is also a native encryption under Solaris . Under other OSes, it varies. Worst care, compile your own or just use an external tool as part of the process:

$ zfs send pool_name/fdion@cloudcamp | openssl enc -aes-256-cbc -salt -pass file:mypasswordfile > /mountpoint

In the cloud

Through an instance running an operating system with ZFS

Ubuntu with zfs on AWS

Some providers use OpenIndiana

Some providers use SmartOS (ie. Joyent)

Special applications:

Manta (map reduce for unix - if you could run EC2 inside S3 plus elastic mapreduce)

ZFS