Exploring APFS

File systems are fascinating and scary. When I heard that Apple was working on its own in 2016, I was immediately interested. Now, in 2018, APFS is standard on both macOS and iOS.

Why are filesystems so interesting? With FUSE, anyone can implement their own filesystem. But these run in userspace, where things are slower but protected from programming errors¹. Real file systems run in kernelspace², and that is where things get interesting because programming errors can crash the system³. But much worse than crashing a system, an error could also cause corruption or loss of data. With that in mind, it becomes apparent that file system authors are incredibly talented people⁴.

Motivations

Before APFS, Apple used HFS+, which was introduced in 1998 as an extension of HFS, and that in turn was introduced in 1985. Both of these were designed for storage media like hard drives.⁵

So what is so special about hard drives? These work similar to an old record player: they have rotating disks (usually multiple stacked on top of each other) with a round track consisting of the data, and they have a pickup that can move to follow the track or jump to another position.

These kinds of storage media are really good at reading and writing sequential data. That means that when you read a file from beginning to end, the speed is quite good. However, if you jump around, reading bits and pieces from different places, the performance rapidly deteriorates. This is because the reading head has to physically move and find whatever you are looking for.

File systems designed for these kinds of storage media have some very specific contstraints. For example, they might try not to split up files. Maybe they will try to put files that are accessed frequently close to another, so that they can be accessed faster. All of these optimizations are built on knowledge of how hard drives work internally.

All was going well for file system architechts and implementors, until the SSD took on a major role in personal computing. As the name solid state drive suggests, the main difference between regular hard drives is that it doesn’t have any moving parts.

As such, it has different properties that file system implementors need to watch out for. For once, when accessing data, there is almost no different between sequential and random access, meaning that the speed is the same if you read a file from beginning to end as if you access small bits in a random order. But the one issue is that SSDs have a limited life span, because the individual memory cells that it is made up of can only take so many writes.

When Apple was designing APFS, one of the reasons behind that was that they designed it from the ground up to work well with SSD storage, which is now more common than traditional hard drives. While it is possible to add SSD support to existing file systems, for example by adding support for the TRIM command, it is easier to just start from scratch.

In fact, Apple themselves say:

Apple File System is uniquely designed to meet the needs of Apple’s products and ecosystem. Apple File System provides strong encryption, ultra-low latencies and limited memory overhead. It is optimized for flash/SSD storage and can be used on everything from an Apple Watch to a Mac Pro.

HFS+ and its predecessor HFS are more than 30 years old. These file systems were developed in an era of floppy disks and spinning hard drives, when file sizes were calculated in kilobytes or megabytes.

Today, people commonly store hundreds of gigabytes and access millions of files on high-speed, low-latency flash drives. People carry their data with them, and they demand that sensitive information be secure.⁶

Features

APFS is a pragmatic successor to the previous HFS+ in that it doesn’t add anything crazy, but it does have some useful bits and pieces that we can look at in detail.

• Encryption support lets your protect your data safe from prying eyes, unless they have the password.
• It is possible to take snapshots of the state of the file system. This means that you can save the current state of your file system without having to do a full backup. This feature is actually used by Time Machine.
• It supports clones of files. This means that when copying large files, they don’t actually need to be copied, but both files (clones) will point to the same data. When you then edit one of them, the file system only needs to record the changes.
• On a single container (think hard drive or partition), you can have multiple volumes sharing the same space. This is like partitioning a hard drive, but instead of having to specify the sizes beforehand, they all share the same space, so you can divit it up as you like.

Encryption

When you create a new APFS-formatted volume, you have the option of creating an regular or and encrypted volume. For the latter, you just need to enter a password that you will need to remember because you will have to enter it every time you mount your volume, unless you allow your computer to save it in your keychain.

This is not exactly different from the process of creating an encrypted HFS+ volume. I think that the only difference is internally—with HFS+ volumes, the encryption was apparently a little more hackish⁷.

Space sharing

Space sharing means that if you have an APFS-formatted drive, you can create multiple volumes on it that share the same space. This is very similar to having multiple partitions on a single drive, with one crucial difference: with partitions, you have to specify their size when you create them, and it is not easy to change their size (grow or shrink them). With space sharing, multiple volumes share the same container, and they can use as much of that container as they like until it is full.

Disk utility has two buttons that can be used to add and remove volumes from an APFS container. Each volume is like a separate file system, so you can have multiple volumes, some with different encryption keys and some with no encryption.

It is even possible to specify some constraints, like reserving a minimum amount of space for a given volume or limiting its maximum size.

This is a really awesome feature, and I think all file systems should allow this. There are some solutions that allow this, like LVM or some file system that natively support it, like ZFS. The only possible downside is that if the file system was corrupted, it could affect all volumes on it, instead of just one as with partitioning. But that scenario is very hypothetical, and it won’t keep me from using this awesome new feature.

Clones

Apple has a nice overview of some of the new features, including the clone mechanism.⁸ Testing the clone facility is quite easy, if you simply copy something in finder, you will see the result: it is possible to copy a large file, say a 2GB movie, an obscene amount of times almost instantly.

This feature is not without criticism. It does not work when you use cp in the terminal, for example, because you need to use Apple APIs to access it. Also, it might not be overly useful since most programs will completely rewrite files when modifying them⁹.

Snapshots

This is the one feature that I haven’t been able to play around with too much yet, unfortunately. I’ve seen that it is actively used on my system, and I can list which snapshots exist using the tmutil tool.

$ tmutil listlocalsnapshots /
com.apple.TimeMachine.2018-09-15-141751
com.apple.TimeMachine.2018-09-17-214432
com.apple.TimeMachine.2018-09-17-224235
com.apple.TimeMachine.2018-09-17-234536
com.apple.TimeMachine.2018-09-18-004444
com.apple.TimeMachine.2018-09-18-192237

From what I understand, snapshots are used before installing apps or updates to create a “known-good” point to get back to in case things get hairy. Apparently, it is also possible to mount them. But I will save this for another article on another day!

Issues

The interesting thing with file systems is that they are good when you don’t notice them. They are there to store your data, and you should never have to directly interact with them. As long as they keep your data, all is good. As soon as they throw error messages at you, or corrupt your data, you know that something in going wrong.

I haven’t encountered any issues with APFS, and I don’t think I will. It seems like a stable product, and since it is used by default on all new macOS installations and on iPhones as well, I’m sure it receives a huge amount of testing to make sure it runs well.

The only issue is that any applications that rely on specific filesystem features need to be adjusted. And that is the only area where I’ve run into an issue.

Time Machine

Currently, Time Machine only supports storing its backups on HFS+-formatted volumes. When you have an APFS-formatted drive mounted and you try to switch to it, it will not show up in the Time Machine preference pane.

I am very sure that this issue will be fixed very quickly, as I presume that Apple wants to switch all HFS+ volumes over to APFS soon, since they probably don’t want to support both file systems indefinitely.

But in the meantime, there is a workaround that is not pretty but it works for me. I basically created a disk image by pressing ⌘N in Disk Utility (File → New Image → Blank Image). I selected the sparse bundle disk image, which is an image that only takes up as much space as it actually uses, gave it a descriptive name, set the size to 500GB which should be plenty for my backups, and selected Mac OS Extended (Journaled) as the file system instead of the default APFS. This is important.

Next, you need to make sure that the disk image is mounted. All you need to do to convince Time Machine to use it as backup disk is use the command-line tool tmutil. If you named your volume something else than “Time Machine”, you’ll have to change that in the command obviously.

sudo tmutil setdestination "/Volumes/Time Machine"

Conclusion

So far, I think that APFS is a very neat and stable system. I think it has some nice features, but it doesn’t go overboard resulting in a theoretically cool, but practically unstable file system.