TimeMachine Backups on AWS

SJ Meyer

If you are a normal person, the first thing that probably went through your mind when you read the title was; why would you even want to do that? It’s quite simple though, because you can! It started when I got sick of running out of space on my external backup disk. I figured AWS might be able to solve this whole scaling issue.

I first looked at the scalable storage problem. Amazon S3 sounded good from a scalability perspective, but it’s an object storage service. So that was a no-go. I need a scalable file system, so EFS (Elastic File System) was the obvious answer. Storage sorted, next I had to make sure I can connect to the EFS file system from my Mac. Since you can mount NFS file systems on your Mac, I was ready to get started. Well, almost. EFS provides you with private endpoints, which means they are only accessible from inside your VPC. Therefore you have to create a connection between your home or office network and the VPC the EFS file system is deployed into. Simple enough, you can use AWS Site-to-Site VPN for that. So let’s go!

I generally don’t like using the default VPC or associated resources, like the default route tables and subnets. Therefore I first created a new VPC, with only a single private subnet, route table, security group, and network ACL, and wired it all together. Since no traffic will be routed over the open internet, you don’t need an Internet or NAT Gateway. I went with a single subnet because I’m going to be using EFS One Zone. If you are actually considering making this a permanent solution, using multiple availability zones would be advisable for availability and durability considerations.

Setting up a Site-to-Site VPN is a bit of an involved process, so I won’t be covering how to do that here. Once the VPN is available and at least one VPN Tunnel has a status of “UP”, the VPN setup is complete. I would like to point out that you can make use of AWS Direct Connect instead of using a Site-to-Site VPN. I do however doubt that organizations that can afford Direct Connect would use this solution for managing backups. Next up, the EFS file system. This part was quite simple. The CloudFormation template snippet below is basically all you need to create an EFS One Zone file system.

{
    "Type": "AWS::EFS::FileSystem",
    "Properties": {
        "AvailabilityZoneName": "af-south-1a",
        "Encrypted": true,
        "FileSystemTags": [
            {
                "Key": "Name",
                "Value": "TimeMachine Backup FS"
            }
        ],
        "PerformanceMode" : "generalPurpose"
    }
}

This is where the “easy” part ends and the finicky bit begins. Well in the end it’s not all that complicated, it just took a while to get it all working together nicely. True to Mac, you can choose to follow the road less traveled, but they will put a fire pit between you and it. A small thing to start with, macOS — I’m running Big Sur — does not support nfs 4.1 out of the box. Therefore if you follow the AWS docs, the command they provide (as shown below) won’t work. Similarly, if you use Finder and hit Command+K, the connection won’t establish. Changing the “4.1” to “4” in the Terminal command will solve the issue. Alternatively you can edit the /etc/nfs.conf file to set the default version to be used. Doing that will solve the Finder connection issue.

Terminal command for mounting:

$ mount -t nfs -o nfsvers=4,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport <efs_dns_name_or_private_ip>:/   /Volumes/<efs_mount_point_name>

Set default NFS version (edit nfs.conf file command)

$ sudo nano /etc/nfs.conf

Add the following line and save:

nfs.client.mount.options = vers=4

File System mounted, I turned my attention to TimeMachine. TimeMachine can only backup to Mac OS Extended (Journaled) or APFS volumes, but an EFS file system is neither of those. To get around this I created a blank disk image to backup to. I opted for a Journaled formatted disk image, the APFS one gave issues, but using an APFS volume has some advantages over the older Journaled. With TimeMachine specifically, the most notable advantage of using an APFS volume is speed. When creating a disk image you have to set the size, but you can later increase the size of the image if needed. The “Image Format” also has to be selected, and here I went with a “sparse” disk image. This is because you can set the sizeof the disk image to something ridiculous like 100 TB, but the disk image will only take up space on disk as files are written to it.

Note: “sparse bundle” disk images have similar behavior to a sparse disk image, but again, that gave issues when trying to mount from the EFS file system.

The last step is to run the backup. But, you guessed it, it’s not that simple. Mounting the newly created disk image will display it in Finder with all your other external devices and in the /Volumes/ path. However TimeMachine won’t recognize it as a backup device. To set the mounted disk image as the backup location, I had to use the TimeMachine command line utility, as shown below.

sudo tmutil setdestination /Volumes/<mounted_disk_image>

And that’s basically it, everything is setup to run TimeMachine backups to your EFS file system in AWS.

Reviewing the solution:

Looking at the positives first, this solution does give you complete control over where your backups are stored and the availability there of. You also have the ability to use the extensive AWS service catalog to build on top of EFS to do all kinds of weird and wonderful things with your backups. Furthermore, given that you can’t run a TimeMachine level backup to iCloud, this will allow you to have a proper cloud backup of your machine. That being said, when looking at cost, speed, and practicality, this solution starts to falter. 

A 1 TB backup in EFS One Zone will be around $45 per month, and then you haven’t paid the $40 for the Site-to-Site VPN. Depending on your geographic location, latency will probably be a bigger pain than the cost. A 9 GB backup took about 5 hours on a 50 mb/ps uplink to an availability zone in the same city. Granted, that is not the fastest uplink ever, but still. Finally the practical aspect. Just to get it working is a bit of a process, but it doesn’t end there.  Disconnecting from the network, or a drop in the VPN connection, will disconnect you from EFS and unmount the backup disk image. So you would have to setup automatic EFS and backup disk image mounting on your machine, especially when using a MacBook.

So, will this setup replace my external hard drive? No, most definitely not. For my personal needs it’s too slow, too expensive, and not really practical. But it was a heck of a lot of fun to mess with!