Slow TimeMachine Backups

I love TimeMachine. It works in the background transparently and has saved me several times. As an added bonus, if the internal drive fails, you can boot off of the TimeMachine disk and finish your work. A few years ago, the disk failed in one of my older MacBooks and I had it back up, restored from the TM backup, and running in about six hours, including the time it took to drive to Best Buy to purchase a new disk.

So I was very disappointed that TM backups were taking an extremely long time on my new, space gray, everything-turned-up-to-11, $4k+ MacBookPro with a 2TB SSD -- it would spend an entire day in the Preparing Backup phase and never start the actual backup before I had to eject the backup disk to bring it home. Next day, same thing. As I discovered, its Achilles heel is dealing with lots of small files. 

Searching the web, I found a lot of useless info, like use Disk Utility to run "first aid" and "fix permissions" on the backup and internal disks, reformat the backup drive, replace the drive cable, upgrade to the latest version or downgrade to a previous version of macOS, uninstall and reinstall MS Office (fun to blame it on MS), reset the PRAM and SMC (has this ever fixed anything?), boot in Safe Mode, face towards Cupertino, bow deeply and mutter, "if only Steve Jobs was still alive...."

Many pages suggest unthrottling backupd with 

    sudo sysctl debug.l​owpri\_throttle_enabled=0​  (=1 to undo)

​Almost all claim this is a CPU throttle, but it's an I/O throttle--see the source code at 

https://opensource.apple.com/source/xnu/xnu-2422.1.72/bsd/miscfs/specfs/spec_vnops.c?txt

Regardless, it didn't speed anything up for me, but I did notice that the machine was less responsive with this disabled. For example, a 'git status', which usually runs instantly, would take 10-15 seconds, Emacs was constantly "beach-balling," and it even seemed like ssh'ing into work was sluggish.

What finally helped was monitoring backupd's file system usage with

    sudo fs_usage -f filesystem backupd |grep fsctl

What I saw was that in the "Preparing" phase, TM was scanning 100s-of-thousands of small files in TeXLive (fonts, font metrics, etc.), GarageBand (loops, samples, ...), and iTunes Podcasts.

It turns out when you upgrade TeXLive, it leaves the old installation in place and simply adjusts a symbolic link to point at the latest one. I had five old installations, amounting to about 400k files. I also had some video files that I had broken out into individual frames, accounting for a further 108k files (about an hour of video).

I subscribe to a number of podcasts on my iPhone, like the NYTimes "The Daily" and Vox "Today Explained," to listen to during my commute.  When you back up your phone to your laptop, it also syncs your iTunes subscriptions and the default behavior in iTunes is to download them daily and keep them forever (!!), so I had a few thousand podcasts lying around.  To change the defaults, in iTunes, select Podcasts, then push the Defaults button at the bottom.  Select Limit Episodes: Most Recent, Download Episodes: Off. When you press OK, it will ask to confirm that you want to delete [large number] podcasts... Yes (Please do!). 

I deleted the old installations of TeXLive and put the video frames in a tar file (no compression, since they were already jpeg).  I'm not sure what to do about GarageBand.

Now, the "preparing backup" stage runs in 10 minutes, and the backups proceed at about 30 GB/hr. That's still over two days for a full backup, but at least the first incremental of the day runs in under an hour.