jmtd → computing → Network Attached Storage (NAS)

Coolermaster case on the right

Φόβος (phobos) is the name of my NAS, backup and home server. The current hardware setup is described at phobos; this page aims to describe the rationale, setup, software and usage.

Software

Debian GNU/Linux for the operating system.

I don't use RAID (which is not backup).

Remote decryption

I use full-disk encryption which necessitates supplying a passphrase when the machine is booted. Since this is a headless box, some additional work is needed to permit supplying this passphrase over the network. Luckily most of the work is done already by installing the dropbear Debian package and reconfiguring keys and authorized_keys files in /etc/initramfs-tools afterwards. This means I can SSH into the pre-boot environment. From there, I just need to run cryptroot-unlock and supply the decryption passphrase.

Notifications

blinkenlights!

I wanted to add a means of notifying me of events on the machine. I bought a Blinkstick Nano, a tiny USB stick with an LED on each side. I've hooked calls to change the light colour into the success/failure paths for the systemd jobs that drive my backups. Further details here: 1, 2, 3.

The light defaults to off. When an interactive job is in progress, it turns on and blue. When the job completes, the light changes to either green or red depending on success or failure. Green means I am safe to remove the drive, in the case of external drives.

When a non-interactive, scheduled job fails, the light turns red. I usually notice this next morning.

Backup software

I backup using Borg.

Remote hosts

Borg works using a push model: a backup client needs to have permission to connect to the Borg backup server, and self-initiates its backups. I prefer to pull from my backup server, and have the clients trust it — less chance that an external host is compromised giving an attacker access to my NAS. The schemes for achieving this in Borg (ssh tunnels) are awkward, so instead I do things in two phases: a simple server-initiated rsync of clients to a holding space in the NAS, followed by local-only Borg backups of the holding spaces.

TODO: At the moment, nothing except scheduling prevents the rsync jobs from running at the same time as the borg ones. I should add the necessary exclusions (After=, I think)

Borg / Borgmatic

The main backups are via Borg, but I use borgmatic as a convenience wrapper and that's what I schedule (via systemd timer) to fire at 3am nightly. The script calls borgmatic --prune --create, which has several different backups configured (currently 9). These run serially.

I deliberately omit --check from borgmatic or it would run a borg check for each of these nine configurations. Instead I run borg check separately, currently scheduled once a week.

It typically takes 1h15m for the backups to run and 5h30m for the check stage to run.

Third-strand external

Every month I backup the Borg backup repository to one of two external hard drives that live off-site the rest of the time. This requires an exclusive lock for the Borg database so I cannot do it if the Borg backups or check jobs are running.

In recent times this has taken anything between 2h20m and 5h.

Schedule

I keep forgetting when the backup jobs typically are running (which excludes doing other things at the same time), so here's a rough sketch.