JshKlsn's Blog

How to back up Unraid to Backblaze B2 via Rclone

I recently got my internet upgraded and started looking into cloud backup solutions that I previously couldn't handle.

I wanted to go with Amazon S3 Glacier Deep Archive, but due to their confusing documentation and rules, I decided it was easier, albeit a bit more expensive, to go with something more straight forward such as Backblaze B2.

While Amazon has cheaper storage pricing at $0.00099 per GB vs Backblaze B2 at $0.006 per GB, Amazon charges per object and their Glacier Deep Archive takes ~12 hours to deliver your data. Backblaze only charges per GB stored, and you get free egress up to 3x your storage.

So while Amazon S3 has the cheaper cold storage cost, their egress and transactional costs will get you. Plus their fees can be confusing to understand compared to Backblaze B2.

So here's a step by step on how to setup your Unraid server to backup to Backblaze B2 via Rclone

Backblaze B2

B2 Account

Start by signing up for a Backblaze B2 Cloud Storage account.

Once you're signed up, you automatically get 10GB for FREE, no credit card required. If you're going to store more than 10GB, make sure you go to the Billing tab (your account name > billing) and input a payment method.

B2 Buckets

Next, we are going to create a Bucket. A Bucket is just a directory on Backblaze. You can give buckets rules, like if they are private or public, encrypted or not, how long to keep old files, etc. It's suggested to make different Buckets for different data if you plan on having different rules. You can also setup different keys to allow different users to access certain Buckets. Remember, B2 is also used by Enterprises, we just happen to be using it to backup our silly little Unraid servers.

For example, I have a Bucket for my Unraid AppData, and I have this set to only keep the latest files. I have another Bucket for my Immich directory, and I have this set to keep 30 days worth of old files. That way if something goes wrong and Immich borks itself and deletes everything, my files will still be available on B2 for 30 days.

Setup your Buckets accordingly.

Application Keys

Next we need to create an Application Key. Go to the Application Keys tab and click "Add a New Application Key".

Give it a name, and setup the permissions to your preference. Personally I allow the key to access all Buckets along with read and write access.

Unraid

Getting the plugins

We are going to be using Rclone and User Scripts to backup and automate this process. Both are available on the Unraid Community Apps as plugins.

Setting up Rclone

In Unraid, under the Plugins tab, you will find Rclone. Simply click the icon to go to the Rclone settings.

You'll notice the WebUI is not running/installed. That's okay. We are using the command line instead.

You will see a big empty "Edit config" box. If you know what you're doing, you can simply put your config in this box. If you don't know, we will use the command line to populate this box via rclone.

Open a new terminal window in Unraid (it's the button in the top right that looks like >_) and type;

rclone config

You wont have any remotes, so you'll want to press n to create a new one.

enter a name for your new remote. To keep things simple, I named mine remote.

Now you'll be asked for a type of storage. Find Backblaze B2 and enter that number below and press enter (at the time of writing, it's option 5)

Next you'll be asked for your Account ID or Application Key ID. It's a bit confusing, but you'll want to enter your "keyID" you got from Backblaze and press enter.

Now you'll be asked for your Application ID. This is where you put your "applicationKey" from Backblaze.

The next question is important. If you set hard_delete to TRUE, when rclone tells Backblaze to delete a file, the file is immediately deleted. If you set this to FALSE, rclone will tell Backblaze to delete the file, and Backblaze will "hide" the file for 24 hours before deleting it. This gives you a small window to recover files you did not mean to delete.

Next, press n to skip editing the advance config.

Next, press y to save the config.

Now if you refresh the rclone plugin page, you'll notice the big white empty box has now been populated with some data.

Testing Rclone

In Unraid terminal, you can type the following to see if rclone is setup properly;

rclone lsd remote:

replace remote: with whatever you named your rclone remote above. If you named your rclone remote happyturtle, you'd enter;

rclone lsd happyturtle:

Making an Rclone command

I am going to tell you how I have my Rclone setup, and you can see how it works. You can then easily modify the command to fit your needs.

I have a bucket called appdataBackups and I want to sync a folder on my Unraid where the appdata backups are stored to that Bucket.

here is my command;

rclone sync "/mnt/user/Backups" "remote:appdataBackups/Backups" --progress --transfers=4 --checkers=8 --buffer-size 16M --fast-list --bwlimit 17.5M

let me break it down for you.

rclone sync

this tells rclone to sync the data from my Unraid server to Backblaze. I personally chose sync because I want my data on Backblaze to be mirrored with the data on my Unraid server. If I delete a file on my server, I want it to be deleted from Backblaze. If you prefer all files to be uploaded to Backblaze and NEVER DELETED, use rclone copy instead.

"/mnt/user/Backups" "remote:appdataBackups/Backups"

the next chunk is specifying a directory on my server, and then the Bucket and directory on Backblaze. The quotes are just so spaces in directory names are recognised.

Notice the remote: part? Again, change that to whatever rclone remote name you set earlier. If it was happyturtle, you'd put that instead.

appdataBackups is the name of the Bucket on Backblaze. You saw earlier the name of your Buckets with the lsd command.

The rest is just specifying the directory on Backblaze. Notice how it's the same as my local structure? I do that on purpose so it's easier in the event of a recovery.

--progress --transfers=4 --checkers=8 --buffer-size 16M --fast-list --bwlimit 17.5M

These are rclone flags.

--progress to show progress during transfers.

--transfers and --checkers are used for setting how many files to transfer in parallel and how many to check. You can adjust these based on your internet speed and CPU.

--buffer-size for setting memory buffer size for reading files for each transfer.

--fast-list for listing directories. Uses more RAM, but fewer transactions.

--bwlimit for limiting the bandwidth speed. This is in MB, so 17.5MB = 140Mbps. I use this to ensure rclone isn't using my entire upload bandwidth, which causes my internet to slow down a lot.

and honestly, that's about it. That single command will sync my Backups folder on my Unraid server to Backblaze, uploading new files, and removing deleted files from the cloud.

But what if you want to upload multiple directories to different Buckets at different times and maybe receive notifications when completed? User Scripts.

Setting up User Scripts

Now that we know Rclone works and is able to access your Buckets, we can write a User Script to do this automatically and even send you a notification!

You can find User Scripts in the same place as Rclone. Go to your Plugins tab and click on the User Scripts icon.

At the bottom click "add new script" and give it a name. If you plan on having multiple rclone scripts, it would be helpful to be descriptive in your naming.

Here is one of my scripts I use for backing up my AppData and MariaDB directories, logging the changes, and then sending me a Pushover notification with the contents of the log file.

#!/bin/bash
LOG_DIR="/mnt/user/Backups/LOGs"
mkdir -p "$LOG_DIR"
TS=$(date +%Y%m%d-%H%M%S)
LOG_FILE="$LOG_DIR/rclone-backup-directories-$TS.log"

PUSHOVER_USER=""		# replace with yours
PUSHOVER_TOKEN=""		# replace with yours
PUSHOVER_DEVICE=""		# optional
MAX_CHARS=1000			# keep under Pushover ~1024 char limit

send_pushover() {
  local title="$1"
  local file="$2"
  [[ -f "$file" ]] || return 0

  # get last MAX_CHARS bytes (fallback to last 200 lines if tail -c not supported)
  local body
  if body=$(tail -c "$MAX_CHARS" "$file" 2>/dev/null); then
    :
  else
    body=$(tail -n 200 "$file" 2>/dev/null)
  fi

  # prefix if truncated
  local full_size
  full_size=$(wc -c <"$file" 2>/dev/null || echo 0)
  if (( full_size > MAX_CHARS )); then
    body="--- log truncated (last ${MAX_CHARS} bytes) ---"$'\n'"$body"
  fi

  # send with curl using --form-string to avoid needing URL-encoding utilities
  curl -s \
    --form-string "token=${PUSHOVER_TOKEN}" \
    --form-string "user=${PUSHOVER_USER}" \
    --form-string "title=${title}" \
    --form-string "message=${body}" \
    ${PUSHOVER_DEVICE:+--form-string "device=${PUSHOVER_DEVICE}"} \
    "https://api.pushover.net/1/messages.json" >/dev/null 2>&1 || true
}

echo "==== $(date +'%Y-%m-%d %H:%M:%S') === rclone: starting MariaDB Backups ====" >> "$LOG_FILE"

rclone sync "/mnt/user/Backups/MariaDB Backups" "remote:myBucket/Backups/MariaDB Backups" \
  --progress --transfers=4 --checkers=8 --buffer-size 16M --fast-list --bwlimit 17.5M \
  --log-file "$LOG_FILE" --log-level INFO
chmod 0777 "$LOG_FILE" || true

echo "==== $(date +'%Y-%m-%d %H:%M:%S') === rclone: starting AppData Backups ====" >> "$LOG_FILE"

rclone sync "/mnt/user/Backups/AppData Backups" "remote:myBucket/Backups/AppData Backups" \
  --progress --transfers=4 --checkers=8 --buffer-size 16M --fast-list --bwlimit 17.5M \
  --log-file "$LOG_FILE" --log-level INFO
chmod 0777 "$LOG_FILE" || true

send_pushover "rclone: AppData & MariaDB Backups finished" "$LOG_FILE"

It seems complicated, only because of the Pushover code.

Pretty much the script is running two rclone sync commands for two different directories, creating a log file in the LOGs directory with the current date/time, then sending a notification to my phone via Pushover to let me know it has finished, and what files have been uploaded/changed/deleted.

If you don't care about notifications or logs, you can get away by just copy/pasting the rclone command from the making an rclone command section.

Scheduling the script

After saving the script in User Scripts, you'll notice a box to the right. Clicking on this box allows you to setup a schedule. I personally use Custom so I can set it up exactly how I want with Cron.

Backing up Immich

I mostly use Backblaze for backing up my Immich files. It uses a very similar script as the one above, but I also shut down and restart the Immich containers before/after I backup my Immich directories. While this is not required, Immich does recommend this in their official documentation to prevent your database and assets from being out of sync.

I sync the entire Immich directory, which depending on how much data you have, may have a bunch of encoded videos. If you do not want to waste bandwidth or storage on encoded videos, you can skip that directory and Immich will restore perfectly fine. You'll just need to rerun the transcoding job if you restore from B2 in the future. Considering I have a pretty powerful server, I may choose to exclude the encoded-video directory in the future. Same goes for the thumbs directory. If you choose to skip it, you will need to rerun the thumbnail generation job in the future.

Keep in mind, by default, Immich does nightly tasks at 00:00 and database dumps at 02:00.

Make sure you set your script to run rclone before or after these tasks, or change the timing of the tasks. Also keep in mind how much data you'll have rclone sync, and your internet speeds, to ensure you leave enough time for rclone to finish and the containers to come back up before the Immich scheduled tasks.

Personally I let Immich do the nightly tasks and database dumps at the default time, and I set rclone to run at 7am. That means there's plenty of time for Immich to finish the tasks, but also ~17 hours for rclone to sync to B2, just in case I dump a lot of data into Immich that day.

Encrypting data

So if you'll notice, we aren't actually encrypting any of our data before sending it over to Backblaze. While this is possible to setup, and really easy, I don't trust myself to not lose my rclone crypt keys, rendering all of my data useless anyway.

If you still want encryption, but don't want to manage any of your own keys, you can turn on encryption on a per-Bucket basis on Backblaze.

By default, you'll be using SSE-B2, which is Server-Side Encryption by Backblaze. You can read more about it on Backblaze docs.

You can also setup SSE-C, which means Server-Side Encryption with Customer-Managed Keys. However, if you're going to be managing your own keys, you might as well just use rclone crypt.

Setting up Rclone crypt

If you want to encrypt your data before it leaves your server, you can do so with Rclone crypt. It's really easy, and builds on what we've already done.

To start, you'll want to go back to the Setting up Rclone and following the steps again. However, this time, instead of selecting Backblaze B2 as the type of storage, you'll want to select "Encrypt/Decrypt a remote (crypt)" which is option 16 as of writing this.

When it asks for your remote, you will want to type the name of your previous, non-encrypted, B2 remote. We named ours remote:, so we will type that here.

Next you can decide if you want filename encryption and directory name encryption.

Next you'll type in a password and an optional SALT.

That's all! Now to use it, you just need to use it in your script instead.

Assuming you named your crypt remote remote-crypt, you'll update your scripts like so;

rclone sync "/mnt/user/Backups" "remote-crypt:appdataBackups/Backups" --progress --transfers=4 --checkers=8 --buffer-size 16M --fast-list --bwlimit 17.5M

Rclone will access your original B2 remote through the crypt remote, encrypting your data before upload.

This is safer and ensures no one can access your files except you. Backblaze never even sees decrypted versions.

Just be aware that recovering encrypted data via rclone later can be a pain. You need to ensure you have the keys, and that your rclone remote is setup the exact same.

This will be something I tinker with in the future, but for now, I don't trust myself to not lose the keys, or setup rclone the same as before to be able to pull and decrypt the data properly.

Conclusion

That's a basic tutorial on how to setup Unraid to sync to Backblaze B2 via Rclone. I hope you found this useful!