Last edited:
HUGE apologies if anything below is incorrect. I am trying to make sure I’ve understood basic backup knowledge on Hyper Backup rotation versions and snapshots. I don’t want to find I’ve made a mistake months or years down the line! And, as after several days, I couldn’t find a guide that gives me the detailed explanation I wanted. Happy to re-edit to correct.
Intro (and thanks to GW2_Jedi_Master) Hyper Backup (HP) is said to be an Object Oriented and incremental type of backup. Running HP will create a list of all the files (and properties) in a “snapshot”, and, creates a list of data chunks (that comprise the actual file’s physical data). When a HP backup is made, it starts by creating a snapshot backup version (and copies every file into the backup). In the snapshot backup it lists all of files to be backed up and all of the metadata (size, permissions, etc) with them. HP goes through all the files and converts them to segmented chunks (a maximum block size). On future HP backups (and snapshots) versions, HP checks to see if that chunk has already been put in the data store. If so, it skips uploading the data. If not, it is uploaded. Finally, the snapshot version is added to the backup archive. If no file has been touched since the last backup, “the snapshot version is the only thing added to the backup archive” because all the chunks are the same. If there are duplicate photos across your volume, again only one copy is stored in the backup archive because the chunks are the same.
When you “Enable backup rotation”, you allow an automatic deletion of old snapshots in HB, depending on your rules you set (eg Smart cycle, custom, from the earliest). When HP runs, and the maximum number of snapshot backup versions has been exceeded, the deletion of the snapshot will remove the snapshot backup "version" from the archive (but not necessarily the data files it represents). When a snapshot is removed, there is the possibility that some data chunks become orphaned, ie no longer referenced by any snapshot. HyperBackup has a vacuum that periodically looks for orphaned data and purges it. It has a rate limit, so if there is a significant amount of data remove it may take a few days for it to happen. If backup rotation setting is left turned "off", then all snapshots versions are kept from that point onwards of running the HP back up task.
Simple Examples
Smart Recycle, Max number of kept versions: 2
I ran manually the same hyper backup (HP) many many many times manually one after the other. The first HP version remained (on the 29th), as part of the Smart Recycle rules. There are two other versions. One was the time of the last HP was run, and the others the last previous saved snapshot. Note the earliest was on 29th, when the first HP was run.
Then rotations option was turned "off" in the HP backup setup task… As expected every HP version was saved...
Then rotations was turned on again for Smart Recycle, Max number of kept versions: 2. and all the earliest snapshot versions stared to automatically be deleted (albeit you are given a warning first!).
The new earliest snapshot then became 12:15 on the 30th (the time I turned back on rotations).
My lessons learned (so far)...
HP backup rotations relate to snapshot “versions” from HP back up events. Think carefully how old (in years) you want your oldest HP snapshots backup versions for? If running HP tasks likely to become over 1+ or 3+ years old, if using “Smart Recycle” option, check if you want to have snapshots older than 1 year, as this may need “Customized retention”. Customized retention allows you to “add” a 1+, 2+ “Version Intervals” as part of your snapshot retentions to be retained.
If files do not change, the HP “data chunk” I don't think these are re-backed up on each running of HP. So, as only “one” version of an unmodified / unchanged file is saved in an HP backup, consider having two (or more) HP tasks saving your files. In my case I’ll have one HP task to my remote NAS, and, I’ll have an HP task to USB backup drive, and maybe a HP task to the Cloud, all running on different days in the week.
Other HP considerations
Backup verification does validates data against stored hashes. It will notify you if data is corrupt. It cannot repair corrupted data.
Synology NAS Snapshot Technology (if NAS running Brts) …and not to confused with HP snapshots backup versions.
Synology NAS' allow snapshots to run over the NAS files. This creates effectively a “read only” of version of your files at the time the snapshot was taken on the NAS itself. So a NAS snapshots are a great solution against some ransomware, as even if the ransomware does access the contents file (or the user accidently deletes or changes a file), they cannot alter the file version that had a snapshot taken on the NAS. So. every time a NAS snapshot is created, a “restore” point is created, which allows you to recover files/folders from a point in time. As I understand, snapshots generally take up very little space and give a lot of flexibility. But. These snapshots are not “backups” in strictest since. If your NAS dies where these snapshots are taken dies, so does all your NAS data. But. Snapshots do allow local quick recovery of files, and they help in facilitating backup tasks to other storage devices or to the cloud.
Useful links found:
Tutorial Snapshot Replication Quick Start Guide - Synology Knowledge Center
Guide: Synology NAS Setup & Configuration Guide (2022) | WunderTech
White Paper: https://global.download.synology.co.../enu/Synology_Data_Protection_White_Paper.pdf
Terms
Incremental backup refers to backing up only changed or newly added files since last full or incremental backup. To recover data from it, the last full backup plus all the incremental backups until the time point of restoration are all required.
Intro (and thanks to GW2_Jedi_Master) Hyper Backup (HP) is said to be an Object Oriented and incremental type of backup. Running HP will create a list of all the files (and properties) in a “snapshot”, and, creates a list of data chunks (that comprise the actual file’s physical data). When a HP backup is made, it starts by creating a snapshot backup version (and copies every file into the backup). In the snapshot backup it lists all of files to be backed up and all of the metadata (size, permissions, etc) with them. HP goes through all the files and converts them to segmented chunks (a maximum block size). On future HP backups (and snapshots) versions, HP checks to see if that chunk has already been put in the data store. If so, it skips uploading the data. If not, it is uploaded. Finally, the snapshot version is added to the backup archive. If no file has been touched since the last backup, “the snapshot version is the only thing added to the backup archive” because all the chunks are the same. If there are duplicate photos across your volume, again only one copy is stored in the backup archive because the chunks are the same.
When you “Enable backup rotation”, you allow an automatic deletion of old snapshots in HB, depending on your rules you set (eg Smart cycle, custom, from the earliest). When HP runs, and the maximum number of snapshot backup versions has been exceeded, the deletion of the snapshot will remove the snapshot backup "version" from the archive (but not necessarily the data files it represents). When a snapshot is removed, there is the possibility that some data chunks become orphaned, ie no longer referenced by any snapshot. HyperBackup has a vacuum that periodically looks for orphaned data and purges it. It has a rate limit, so if there is a significant amount of data remove it may take a few days for it to happen. If backup rotation setting is left turned "off", then all snapshots versions are kept from that point onwards of running the HP back up task.
Simple Examples
Smart Recycle, Max number of kept versions: 2
I ran manually the same hyper backup (HP) many many many times manually one after the other. The first HP version remained (on the 29th), as part of the Smart Recycle rules. There are two other versions. One was the time of the last HP was run, and the others the last previous saved snapshot. Note the earliest was on 29th, when the first HP was run.
Then rotations option was turned "off" in the HP backup setup task… As expected every HP version was saved...
Then rotations was turned on again for Smart Recycle, Max number of kept versions: 2. and all the earliest snapshot versions stared to automatically be deleted (albeit you are given a warning first!).
The new earliest snapshot then became 12:15 on the 30th (the time I turned back on rotations).
My lessons learned (so far)...
HP backup rotations relate to snapshot “versions” from HP back up events. Think carefully how old (in years) you want your oldest HP snapshots backup versions for? If running HP tasks likely to become over 1+ or 3+ years old, if using “Smart Recycle” option, check if you want to have snapshots older than 1 year, as this may need “Customized retention”. Customized retention allows you to “add” a 1+, 2+ “Version Intervals” as part of your snapshot retentions to be retained.
If files do not change, the HP “data chunk” I don't think these are re-backed up on each running of HP. So, as only “one” version of an unmodified / unchanged file is saved in an HP backup, consider having two (or more) HP tasks saving your files. In my case I’ll have one HP task to my remote NAS, and, I’ll have an HP task to USB backup drive, and maybe a HP task to the Cloud, all running on different days in the week.
Other HP considerations
Backup verification does validates data against stored hashes. It will notify you if data is corrupt. It cannot repair corrupted data.
Synology NAS Snapshot Technology (if NAS running Brts) …and not to confused with HP snapshots backup versions.
Synology NAS' allow snapshots to run over the NAS files. This creates effectively a “read only” of version of your files at the time the snapshot was taken on the NAS itself. So a NAS snapshots are a great solution against some ransomware, as even if the ransomware does access the contents file (or the user accidently deletes or changes a file), they cannot alter the file version that had a snapshot taken on the NAS. So. every time a NAS snapshot is created, a “restore” point is created, which allows you to recover files/folders from a point in time. As I understand, snapshots generally take up very little space and give a lot of flexibility. But. These snapshots are not “backups” in strictest since. If your NAS dies where these snapshots are taken dies, so does all your NAS data. But. Snapshots do allow local quick recovery of files, and they help in facilitating backup tasks to other storage devices or to the cloud.
Useful links found:
Tutorial Snapshot Replication Quick Start Guide - Synology Knowledge Center
Guide: Synology NAS Setup & Configuration Guide (2022) | WunderTech
White Paper: https://global.download.synology.co.../enu/Synology_Data_Protection_White_Paper.pdf
Terms
Incremental backup refers to backing up only changed or newly added files since last full or incremental backup. To recover data from it, the last full backup plus all the incremental backups until the time point of restoration are all required.
