How Increase LAN Speed for NAS to NAS Backup?

[rkruz3]

When using the Hyper Backup app (DSM7.01) from a Synology NAS (DS220J) to a NAS (DS420J), both on the same home LAN, the network data rates are around 10 to 30 MB/s, well below the 1Gb Network capacity of the system. I'm seeing at best about 27 MB/s of backup data passing over the LAN between NAS's, perhaps about 10% of the capacity of the LAN and I typically see other data transfers, e.g. Torrent or other to the DS220J on the LAN more on the order of 60% capacity. The meager 500MB RAM of the DS220J is a steady 74% used and CPU is around 17%.

So the question: If I go to a DS220+ with its factory 2GB RAM, do you think I will see a marked improvement over the Hyper Backup transfer rates or, are there some techniques to significantly improve the backup speeds over the LAN?

 

[akahan]

Before dropping the $$ for a new NAS, you could see if you get improvement by setting Hyperbackup to NOT use compression, SSL, or encryption. You certainly don't need SSL over the LAN!

 

[Rusty]

Also try and test regular SMB transfers between the devices. HB and other backups tools behave differently then a pure copy transfer scenarios when it comes to speed. Snapshot app is generally faster then HB for the same task, and pure SMB should be even faster. So you should get better speeds outside HB just to see if you can utilize more % of your 1G connection.

Try and test a number of smaller files, as well as one large file to see the difference as well.

 

[EAZ1964]

Fully agree with both previous posts.
Additionally, not sure what your use case is, why is time so important?
My backups run daily, incremental, scheduled in the out of office hours, and typically take 15 min max.
Who cares if they run for 30 minutes?

 

[Robbie]

@rkruz3 I agree that HB is not quick and that there are many users who have a wish or need for it to run faster.

Not all your performance information is shown but it looks like the Volume is actually working pretty hard and it is sometime too easy to forget how slow mechanical drives can be with only multiple small read and writes to feed on. The SATA interface itself has the limitation of either read or write rather than being able to transport both at the same time (unlike SAS or PCIe drives).

Increasing the raw performance capability of the NAS does help (CPU, RAM, number of drives, drive speed etc) or lowering the burden (smaller backups, less increments retained, smaller retention window) but they will only go so far before you see the Volume / HDDs at 100% utilisation and you are stuck with the mechanical and interface limitations.

 

[rkruz3]

Fully agree with both previous posts.
Additionally, not sure what your use case is, why is time so important?
My backups run daily, incremental, scheduled in the out of office hours, and typically take 15 min max.
Who cares if they run for 30 minutes?

well...at 1day into the first full backup, its at 20%complete so looks like 5 days to complete a raw data 3TB backup. Speeds are 3MB/s this morning. The DS220J plays videos well even during the backup. But I cannot save qbittorent files to the drive when its backing, those are much to slow or just stop due to network errors. Once it get to incrementals Im sure the times will be much more reasonable.

 

[EAZ1964]

So what is your use case then? Don't tell me you are creating full backups every time?
Backup is much, much more than a copy, it is compressing, indexing etc etc. It is not realistic to expect full "big files disk write performance.

A first backup always takes a few days, but all next backups should only backup the new/modified files and you should be done in minutes.

 

[Robbie]

EAZy, given the 3MB/s transfer rate I don't see anything in rkruz3's posts to suggest these backups are overly large rather than just plain old slow....

 

[EAZ1964]

@Robbie Not sure what you mean. In my experience variability between 3 and 30 is "to be expected" speed for a first HB.
Hope that the bittorrent folders are excluded from the dataset, otherwise HB will try to keep up with the changes.
Just want to understand if this is a first backup, that will be an incremental one in the future. then we should just be patient, It will normalize in next runs.

 

[rkruz3]

@rkruz3 I agree that HB is not quick and that there are many users who have a wish or need for it to run faster.

Not all your performance information is shown but it looks like the Volume is actually working pretty hard and it is sometime too easy to forget how slow mechanical drives can be with only multiple small read and writes to feed on. The SATA interface itself has the limitation of either read or write rather than being able to transport both at the same time (unlike SAS or PCIe drives).

Increasing the raw performance capability of the NAS does help (CPU, RAM, number of drives, drive speed etc) or lowering the burden (smaller backups, less increments retained, smaller retention window) but they will only go so far before you see the Volume / HDDs at 100% utilisation and you are stuck with the mechanical and interface limitations.

I believe you. Great analysis. Digging deeper into the DS220J performance metrics, I see the drives are at 100% utilization. See the attached Disk Utilization. However, there must be a significant Hyper Backup (HB) processing slow down though as if I remember correctly, I did a straight copy of the data over to the destination NAS in about 1/4 the time this HB is taking for the same files or about 3TB. The destination NAS Disk Utilization is at 20%. There is no encryption but there is compression. I thought selecting HB compression would improve the backup speed with smaller backup files, but maybe the compression processing is killing the speed. The low CPU usage might not support that theory though. I'm tempted to take my losses and try it again without compression.

-- post merged: 7. Jan 2022 --

@Robbie Not sure what you mean. In my experience variability between 3 and 30 is "to be expected" speed for a first HB.
Hope that the bittorrent folders are excluded from the dataset, otherwise HB will try to keep up with the changes.
Just want to understand if this is a first backup, that will be an incremental one in the future. then we should just be patient, It will normalize in next runs.

Good point to exclude active torrents from the backup. There are no torrents for this backup but for future I will modify the back to exclude.

-- post merged: 7. Jan 2022 --

So what is your use case then? Don't tell me you are creating full backups every time?
Backup is much, much more than a copy, it is compressing, indexing etc etc. It is not realistic to expect full "big files disk write performance.

A first backup always takes a few days, but all next backups should only backup the new/modified files and you should be done in minutes.

This is the first backup. All subsequent will be incremental. I'm guessing 5 days to complete the full backup.

 

[Robbie]

In my experience variability between 3 and 30 is "to be expected" speed for a first HB.

Yikes - 30MB/s would be a solid minimum for a first HB for me, when it is chewing on the harder bits and running with compression etc.

With easy sequential contiguous data mine will peak way-over 100MB/s and closer to 200MB/s peak towards the start of a first backup, even when firing the data at a single mechanical drive. But if your Volume is hitting 100% utilisation you are stuck at whatever that equates to.

 

[rkruz3]

What is your NAS models and configuration? Are you backing up NAS to NAS over a 1Gb LAN using Hyper Backup?

 

[Robbie]

I have a 10 GbE network with a 10 GbE prime NAS and others either bonded or running with a 5 GbE link. But it matters little when the Volume itself is the bottleneck.

 

[rkruz3]

I have a 10 GbE network with a 10 GbE prime NAS and others either bonded or running with a 5 GbE link. But it matters little when the Volume itself is the bottleneck.

yup. fer sure

 

[EAZ1964]

Anyway, after your first 5 days, I expect your backup to be finished in minutes and normally no reason to heavily invest in higher specs NAS for this single reason.
Having said that, I do not recommend J types ever. Most people regret it after some time.

 

[Robbie]

What is your NAS models and configuration?

My NAS models are as per signature block and pictured below:

I also have a macOS server and a Windows server running too, both on 10GbE.

️

 

[EAZ1964]

Another thing on HB :
I do recommend to split the 1.6Tb backup into smaller, logical backups. (based on eg fast movers-documents, slow movers video/photo/music).
It gives you much more flexibility (in frequency, in versioning, in backup locations, speed of a restore).
Also, it is less of a single point of failure in case a file error occurs.

 

[rkruz3]

Anyway, after your first 5 days, I expect your backup to be finished in minutes and normally no reason to heavily invest in higher specs NAS for this single reason.
Having said that, I do not recommend J types ever. Most people regret it after some time.

yes Im one of them. No biggie, I learned a lot with the 220J. Everything on my network seems to work find during the back, even running videos from the NAS so Ill just let it run. Thanks for the advise.

 

[sktn77a]

"data rates are around 10 to 30 MB/s, well below the 1Gb Network capacity of the system"

What kinds of speed are you expecting? 30MB/sec is 240Mb/sec. Your Gb network is probably only getting you 75% of that (750Mb/sec) at best. After that a lot of other factors come into play, further reducing your speed. The best I ever get transferring files over my wired Gb network is about 350Mb/sec. So you're not alone.

 

[Robbie]

@sktn77a - Sounds rather low; 900Mbps+ is more normal for 1 GbE and a reasonable wifi system should hit speeds in excess of your stated wired figure too.

Is it possible that you are hitting the limits of a client or host with your file transfers?

[edit:]

Just done a large file transfer from a macOS client to my RS217 - a very underpowered (ie gutless!) unit via 1 GbE:

- File transfer rate 114.1 MB/s or 912.8 Mbps

Running a quick Disk Speed Test:

Again, just normal stuff for 1 GbE links.