Hyper Backup to USB HDD takes up most resources of NAS :(

Currently reading
Hyper Backup to USB HDD takes up most resources of NAS :(

38
4
NAS
DS718+
Since I started anew with a Full NAS backup task to a USB3 HDD I noticed this behaviour again. During the backup process (Local Folder & USB - single version), some of the NAS features simply stop working. In the resource center you "only" see that Disk Activity is very high and Volume Activity is maxed out, however it seems mostly docker is very much hit by a running backup. My Home Assistant basically does not update anything anymore. Not from it's internal logic (Data is stored in Synology package Maria DB), not from other docker images (mqtt, zigbee) and related nothing comes in from the USB connected zigbee antenna. Also the more advanced data analysis functions of Unifi crawl to a hold.

My docker containers are running in Med CPU priority. I could up this, but it feels overkill, for when I'm not backing-up, which is most of the time. Is there a way I can lower Hyper Backup's impact on the overall system?
 
Source (DS718+):
- is it single or RAID1/SHR?
- what specific disk drive/s, capacity & PN
- what capacity is used now in the source? (%)
- file system
- total data size and total files for the backup task

Target (USB)
- is it enclosure or USB/SATA adapter?
- what exact disk drive, capacity & PN?
- what capacity is used now in the target? (%)
- file system
- when this drive isn’t new, is there a fragmentation of an existing data? Check it pls.
 
Volume Activity is maxed out
Yep, you run out of IOPS by the looks of it and everything is impacted. Some services and apps react and respond to it more than others but it could be that there is a problem even before you start your backups. Have you checked your volume utilization when backup is not running? Is it also high/spiking?
 
Thanks for the quick replies!! I'll look up the details later today. Backup is still running, it is a first run of a new backup task of around 4TB. Never seen any suspicious activity when not running backups, but'll I'll check when this one is done in a couple of hours.

DSM on the other hand is still pretty responsive. I think it's mainly docker and it's containers that get hit.
 
So first backup finished. (Now running another job, including encryption, so more load :) ) Weird thing is, as long as the USB HDD is connected, even when no backup task is running, so when not used, I have issues with my Conbee Zigbee USB stick. They're each connected to either of the back USB ports. I assume those 2 ports are seen as 1 hub and apparently there is some conflict there. For the Conbee stick to work I have this in a startup.sh
sudo insmod /lib/modules/usbserial.ko
sudo insmod /lib/modules/ftdi_sio.ko
sudo chmod 777 /lib/modules/usbserial.ko
sudo chmod 777 /lib/modules/ftdi_sio.ko

Source (DS718+):
- is it single or RAID1/SHR: RAID1
- what specific disk drive/s, capacity & PN: WDC WD80EFZX-68UW8N0 7,3 TB
- what capacity is used now in the source? (%) 57%
- file system BTRFS
- total data size and total files for the backup task: No idea, not in the mood to have my NAS calculate that now. Or is there an easy way to retrieve?

Target (USB)
- is it enclosure or USB/SATA adapter? Not sure what you exactly mean, It's an off the shelf USB3 HDD with seperate PSU. I assume it has a SATA HDD inside with some sort of adapter, but I haven't opened the case.
- what exact disk drive, capacity & PN? Western Digital Technologies My Book 2SEE 7,3 TB
- what capacity is used now in the target? (%) It's filling up now, but it will be around 50% as the backups don't contain the Drive version history if I'm not mistaken
- file system: ext4
- when this drive isn’t new, is there a fragmentation of an existing data? Check it pls. I guess so indeed.
 
so
1. 4TB backup task of unknown structure (e.g. hundred thousand of small)
you can check it by click to the shared folder/s properties

2. from CMR disk drive to unknown disk drive inside the WD My Book
you can check the drive details by:
Code:
hdparm -I /dev/"device name"
found by
Code:
fdisk -l

3. There isn’t mentioned a method of the backup used in the task: standard backup or r-sync.

so for HB Scanning and Backup process is really heavy I/O when you have task with such large capacity, even worse when you have small files there, as was mentioned by @Rusty above.

Better for the first backup use clean re-formated disk drive (target), to get pure sequential records of the data.

Are you sure that you don’t have any other running process in the source disk group, what can heavy eat resources of your NAS? Could you pls make a screenshot?

Assumption:
follow time of your first request and confirmation of the finishing = your backup of 4TB spent 12 hours = approx. 97MB/s .... corresponding to small files in the backup task

Check pls. also Data Scrubbing of your source disk group.
 
I may gather the additional info you asked, but I'm becoming pretty sure now the issue is not the actual load/disk-activity, but simply the USB connection itself. As long as the USB HDD is connected and turned on, even without any disk access, I have issues with the Conbee Zigbee stick in the other backpanel USB port...

As soon as I ejected the drive, everything started to work as normal again.

I'm gonna check what happens if I plug the HDD in the front USB port.
 

Create an account or login to comment

You must be a member in order to leave a comment

Create account

Create an account on our community. It's easy!

Log in

Already have an account? Log in here.

Similar threads

you did excellent job so magic of the network tune is based on try/error strategy. I have few sites in...
Replies
17
Views
6,881
  • Question
Just to summarize here: did two full backup using two external hard drives, all completed, one took days...
Replies
15
Views
4,304
You can almost certainly determine what brand/model of HDD is in the enclosure using (free)...
Replies
14
Views
14,095
Unfortunately, HB does not support QC. It's either, a public IP, DDNS, or LAN IP.
Replies
1
Views
368
  • Question
That will work with 4.1 version. Looks like user feedback has been implemented.
Replies
4
Views
1,207
You are welcome! Glad that you got it eventually working. You can still create a Synology support ticket...
Replies
11
Views
1,972
Well in a way this is expected. Your DSM6 machine (host) is still on an older version and anything other...
Replies
1
Views
650

Welcome to SynoForum.com!

SynoForum.com is an unofficial Synology forum for NAS owners and enthusiasts.

Registration is free, easy and fast!

Back
Top