new to NAS got ds418 to backup 5 million files btrfs or ext4?

Currently reading
new to NAS got ds418 to backup 5 million files btrfs or ext4?

Status
Not open for further replies.
M

morthanley

i have about 5,000,000 small files (<100 k) and 100,000 large files (>100 megs) and I cant decide if i should choose btrfs or ext4 on my new ds418. I plan on using RAID1 for now (2 drives). im afraid of how long its going to take to copy the files over smb with robocopy from a windows pc internal drive (wd blue) and a usb drive (wd elements). part of me wants to use btrfs+shr and another part of me wants to use ext4+raid1
 
Rationally, you must decide if you want. Balance the pro and contra:
+BTRFS:
  1. Snapshot replication, probably only needed if you need the uptime and flexibility in business applications and sent the snapshots to other NAS.
  2. Data integrity is better, based on checksum technology
  3. Somewhat better disaster management
-BTRFS:
  1. takes more overhead (-5% available disk space)

Ext4 SHR is fine for almost every SOHO user.
I think both do a good job. Snapshots are probably the most convining benefit, if you use it.
 
I use BTRFS + RAID1

I wouldn't use anything else than BTRFS (or ZFS) for valuable data,
except I had another way to validate data integrity (like doing checksuming with another application)
 
absolutely agree with answers from Gents above

re your concern about speed of transfer for such data samples from the PC to the NAS:
1. the internal drive:
- you have 5 specific bottlenecks
- first is the big portion of small files (what is absolutely normal from operation point of view). You will need big support from all below mentioned parts. Instead you will get really slow speed. Don’t be surprised.
- second is your PC architecture (not mentioned)
- third is a setup of your LAN (not mentioned) ... you have an option to use LACP with two 1Gps NICs in the NAS for faster transfer. Only if you have support in uour PC and LAN (switch).
- forth is the NAS itself, specially the CPU (Realtek) inside in your NAS. Because all the data handling is performed by the CPU (ethernet, sw RAID, BTRFS, ...).
-fifth - first copy will be fine. It will be pure sequential write. Next will be more complicated for your NAS CPU due the random IOPS.
Finally, used drives in the RAID. But you don’t need fast HDDs, even SSDs, because you can’t utilize them with your NAS CPU and w/o LACP.


2. the USB drive:
- it’s better to connect the drive directly to NAS
- you will get faster transfer, than through the PC
- but don’t expect a speed of light

Last point:
- you can check Syno Active backup for business (AB4B) for a comparison with the Robocopy. There are more advantage features, e.g. backup of used files, and more useful ...
 
Another small thing on data integrity protection of BTRFS: Synology advises not to use it when you use databases on the disk or when using surveillance station. It would stress your system to continuously rebuild the checksums apperently.
 
well i got my new nas from amazon today (they decided to ship it with a leaky bottle of laundry detergent!!!!) put my hard drives in and turned the machine on and it decided to do SHR with btrfs without asking me lol
 
SHR with btrfs is the default. There is a point in the initial setup that asks about setting up disks but phrases it as performance / flexibility (or suchlike). Very easy to miss and really needs a claxon and DISK SETUP in flashing lights.

i never got that choice it all happened in the background before i could do anything except create an account
 
Status
Not open for further replies.

Similar threads

Have you thought of using the DS918+ as a Hyper backup target? You can then restore any file or the whole...
Replies
3
Views
880
Thanks PunchCardBoss, Changing the subnet on the Deco to the default Netgear, basically worked. I was...
Replies
2
Views
1,065
Considering the current state, just connect to that IP address and continue with dsm installation. The...
Replies
4
Views
1,049
Does browsing for Synology Web Assistant find your NAS? That’s one way to try to find it on the LAN. Other...
Replies
3
Views
1,136
The 1621+ would be a logical replacement, I chose the DS1522+ as it is slightly newer. Answers: A)...
Replies
1
Views
1,221

Welcome to SynoForum.com!

SynoForum.com is an unofficial Synology forum for NAS owners and enthusiasts.

Registration is free, easy and fast!

Back
Top