Help please - Data Migration to new DS1821+ - Best Practice for Speed and Redundancy?

Currently reading
Help please - Data Migration to new DS1821+ - Best Practice for Speed and Redundancy?

2
0
NAS
DS1821+
Operating system
  1. macOS
  2. Windows
Mobile operating system
  1. iOS
Last edited:
How should I set this up for:
1) Fastest possible aggregation /syncing / copying of various types of data from various sources to my brand new NAS.
2) Future reasonable speed reads, especially for doing photo processing and cataloging with Lightroom.
3) A reasonable level of data security and redundancy, within the confines of the hardware I just acquired, or already have. If I do need something else, like a PCIe or SSD M2 cache, to make this work, I'm open to that. I just don't want to buy 4 more hard drives to double copy everything.
4) Be able to increase the hard drive sizes as new ones become available and affordable, without losing the space to the lowest common denominator
5) Dream: No more Mac dot double files! (not just the .DS_Store files; I'm talking about the dot double files you have to use dot_clean to get rid of or merge...which doesn't always work, and can take a long time)

Here's the equipment and software I have:
* A brand new Synology 1821+ and 4 16TB Toshiba drives.
* A 16TB Exos drive in a hard drive enclosure. It's already got stuff on it; I've had it for maybe a year.
* NetGear ReadyNas 214 with 4 HGST Hard drives (8TB), Raid 5, 21 out of 24TB used.
* Sooooo many portable hard drives. Sooo many. Lots of duplicate data on them from fear-copying. Randomly organized, randomly duplicated. Such bad practices on my part.
* ChronoSync and Gemini II that I was/am intending to use to try to sync and organize my hot mess.
* Cat5 ethernet cabling running through the walls of my house.
* Brand new NightHawk RAX48 router, to which I've connected both RJ45 cables that came with the drive.
* Older Netgear ethernet hub.
* Fairly new-ish Windows Laptop for my jobby job
* 2020 MacBook Pro for my jobby job
NOTE: I *can* use these two computers for a limited amount of personal stuff, but I would rather not unless necessary.
* 2015 Macbook Pro that's my personal computer
* NOT HERE YET: 2021 MacBook Pro Max - on order - should be here at the beginning of December.
* Software: ChronoSync and ChronoSync Agent / Gemini II

Data:
I have 64-ish TB of photos, music, compositions, writing, video, documents and code. As I mentioned earlier, a decent amount of duplication and a "beautiful minds" amount of disorganization. To give you an idea of the quantity of, for example, photos - I have over 700,000 of them. Over 500,000 of them are RAW. I also have artifacts from my web dev agency that I owned for 12 years. The business artifacts are all archivable. I don't need to access those really.

Because I've been primarily Mac based, I have an untenable number of those double dot files it would be delightful to purge.

Skill Set: I've been in Tech for 20+ years, but mostly on the Dev / DBA side of the house. Not really networking, not really hardware. Also, I'd rather not spend the rest of my days getting this project done.

Process Ideas:
* Dump everything on the new NAS and sort and aggregate it from there.
* Use the 16tb drive that already has some stuff on it, as a scratch place to organize the directory structure, and sync the little portable drives to, getting rid of duplicates, etc., during that process. Then sync THAT drive up to the NAS. After a sync is done to a particular area, clean that off my 16tb scratch portable hard drive, so I can use it for the next little portable sync organization. the advantage of this, to me, is that I'm not waiting through RAID5 indexing while I'm also trying to sync and organize. Getting it from the scratch drive to the NAS could potentially run when I'm doing my jobby job during the day.

Based on the small amount of syncing I've already tried to do this week to get the first NAS a little cleaned up, getting this data up there and organized feels like a rest-of-my-life project.

What kind of additional hardware, cabling, cards might speed this up? Should I use SHR, or what kind of RAID or equivalent should I choose? Is there a way to stop those dang double dot files from getting created?

What kind of strategy makes sense here so I don't die on this data migration hill? I really want to do this migration once in my lifetime and have it hold up, except for replacing hardware that ages or fails.

Thank you for reading this if you have, and for any help you can give. Also, if it's in the wrong topic, where should it be? I'm new here (clearly).
 

Create an account or login to comment

You must be a member in order to leave a comment

Create account

Create an account on our community. It's easy!

Log in

Already have an account? Log in here.

Similar threads

  • Question
Just make sure not to mistake SHR for BTRFS. One is raid type the other is the file system. Also...
Replies
3
Views
1,844
You could power down the NAS and swap the disks around. See what happens when it powers up. My guess is...
Replies
12
Views
2,051
Also bear in mind that if you go 2x RAID1, all your apps will be installed in the volume on the first RAID...
Replies
12
Views
2,349
  • Solved
Fantastic! for me it was something in the upgrade process that messed with it. No matter a format and...
Replies
5
Views
693
I have found the section where this topic on the DS 415+ clearly had a known issue that blighted way too...
Replies
3
Views
2,348

Welcome to SynoForum.com!

SynoForum.com is an unofficial Synology forum for NAS owners and enthusiasts.

Registration is free, easy and fast!

Back
Top