Tell me about your arrays, pools, drives and strategy.

[Phone Guy]

@fredbert has sparked my curiosity in another thread about pool size, array type and layout, etc... so those of you reading this, share with us what your pools, sizes, array type, and strategy behind it. I think we could all get some insight behind other peoples nas plans. I am sure many of you are like me and still learning, and others are probably way more experienced at this than we are.

My main NAS

1. ds1621+ with 6 bays
2. Currently has 6x 4tb wd red nas drives (plus and pro only no smr efax series)
3. All setup as a single volume pool on a SHR1 array.
4. Inside volume1 I have many shared folders appropriately labelled as to their contents or use case. I only recently realized when I created those folders I did not check and enable data integrity for data scrubbing because I didnt know what that was, I later discovered thats a feature for the nas to help prevent "bit rot", and also performs the data scrubbing on improper shut down and other times I'm sure?
5. I didnt have any stragety, its just 1 big pool and volume because I had no strategy
6. I am using Photos for phone picture backups (formerly moments) and my whole family uploads pictures whenever connected to my wifi.

OK Your Turn!

 

[Coop777]

Primary NAS:

1. DS1819+ 7 x 10 TB HGST deskstar drives configured SHR-1, single volume btfrs formatted.
2. Data scrubbing is scheduled monthly
3. Folders labeled with their content - music, video, backups, etc

Secondary NAS:

1. DS1815+ 6 x 8 HGST Ultrastars, 1 x 10 10 Ultrastar. Each drive is an independent storage. They are a all refurbed drives so did not configure as RAID/SHR
2. No data scrubbing
3. Serves as a full backup of my DS1819+, files backed up manually

 

[Rusty]

718 - RAID1 (2x500GB SSD)
918 - same as 718, 2 more bays empty atm
3614 - 12x4TB ent (RAID6 + hot spare)
3614 - 4x4TB red (RAID5), 4x3TB (RAID5), 4x4 cold spare

 

[Jan Janowski]

I wish I could say I’m IT knowledge-able! (But I'm not!)
I’m making this up as I go along!
So far: done 2X: SHR upgrades: 2-4TB and 4-6TB.
In future I’ll be doing a 1-“XTB” when money and sales end up in the same place simultaneously!

 

[Jan Janowski]

In about 1976……. Heavy as Hell... It took up maybe 10-15 Rack Units of rack space, and I think it was 500KB, and was used on a Data General Computer with 16K of core Memory, for video editing with multiple 2" Ampex Quadraplex Machines.. The most exciting thing about it was the Llambda Power supplies that were SO HOT--You'd burn yourself if you touched them!! That pre-dates a bunch of Fujitsu drives on Quantel Paintbox’s in early 1980’s (only did picture storage)! Followed by Quantel: "Hal", "Harry", "Henry" SD Video Editing systems (By then the drives were down to about 40-60 pounds in weight per drive), Until "Henry" where SCSI Raid was adopted, and storage was by combinations of 2 drives: One Drive per Video Field [NTSC is Interlaced: two fields per video Frame] ). Abekas had some great stand-alone products for SD Analog, D1, D2 I/O based on Drives who's MFG had a firmware issue that was so widespread (remember swapping proms about 6-7 times per drive, and each was better than previous, but not fully correct, that the Drive company failed, taking Abekas with them, as they were in bed with ONLY that one drive MFG, and their unique disk I/O!
Side Story: Bad sectors on drives started out as a Prom on physical drive that you'd have to remove, copy, edit with new "Bad Frames Map", re-burn, and replace in drive..... if done in field, which was then changed to a software program on the System...
Then, of course, there were more than a few Discreet Logic Video Editing systems with their Custom Storage Raid Systems that were so 'locked down' by their own company paranoia as to Drives,license, and support with their unique Software & Firmware on EVERYTHING -- that they became a target for Replacemnt from multiple other companies...... Like Adobe, Apple, Avid (which became nearly as Paranoid on their systems!).. Drives on those systems numerous but usually Seagate, or companies that do not exist now.

Mid 1990's, I made 2x Home-made SCSI I, then SCSI III 160 MC 8 Drive Raid 5 chassis and arrays for early W95R2 then XP PC video editing system(s).. 2000's it was revised to 1 Spinning SATA array, revised upward to 7200 RPM Drives... Revised Yet again about 2006-7 to Hot Swap chassis in Supermicro server (XP PRO/Vista Pro/W7-- That to this day: boots W7 off an external SCSI III 320Mc drive, (leaving entire SATA Buss throughput for Video editing) with Spinning and SSD Raid 0's as increased Speed was needed for video storage and HD editing.

Use of NAS is a fairly new development for me, as IT Rules and Regulations are like learning a new language: NAS Used purely for archive use (It's not fast enough for video editing)... Prior to NAS: Archive was accomplished with External USB Drives connected as needed....
7200 RPM Ironwolf is NAS drive of choice, but one NAS has 2x 8 year old 7200 Barracuda's (with patched firmware) retired from Raid 0 use in Supermicro, when spinning drives were replaced with SSD's...

 

[fredbert]

DS215j - 2x Basic (3TB WD [CMR] Red)
DS218+ - SHR-1 (2x 4TB WD [CMR] Red)
DS1520+ - SHR-1 (4x 8TB Ironwolf + 1x hot spare)

Sometimes I replace the hot spare with a temp drive if I need a short term store. And I know hot sparing puts wear/time on the drive.

 

[RonV42]

DS1019+ - SHR-1 (4x 4 TB Ironwolf + 1 hot spare + one cold spare on the slelf)

 

[Robbie]

RS1221+ (8 bays)

• Bays 1 to 6, volume 1:
  • 6 x 4TB SSDs
  • High Performance volume
  • SHR 1 (RAID 5)
• Bays 7 & 8, volume 2:
  • 2 x 18TB 7.2k HDDs
  • Internal Hyper Backup Volume for SSD array
  • SHR1 (RAID 1)

RS819 (4 Bays)

• Bays 1 & 2, volume 1:
  • 2 x 1TB SSD
  • Performance volume
  • SHR1 (RAID1)
• Bays 3 & 4, volume 2:
  • 2 x 14TB 5.4k HDDs
  • Hyper Backup Volume #2 for RS1221+
  • SHR1 (RAID 1)

RS217 (2 Bays)

• Bay 1, volume 1:
  • 1 x 500GB SSDs
  • Performance and Test Volume
  • SHR1
• Bay 2, no volume:
  • Space for disk recovery or pre-use drive testing

DS1517+ (5 Bay)

• Bays 1 to 5, no volume:
  • No drives / air gapped archives
  • Disaster recovery unit only

I think that covers it. Looks more complicated than it is.

️

 

[Phone Guy]

Use cases?
Mine is backup for Pictures (phones mainly a few scanned), music (though I dont know why, we pay for spotify anyway), media (emby/plex - lots and lots of media ) trying to setup backups from my computers to backup, but havent gotten there yet... I do have a couple of c:/ drive backups I made manually with Macrium stored, from a couple of pc's.

 

[marco]

@Phone Guy since for multimedia you don't need versioning, I suggest to evaluate rsync command to copy only the new files

 

[Telos]

since for multimedia you don't need versioning, I suggest to evaluate rsync command to copy only the new files

Be aware that if you choose to use rsync, any corruption of the source material will overwite your so-called backup, and it too will become corrupted.

It is safer to use versioning. If a second version is never created then no additional space will be used on your backup.

 

[marco]

Versioning make sense for documents or small files, not for large video files.

On my case I run manually rsync, when I have done enough changes, and I don't replace files with different content or timestamp, but only files with different name. If I modify a file and I leave the same file name, then I also manually remove it from the backup, to allow rsync to recopy it later.

I have the two NAS (data on Synology, and backup on Raspberry), on two different place, connected via VPN.

The command I use:

rsync --no-perms --omit-dir-times --recursive --ignore-existing --delete --verbose --progress "//192.168.x.x/video" "//192.168.y.y/video"

 

[Phone Guy]

Versioning make sense for documents or small files, not for large video files.

On my case I run manually rsync, when I have done enough changes, and I don't replace files with different content or timestamp, but only files with different name. If I modify a file and I leave the same file name, then I also manually remove it from the backup, to allow rsync to recopy it later.

I have the two NAS (data on Synology, and backup on Raspberry), on two different place, connected via VPN.

The command I use:

rsync --no-perms --omit-dir-times --recursive --ignore-existing --delete --verbose --progress "//192.168.x.x/video" "//192.168.y.y/video"

Using that command ask a scheduled task I assume?
How would be different from weekly folder sync?

 

[marco]

To run this command from Windows, just install the Cygwin terminal from Cygwin Installation
Make sure that during the install procedure you include rsync from the list.

Me I created a backup.cmd file that I manually run, and it contains:

@echo off
C:\cygwin64\bin\bash.exe --login -c "cd c:/cygwin64/scripts && ./rsync.sh"

And on the c:/cygwin64/scripts/rsync.sh file I have:

#!/bin/bash
rsync --no-perms --omit-dir-times --recursive --ignore-existing --delete --verbose --progress "//192.168.x.x/video" "//192.168.y.y/video"

Of course the IP addresses must be changed.
The --delete option, if omitted, it will not remove on the target path ("//192.168.y.y/video"), any files and directories not present on the source path ("//192.168.x.x/video").

The source path is never changed.

What is weekly folder sync?

 

[Phone Guy]

What is weekly folder sync?

Because I had issue with HB rsync single copy between 2 nas, I am using folder sync between 2 nas (control panel>file service>advanced>folder sync)
choose my source folders on source nas (pictures,movies,documents,etc)
set my target as nas2
set schedule to run sunday at 2am (or whatever)

so the first sync took a while, but it created all the selected folders from source nas1 on target nas2. Then the next time it sync's it only transfers differences (is what I am lead to believe). so if I delete from nas1 its deleted from nas2 (not ideal). if nas1 is corrupted/compromised syncs that too, again not ideal, which is why its only once a week. But any new files get added to existing directories within the selected sync'd folders. It was the only way I could get it working.

Otherwise using HB rsync single copy, source nas1 has 8tb of data (maybe more now), and when transfering to target nas2 it would send data until target nas2 (14tb) was filled up and eventually failed, no one offered any solutions so after days of failed attempts I gave up defeated, and switched to folder sync on weekly schedule. I documented it as well as I could with pictures in this thread below.
Edit to add: source nas1 6 bay 6x 4tb (18tb usable 8tb used) .... target nas2 5 bay 5x 4tb (14tb 0tb used)

HB rsync single copy backing empty space?

I am trying to setup a hyperback rsync single copy mode between 2 synology nas. Source has 6x 4tb SHR1, Target has 5x 4tb SHR1. Source has

www.synoforum.com