Oh, No... DSM 7.1 *REMOVED* critical SSD/NVME Cache Functionality!!! :-(

[blogthis]

DSM 6.2 through 6.2.4 Flash-Cache (SSD/NVME) worked great.
DSM 7.01 removed only one key feature.
DSM 7.1 now works entirely differently, removing many key features while adding just one feature.

This post encourages discussion on the huge differences in these DSM Flash-Cache features now removed....

Synology DSM 7.0.1 Update-3, worked great with the Synology 10Gb Ethernet and M.2 Adapter Card E10M20-T1, RJ-45; 1 Port SSD/10G Combo-Card on my older DS3617xs.. It *WAS* awesome while it lasted (DSM 7.01 was first to support E10M20-T1 in DS3617sx):

With DSM 7.0.1, Synology's cache system and specifically in my case using this combo-card let you create two separate cache types for use in the same Storage Pool #1: One from SATA-SSD's for Cache #1 to use in Volume #1, and then Second Cache #2 from NVME-SSD's Combo Card Volume #2, for use on *SAME* Storage Pool... BUT NOT anymore!

MAJOR FUNCTIONALITY DELETED DSM 7.1 (using released Update-1), that directly adversely affects Synology's cache strategy and somewhat KILLS my config and this E10M20-T1 combo-card for me, as follows.... :-(

Now, Storage Pool #1 consisting of SATA SSD's and another consisting of the NVMe SSD's this combo-card uses, NOW must use separate PHYSICAL DRIVES to create a MULTIPLE Storage Pool #2 (not just Volume #2). For clarity: NO: You CANNOT just use free space from your existing Storage Pool #1 to use for a Cache Pool #2 to keep using this comb-card. Ouchies.

Also Now, you get an ERROR that Cache Pools cannot contain both SATA-SSD and NVME-SSD. They must be of the same type. This is a HUGE problem for my config, which was fine in 7.0.1 but is now totally broken for my use in 7.1..

NOW: Synology has now posted this info in its official DSM 7.1 "SSD Cache release notes new for 7.1", to try and slow down confused customers like me, asking Synology's Tech Support what is going wrong. It's not wrong: It's a HUGE design change.

Literally, what was working great in DSM 7.0.1 is now IMPOSSIBLE in DSM 7.1 (same storage-pool with multiple volumes cached by different cache types (e.g. SATA/NVME), is NO LONGER SUPPORTED).

Basically, Synology's decision to KILL their best 2 caching features AND this card with these harsh DSM 7.1 limitations being UNABLE to use it for multi-volumes in a *SINGLE* mixed storage pool of SSD and NVME now, is really bad for my uses...

So, now I'm losing the expensive NVMe flash cache for my other volume(s), because I cannot afford to just add PHYSICAL storage bays with expensive EXTRA PHYSICAL HDD's on a whim to accomdite this DSM software change.... This really hurts.

I sure hope Synology puts having MULTIPLE cache pools of a differet type, for a SINGLE storage pool's multiple Volumes, back on their development roadmap ASAP!!

Don't believe me? Here are the links to official Synology articles on this obsecure yet to me very important topic.

DSM 6.2 to 6.2.4 Cache functionality:
Storage Pool:
Create a New Storage Pool | DSM - Synology Knowledge Center
Flash-SSD/NVME Cache:
SSD Cache | DSM - Synology Knowledge Center
Support for Multiple-Volumes SSD caching in same Pool = YES.
Support for caching in-transit files bigger than 1mb (aka Sequential I/O) = YES.
Support for mixing different Cache-Types in same Storage Pool = YES.
Support for Cache Pool of Same-Type to be split-up to Volumes on Same Storage Pool = NO.

DSM 7.0.1 Cache functionality:
Support for caching in-transit files bigger than 1mb (aka Sequential I/O) = NO!
Note: This interim version of DSM had other cache problems they hot-fixed by disabling this key feature, but it's only in various user-forums and not really formally addressed by Synology articles. All other features from DSM 6.2.x are the same.

DSM 7.1 Cache functionality:
Storage Pool:
Create a Storage Pool | DSM - Synology Knowledge Center
Flash-SSD/NVME Cache:
Create an SSD Cache | DSM - Synology Knowledge Center
Support for Multiple-Volumes SSD caching in same Pool = NO!
Support for caching in-transit files bigger than 1mb (aka Sequential I/O) = NO!
Support for mixing different Cache-Types in same Storage Pool = NO!
Support for Cache Pool of Same-Type to be split-up to Volumes on Same Storage Pool = YES.

.....I did not know any of this at the time, and after I followed Synology's recommendation to delete and recreated my flash-caches, now it's too late because Synology prohibits anyone to downgrade DSM in order for me to get these HUGE features back that are now GONE.. :-(

Edit: Clarified DSM 7.0.1 worked almost-the-same but not exactly-same as DSM 6.2.x, as this is quietly undocumented by Synology.

 

[Robbie]

I am beginning to think that, with their march to their own brand of drives, Synology DSM drive support is going all pear-shaped.

️

 

[blogthis]

I am beginning to think that, with their march to their own brand of drives, Synology DSM drive support is going all pear-shaped.

️

The DSM Flash-Cache with Synology drives, works the same as any other brand of drive.
It's the version of DSM that keeps removing key caching features, not the drive itself.

 

[jeyare]

So, you need to go back to the past when the Flashcache was invented by Facebook, similarly also BTRFS. It was long time ago.

The project is officially abandoned (2014) and just few companies continue - Syno, NetApp.

I have idea that some people from Syno, which was responsible for storage architecture or sw analysis leave Syno and now they can’t continue. Similarly to the case with SMART integration of Smartmontools.
This is just my opinion.
Somewhere must be a reason. I don’t believe that they just switched off the features between minor (again minor) version update.

 

[Ewie]

SSD Cache Hardware errors [not sure if this post is to correct location]

Message: DSM 7.1 *REMOVED* critical SSD/NVME Cache Functionality​

Has anybody had SSD Cache hardware problems? I have had THREE.

I added a single SSD Cache, but then learned that only provided READ cache, so I added the second to add WRITE cache before Ver 7 update. Then after some time with Ver 7 update the operating system flagged hardware problem - no problems ... returned and got a warranty replacement. A month or two later DSM 7 flagged the other SSD as a problem and as a safety measure, downgraded the SSD Cache drive.
Again, no problem just return an get a warranty replacement. This time the replacement took longer due to current supply chain hold ups, and just after it arrived in the mail before I installed this replacement, NOW the other SSD cache has been downgraded by DSM 7, Gasp! If I replace the newly arrived item, I will still only have one SSD available and a lengthy replacement cycle AGAIN.

So my question is : Am I just having bad luck or do the read/write cycles count hit a limit ?? Maybe some other reason?

Any similar experiences?

Hardware is:
SAMSUNG 970 EVO PLUS 250GB M.2 NVME SSD MZ-V7S250BW Samsung 250GB SSD, 970 EVO Plus, M.2 2280 NVMe PCIe Gen3, Read up to 3,500MB/s, Write up to 2,300MB/s, IOPS R/W up to 250/550K, 1.5M Hours MTBF, 150TBW 5 Year/s Warranty

 

[Robbie]

The DSM Flash-Cache with Synology drives, works the same as any other brand of drive.
It's the version of DSM that keeps removing key caching features, not the drive itself.

However you want to put it, Synology's direction is to remove or reduce support for other manufacturers drives and limit the use of 'their' drives to Synology NASes only. Synology is both vendor-locking and feature-locking to their badged brands. There is plenty of evidence to support that and read up on, including on this forum.

️

-- post merged: 5. Jul 2022 --

So my question is : Am I just having bad luck or do the read/write cycles count hit a limit ?? Maybe some other reason?

Did you record the SMART data on the SSDs before you sent them back?

️

 

[Ewie]

Hi Robbie, Thanks for your reply. Not aware of how to record SMART data, but I have the existing 'SSD had degraded ... Automatic Protection Mechanism...'
Previously I just followed the Synology direction on how to remove and replace the SSD Cache.
Is there another step to record the SMART Data?

Secondly, should the plan be to change to Synology hardware?
(The existing hardware is all there with a warranty, but hard to appreciate the Cache benefit given how long its been degraded so often without both SSD devices operating concurrently)

 

[jeyare]

smartctl -x <for the specific drive>

or read here, you will find also a path to historical SMART logs and so on:

DSM 7.1 - DSM7.1 introduces new drive vendor-locking parameters

How unfortunate am I to have 6 (non-Synology) SSDs to all go from perfect health to 'Critical' with the completely coincidental installation of the latest DSM7.1. No matter if the drive is young or older, no matter what batch they are from or how many bytes written. Or, is it a new way of...

www.synoforum.com

 

[Ewie]

I must admit confusion reigns here....

1. Widget gives System Health warning - go to Storage Manager for more information
2. Storage Manager - SSD Cache 1 - Warning Not the latest version. I/O acceleration Disabled... now you can choose to repair or remove the cache
3. Storage Manager HDD/SSD : SSD Displays Green Tick "Normal"
4. Storage Manager : Action > Deactivate (to remove and repair etc) Message:
"Unable to Deactivate this drive because SSD Cache 1 has reached its maximum drive fault tolerance. Deactivating the Drive will cause SSD Cache 1 to crash."

From that I understood that the SSD should be removed and reinstated with the 7.1 format, but that action appears blocked.
Gasp ! Seems a circular problem? No sure of a solution. The system is still working ok without the SSD Cache routine.

 

[Dvoyles]

I must admit confusion reigns here....

1. Widget gives System Health warning - go to Storage Manager for more information
2. Storage Manager - SSD Cache 1 - Warning Not the latest version. I/O acceleration Disabled... now you can choose to repair or remove the cache
3. Storage Manager HDD/SSD : SSD Displays Green Tick "Normal"
4. Storage Manager : Action > Deactivate (to remove and repair etc) Message:
"Unable to Deactivate this drive because SSD Cache 1 has reached its maximum drive fault tolerance. Deactivating the Drive will cause SSD Cache 1 to crash."

From that I understood that the SSD should be removed and reinstated with the 7.1 format, but that action appears blocked.
Gasp ! Seems a circular problem? No sure of a solution. The system is still working ok without the SSD Cache routine.

I have the exact same problem.

I have no way of enabling this SSD, so now I'm down to 1. System seems to be working OK, but I went from having 2 SSDs to 1, per Synology's recommendation that I update the cache.
Crazy that they don't have a fix for this.

 

[blogthis]

Is the status "I/O Acceleration Disabled"? If so, that sounds to me like the cache is in read-only mode or entirely offline. As long as it's NOT in read-write, then it can drop and not crash your volume (in theory).
If DSM allows you to safely fully remove the cache, that's what you want to do, so you can then recreate it.
If it's giving alarming messages like crashing the cache, then you'll have to contact Synology support to have them remote connect to take your cache offline to remove it and then recreate it with new SSD's.
Note that Synology support may give you a hard time if you're not using a supported SSD on your units compatibility list, so... if they do, I'd suggest telling Synology support you want to remove your consumer-grade SSD's, so in fact you can then install ones on the compatibility list (so they're more inclined to give you support).

-- post merged: 15. Nov 2022 --

Hi Robbie, Thanks for your reply. Not aware of how to record SMART data, but I have the existing 'SSD had degraded ... Automatic Protection Mechanism...'
Previously I just followed the Synology direction on how to remove and replace the SSD Cache.
Is there another step to record the SMART Data?

Secondly, should the plan be to change to Synology hardware?
(The existing hardware is all there with a warranty, but hard to appreciate the Cache benefit given how long its been degraded so often without both SSD devices operating concurrently)

You do not want to use cheap Samsung EVO's. They don't even have power loss protection (PLP), and are not made for being in RAID configurations (eg. as Raid1 Mirrored Cache).
If you want to deviate from the expensive Enterprise drives on Synology's supported list, then you'll at least want to find SSD's with PLP feature such as the older thus more affordable Intel Enterprise SSD's.

Also: Note that you cannot mix SATA SSD with NVME SSD as cache. That was one of my primary complaints in original post.

 

[Dvoyles]

I guess my concern is that the M.2 NVMEs have been working fine until I followed Synology's suggestion of updating the cache. I understood that the SSD should be removed and reinstated with the 7.1 format, from 6.X.

• Now I cannot do anything with that deactivated drive.
• The other M.2. NVME continues to work fine.
• When I attempt to disable that one, Synology throws :

"Unable to Deactivate this drive because SSD Cache 1 has reached its maximum drive fault tolerance. Deactivating the Drive will cause SSD Cache 1 to crash"

So with their latest update Synology has restricted which drives you can enable. Crazy.

 

[Dvoyles]

I have a fix.
Restart the machine.
It enables me to fix the SSD cache.

 

[Jan Janowski]

Late to party, but a year ago tossed 2x Seagate 510’s & Samsung consumer (Synology supported) NVME’s back & forth between two 720+’s as R/W Cache a few times trying to determine if twice the cache space equated to an observable speed difference…. and never saw any issues. Now both NAS’s on -2, and I have never since gone further than Healthy Cache prompt in Storage manager on either 720+….. Am I missing something?

 

[blogthis]

smartctl -x <for the specific drive>

or read here, you will find also a path to historical SMART logs and so on:

DSM 7.1 - DSM7.1 introduces new drive vendor-locking parameters

How unfortunate am I to have 6 (non-Synology) SSDs to all go from perfect health to 'Critical' with the completely coincidental installation of the latest DSM7.1. No matter if the drive is young or older, no matter what batch they are from or how many bytes written. Or, is it a new way of...

www.synoforum.com

There is a freely available GitHub project that ADDS ALL YOUR DRIVES to the "compatibility" database list. This removes all errors and re-enables 100% of all functionality:

GitHub - 007revad/Synology_HDD_db: Add your HDD, SSD and NVMe drives to your Synology's compatible drive database

Add your HDD, SSD and NVMe drives to your Synology's compatible drive database - 007revad/Synology_HDD_db

github.com

-- post merged: 21. Jan 2024 --

DSM 6.2 through 6.2.4 Flash-Cache (SSD/NVME) worked great.
DSM 7.01 removed only one key feature.
DSM 7.1 now works entirely differently, removing many key features while adding just one feature.

This post encourages discussion on the huge differences in these DSM Flash-Cache features now removed....

Synology DSM 7.0.1 Update-3, worked great with the Synology 10Gb Ethernet and M.2 Adapter Card E10M20-T1, RJ-45; 1 Port SSD/10G Combo-Card on my older DS3617xs.. It WAS awesome while it lasted (DSM 7.01 was first to support E10M20-T1 in DS3617sx):

With DSM 7.0.1, Synology's cache system and specifically in my case using this combo-card let you create two separate cache types for use in the same Storage Pool #1: One from SATA-SSD's for Cache #1 to use in Volume #1, and then Second Cache #2 from NVME-SSD's Combo Card Volume #2, for use on SAME Storage Pool... BUT NOT anymore!

MAJOR FUNCTIONALITY DELETED DSM 7.1 (using released Update-1), that directly adversely affects Synology's cache strategy and somewhat KILLS my config and this E10M20-T1 combo-card for me, as follows.... :-(

Now, Storage Pool #1 consisting of SATA SSD's and another consisting of the NVMe SSD's this combo-card uses, NOW must use separate PHYSICAL DRIVES to create a MULTIPLE Storage Pool #2 (not just Volume #2). For clarity: NO: You CANNOT just use free space from your existing Storage Pool #1 to use for a Cache Pool #2 to keep using this comb-card. Ouchies.

Also Now, you get an ERROR that Cache Pools cannot contain both SATA-SSD and NVME-SSD. They must be of the same type. This is a HUGE problem for my config, which was fine in 7.0.1 but is now totally broken for my use in 7.1..

NOW: Synology has now posted this info in its official DSM 7.1 "SSD Cache release notes new for 7.1", to try and slow down confused customers like me, asking Synology's Tech Support what is going wrong. It's not wrong: It's a HUGE design change.

Literally, what was working great in DSM 7.0.1 is now IMPOSSIBLE in DSM 7.1 (same storage-pool with multiple volumes cached by different cache types (e.g. SATA/NVME), is NO LONGER SUPPORTED).

Basically, Synology's decision to KILL their best 2 caching features AND this card with these harsh DSM 7.1 limitations being UNABLE to use it for multi-volumes in a SINGLE mixed storage pool of SSD and NVME now, is really bad for my uses...

So, now I'm losing the expensive NVMe flash cache for my other volume(s), because I cannot afford to just add PHYSICAL storage bays with expensive EXTRA PHYSICAL HDD's on a whim to accomdite this DSM software change.... This really hurts.

I sure hope Synology puts having MULTIPLE cache pools of a differet type, for a SINGLE storage pool's multiple Volumes, back on their development roadmap ASAP!!

Don't believe me? Here are the links to official Synology articles on this obsecure yet to me very important topic.

DSM 6.2 to 6.2.4 Cache functionality:
Storage Pool:
Create a New Storage Pool | DSM - Synology Knowledge Center
Flash-SSD/NVME Cache:
SSD Cache | DSM - Synology Knowledge Center
Support for Multiple-Volumes SSD caching in same Pool = YES.
Support for caching in-transit files bigger than 1mb (aka Sequential I/O) = YES.
Support for mixing different Cache-Types in same Storage Pool = YES.
Support for Cache Pool of Same-Type to be split-up to Volumes on Same Storage Pool = NO.

DSM 7.0.1 Cache functionality:
Support for caching in-transit files bigger than 1mb (aka Sequential I/O) = NO!
Note: This interim version of DSM had other cache problems they hot-fixed by disabling this key feature, but it's only in various user-forums and not really formally addressed by Synology articles. All other features from DSM 6.2.x are the same.

DSM 7.1 Cache functionality:
Storage Pool:
Create a Storage Pool | DSM - Synology Knowledge Center
Flash-SSD/NVME Cache:
Create an SSD Cache | DSM - Synology Knowledge Center
Support for Multiple-Volumes SSD caching in same Pool = NO!
Support for caching in-transit files bigger than 1mb (aka Sequential I/O) = NO!
Support for mixing different Cache-Types in same Storage Pool = NO!
Support for Cache Pool of Same-Type to be split-up to Volumes on Same Storage Pool = YES.

.....I did not know any of this at the time, and after I followed Synology's recommendation to delete and recreated my flash-caches, now it's too late because Synology prohibits anyone to downgrade DSM in order for me to get these HUGE features back that are now GONE.. :-(

Edit: Clarified DSM 7.0.1 worked almost-the-same but not exactly-same as DSM 6.2.x, as this is quietly undocumented by Synology.

There is now a freely available GitHub project to "FIX" this issue, by re-enabling, post-installation, the FULL and REAL read-write aka "Streaming-I/O" SSD cache in a script (because checkbox was removed by Synology from GUI):

GitHub - 007revad/Synology_enable_sequential_IO: Enables sequential I/O for your SSD caches

Enables sequential I/O for your SSD caches. Contribute to 007revad/Synology_enable_sequential_IO development by creating an account on GitHub.

github.com

 

[Coop777]

Keep in mind this is technically a "hack" that will likely void warranty support.

 

[007revad]

There is now a freely available GitHub project to "FIX" this issue, by re-enabling, post-installation, the FULL and REAL read-write aka "Streaming-I/O" SSD cache in a script (because checkbox was removed by Synology from GUI):

GitHub - 007revad/Synology_enable_sequential_IO: Enables sequential I/O for your SSD caches

Enables sequential I/O for your SSD caches. Contribute to 007revad/Synology_enable_sequential_IO development by creating an account on GitHub.

github.com

I've released a new version of GitHub - 007revad/Synology_enable_sequential_IO: Enables sequential I/O for your SSD caches that solves an issue a few people had where Sequential I/O was disabled after a reboot. The script should be scheduled to run at boot with the --volumes option.