How well does an SSD Cache work?

[Rusty]

Hi, that means: 2 SSDs in RAID1, same size: 1TB each , and choose the mode: Read/Write Cache setup when configuring the cache.

-- post merged: 27. Nov 2020 --

(Buy 3, 1 on your shelf in storage) So when SSD drive X dies, you just swap it in and it will rebuild the RAID1.

Wasn’t this already discussed?

 

[Exsosus]

Hi Rusty,
If you do see redudancy come from me with repeated posts its not intentional, is most likely, rather, a long work-week that has come to an end.

Apologies to moderators and admins of this site.

 

[Rusty]

Hi Rusty,
If you do see redudancy come from me with repeated posts its not intentional, is most likely, rather, a long work-week that has come to an end.

Apologies to moderators and admins of this site.

Dont worry about it. I was just wondering if I said something inaccurate that’s all. Thank you for your input. Your opinion and knowledge is valuable just like any other member on the forum.

 

[chenks]

The screenshot has 4.66 TB for today. That’s why it recommends 4.8 TB for SSD cache.

well i hadn't moved or accessed 4.66TB on that day.

here is it again run just now. clearly these figures are nonsense.
31GB today? i'd like to now know how it got that figure. i haven't downloaded anything, haven't added anything to the NAS, haven't moved anything. it's basically been sitting idle other than playing a couple of TV episodes via Plex.

 

[Dave123]

Something is wrong, I can feel it!

 

[Robbie]

It is another way of saying SSD cache is not for you; at least during that assessment.

 

[speedmaster]

I have 4 harddrives in RAID10.
Filesystem is BTRFS.
2 x 1TB NVME's Samsung 970 EVO plus.
RAID1 read/write cache with BTRFS metadata pinned.

This NAS is for family use and Plex.
Cache is 100% full, and cache hit rate over the last month is 94%.
As others have mentioned, OS will feel a lot snappier, which I like/prefer.
Filetransfer between PC's and NAS depends on your usecase.
"Bigger" files won't benefit at all (I havn't been able to find the cache filesize cutoff).

Mind you, if you opt for RAID1 read/write cache Synology recomend you use UPS.
If you go for RAID1 cache, remember to go into your "Storage manager", find you SSD cache, click the 3 small dots, select "configure" and read if "enable automatic protection mechanism" is needed.

My "SSD cache advisor" gave me weird astronomical numbers too. But that was before Synology added the analyzer to run for up to 30 days.
So I worked my way through a 250GB read cache, that quickly filled up. Then a 500GB read, that also got filled up quickly.
1TB took a while to fully utilize, so I settled for that, and upgraded to read/write.
Not the most efficient way to measure cache performance, but my point is, daily cache advisory numbers was not usefull in my case.
No problems, that I know of, so far.

 

[jeyare]

there isn’t cache driven by BTRFS because LVM is the master of the storage operation in Syno BTRFS RAID architecture
but there isn't LVMcache in operation, because Syno has implemented Flashcache block system (out of support now). You can check it here:

This project is not actively maintained. Proceed at your own risk!_**

GitHub - facebookarchive/flashcache: A general purpose, write-back block cache for Linux.

The last pulled request was in 2018. OMG.

And here is another evidence from Synology DiskStation MIB Guide for DSM 7.0 and above (page 12) - The Synology FlashCache MIB monitors the resource usage of SSD cache:

https://global.download.synology.com/download/Document/Software/DeveloperGuide/Firmware/DSM/All/enu/Synology_DiskStation_MIB_Guide.pdf

I'm slowly realizing why Syno is pushing to use its disks in Rackmount NASes. Because they had to adjust the write-trough cache off directly in the disk controller, otherwise there would be duplication of the cache at the SSD cache (primary) and disk drive cache (secondary) levels. This must naturally lead to inefficiency, directly to the possibility of data loss, especially when using parity RAIDs such as 5/6. Plus write-through caches are not persistent across a device removal or a reboot.

Another kind of enterprise-driven idea?

 

[jeyare]

and the current Syno NAS SSD cache architecture wasn’t changed from DSM 5:

https://global.download.synology.com/download/Document/Software/WhitePaper/Firmware/DSM/All/enu/Synology_SSD_Cache_White_Paper.pdf

So, another reason of the old (near unsupported) Linux kernel version in the DSM7, because the flashcache is not merged into the kernel mainline (no newer kernel support).
… you need to pour a clean wine. The new DSM7 is a smoke screen foundation.

 

[Robbie]

@jeyare - Are you sure that SSD caching has not changed from DSM5?

Synology made a big deal over the changes to SSD caching for DSM7 and the white paper you have linked to above is out of date and is limited to DSM5.2.



(I don't use SSD caching as it didn't help with performance for my workload and just seemed like an expensive NAND stress-test. My NASes now have an SSD 'performance' volume in addition to HDD volumes.)

 

[jeyare]

@jeyare - Are you sure that SSD caching has not changed from DSM5?

Synology made a big deal over the changes to SSD caching for DSM7 and the white paper you have linked to above is out of date and is limited to DSM5.2.

it wasn't an error;
it was a feature
also surprised

 

[jeyare]

./usr/syno/etc/flashcache.conf
./usr/lib/modules/flashcache_syno.ko
./usr/lib/modules/flashcache.ko
./var/lib/dpkg/info/flashcache-cedarview-bin.md5sums
./var/lib/dpkg/info/flashcache-cedarview-bin.preinst
./var/lib/dpkg/info/flashcache-cedarview-bin.list
/usr/bin/flashcache_create
/usr/bin/flashcache_load
/usr/bin/flashcache_destroy
/usr/bin/flashcache_expand

license=GPL author=Mohan - based on code by Ming description=device-mapper Facebook flash cache

Include an alert that the flashcache new mode only supports 64KB block size

and signature from the GCC: crosstool-NG 1.20.0 20150311 ...... 11.03.2015 in one of the files.

and here is some description from the Synology DiskStation MIB Guide for DSM 7.0 :

flashCache MODULE-IDENTITY
	LAST-UPDATED "201407170000Z"
	ORGANIZATION "www.synology.com"
	CONTACT-INFO
	"postal: Awon Lai
	email: [email protected]"
	DESCRIPTION
	"Characteristics of the flash cache information"
	REVISION "201407170000Z"

 

[Robbie]

Not changing the MIB does not always mean that something fundamental hasn't changed though. Usually some effort goes into not changing the MIB unless you really have to to avoid breaking dependancies. It is just a look-up table so that everyone (inc developers etc) know what is there to use on the network / bus / snmp whatever.

Still, an interesting topic, especially for a brand that pushes SSD caches, doesn't permit them to be used as NVMe volumes and usually handicaps its systems with 1GbE ports and weak CPUs.

Of course, if you buy the rack mount version of a M2-equipped DS unit the first thing Synology tosses from the RS spec are the two M2 slots.

 

[jeyare]

Right.
Back to the analysis background:
- 2014 - release year of the DSM5
- the DSM5 contained new feature: SSD cache, as All-around performance booster
- the cache has been sourced from Facebook Flashcache (never defined by Syno, but you don’t need more evidences)
- same year was delivered MIB for the cache (mentioned above). Never changed.
- 2016 - release year of the DSM6
- 2018 Facebook officially cancelled the support of the Flashcache
- the Flashcache is not standard module for Linux kernels (mainline)
- then Syno operated the main I/O performance booster on the unsupported technology
- 2021 - release date of the DSM7 … nothing was changed from DSM5 in the cache booster (still same Cache Advisor, same Cache Setup)

Try to download the DSM5, put it to VM and try to find mentioned Flashcache files, you will find the same. Also same UI setup.
Don’t mention the unclear math about the Recently Accessed files (Hot/Warm), what is same from the DSM5 (just compare the link from my post above). Where is also added transfers within same disk group (tested by me).
The cache system setup used in Syno NASes is simple and fixed engine depends on drive mapper, no custom setup possible- based on operation scenarios. Useful for the home users. But when >80% of them use the NAS for video storage or single large file transfer (backup) I can’t see a usefulness in such market. Maybe a reason, why Syno don’t change the flashcache to another better sys like dm_cache. Specially in the new revolutionary DSM7 system.
Edit:
I can’t see a usefulness of the Cache also for the SynoDrive for home users, when few small files are moved across internet (low speed channel). Don’t mention about the SME segment in case of the Drive (primary document storage).
I can imagine, that OLTP exchange is <1% of the current SME Syno NASes workload. And regarding the OLAP, hm - how many of them use the NAS for a DataLakes?

 

[jeyare]

We Can Learn From Mistakes, Not From Fairy Tales

this is no longer the case for leaders in their market who have not been able (willing) to change:
Blockbuster, Kodak, Nokia, ...

 

[Robbie]

A few things did change with DSM7 SSD cache:

You can now add or remove a cache without pulling the entire volume off-line. DSM7 offers more raid options for the SSD cache itself and the skipping of sequential writes is now in the algorithm rather than a user option. All Btrfs metadata can be (and should be) pinned to the SSD cache rather than dumped-out by the vagaries of the previous algorithm.

The UI has changed and is simpler in that an non-initialised SSD offers you 4 choices for use (change raid, hot spare, create pool or SSD cache and has no questionable blinking lights graphic. Cache advisor now gives the I/O figures rather than the simple (and meaningless) maths of giving the total size of data accessed in the last 7 days as 'hot data'. I also thought the addition of a benchmark function (inc IOPS) to Storage Manager a nice touch too.

There may be other changes but as said, I don't use it as I prefer to run pure-SSD volumes.



[No idea where your fairy tails quote came from as it does not (or no longer) appears in this thread.]

 

[jeyare]

Add/remove cache is standard feature of the Cache systems based on device mapper control. No change from 2014.

Skipping of the sequential... is one if the flashcache CLI setup (skip_seq). It was changed from UI to direct the only. So no new algorithm, just setup change.

Others are just UI enhancements for newbies.
For optimal setup you need use FIO benchmarking. Up to your operation scenario. E.g. To find a golden size of the block for the cache. No way to setup in Syno UI.

 

[jeyare]

All Btrfs metadata can be (and should be) pinned to the SSD cache rather than dumped-out by the vagaries of the previous algorithm.

This setup only applies to the writeback cache mode. Writethrough and writearound modes store no cache metadata at all.

Btw:
Flashcache can be put in one of 2 modes - Cache Everything or Cache Nothing. These 2 modes have a blacklist and a whitelist. The algorithm works as follows in "cache everything" mode:
1) If the pid of the process issuing the IO is in the blacklist, do not cache the IO. ELSE,
2) If the tgid is in the blacklist, don't cache this IO. UNLESS
3) The particular pid is marked as an exception (and entered in the whitelist, which makes the IO cacheable).
4) Finally, even if IO is cacheable up to this point, skip sequential IO if configured by the sysctl.

Scenario examples:

1. You can make the global cache setting "cache nothing", and add the tgid of your pthreaded application to the whitelist. Which makes only IOs issued by your application cacheable by Flashcache.

2. You can make the global cache setting "cache everything" and add tgids (or pids) of other applications that may issue IOs on this volume to the blacklist, which will make those un-interesting IOs not cacheable.

Note:
/proc/flashcache_pidlists shows the list of pids on the whitelist and the blacklist.

The question is: who will decide (instead of you) what TGID or PID will contain the black/white lists? Syno?


Btw, more statistics directly from the flashcache you can get from:

[CODE]
dmsetup status <cache name>
dmsetup table

[/CODE]
or simple by Proc directory:
flashcache/<cache name>/flashcache_stats

"We Can Learn From Mistakes, Not From Fairy Tales"
[No idea where your fairy tails quote came from as it does not (or no longer) appears in this thread.]

someone smarter than me said in the past