Out with the old, in with the new - 'Tis the Season - SSD

[Robbie]

Out with the old:

In with the new:

Another 4TB WD Blue to add to the volume - I've been very happy with these drives to date.

It's only since the trading the RS1221+ fan noise for the Noctua silence do you get to fully appreciate the peace of SSDs. Usually when I open the door to the network cupboard our new puppy skulks away from the noise. I turned around this time to see him asleep behind me - he is over 4-months old now so has sleep and high-speed mode as the 2 main options.

️

 

[ppeterr]

I almost did the same, only with his little brother.

 

[Rusty]

Another 4TB WD Blue to add to the volume - I've been very happy with these drives to date.

have 4 of those blue ones, running great indeed.

 

[Robbie]

have 4 of those blue ones, running great indeed.

To think for the price of 4 blue ones you could get just 1 of these miserable offerings:

If in stock when you actually need one... the Synology madness.

️

 

[jeyare]

You know my opinion to the Syno disk strategy.
But in this specific case of comparison of Syno SAT5200 vs WD Blue we need compare also another point of view - possible lifespan due to TB written. And here is almost 10x better value for the SAT5200.

For an example:
100GB re-written daily
in 300 days per year
in 5y
mean … 146TB written within 5y

What is +50% over the WD Blue

 

[Robbie]

If you fully believe the lifespan stuff. The only long-term testing I have seen was on earlier SATA SSDs and they achieved petabytes above their rated life. The bottleneck of SATA with modern NAND seems to give them an easy life relative to more extreme PCIe SSD types and workloads.

Still no idea why we are stuck with SATA backplanes rather than SAS though.

 

[jeyare]

I’m the last one who believe to the value like AFR or average lifespan, …
because from fundamental point of view any average calculation are absolutely unsuitable for a behavioral interpretation. Such value has only one advantage - it is very easy to calculate. Nothing more.

And I don’t trust also to the Syno disk calculation, due their misleading with 300k hours of testing and etc. when they announced their disks brand.

And yes, no one with health brain will care about an existence of the Syno SAT5200.

But,
there is still one reason in comparison two devices:
- be clear in all available data comparison.

One of them is the TB written (TBW)value = ((SSD capacity) x (PE NAND Cycles))/WAF

where:
SSD capacity is known
PE NAND Cycles is: Program/Erase cycles in the NAND .. simple definition - how much of cycles is possible for the cell physical damage
WAF: Write Amplification Factor, important from the setup of the controller, when same host is used in the comparison

When some low end technology uses small TBW, as is in the WD Blue case about 600 for the 4TB disk mentioned here.
Then some developed technologies uses better values, e.g. the Syno SAT5200= 9162.

From the math point of view:
- when you have similar value for the SSD capacity
and
- similar WAF
then you need take into consideration really heavy difference in the most important factor = PE NAND Cycles =what is implying to the two different endurance of used cells.

 

[Robbie]

Not for me, I don't believe the stuff put forward by marketing teams and I have seen no evidence that there are major differences in SSD NAND quality and longevity from the (few) major NAND manufacturers supplying all the SSD brands. I have seen nothing to suggest that WD, as one of the top 4 itself, produces 'low end technology' NAND.

As you know, Synology does not make SSDs or NAND and has shown zero inclination to stick with any drive partner or NAND for that matter. Over time it has pushed WD, then Seagate and now Toshiba (under an expensive sticker). Toshiba itself has moved NAND supplier around, even under the same model name. Last one I looked at had Intel NAND inside.

The only reason I give credit to the potential increased TBW or data security provided by the Toshiba/Synology drive is the generous over-provisioning of NAND and the power protection caps. The marketing guff like to ignore it but those 2 factors are the only benefits provided by these stupidly expensive SSDs.

 

[Robbie]

I always forget how long a RAID expansion can be.

2hr15min and 18% done, so that will be over 12 hours for an SSD volume. Still, far removed from the painful HDD experience.

Should have started earlier but Sunday and everything...

️

 

[Robbie]

For those following along the expansion of the SSD volume took a total of 7hr55min, with a TRIM session immediately after that. It will be interesting to see how DSM7.1 improves such things (this NAS is currently on DSM6.2.4-2).

️

 

[jeyare]

nice performance!

---------

@Robbie
for proof of the NAND endurance, you need just pure physics and understanding of the technology processes of the diff. NAND created. There isn't a marketing intent, because it can be easily checked.

As the person who has a background in the Si wafers creation, chip cutting, layering/stacking of the final products I can tell you that there are just pure physics reasons for the endurance of the cells = then you can translate them into math or values.

First:
four levels of quality for the NAND chips are known (don't mix it with the cell levels). Standardized.
An example: quality of final chips based on the cutting of stacked wafers. There are specific quality aspects for the chips from the edge of the wafers stack and the wafer centre. Heavy science is behind it.
I will try to find a scientific document for a deeper understanding.
Btw - the same reason why we have different performance and lifespan from solar cells (similar approach).

Second:
P/E NAND Cycles - is not a "marketing" kind of tag. This is not properly understood.
Reasons:

- Due to the way NAND Flash memory works, it is required that a cell is erased before it can be written to.
- The process of erasing and then writing a cell is called a P/E cycle.
- Each time a cell is erased it is damaged or worn out, so there is a lifetime for each cell. This is exacerbated by the makeup of NAND Flash. A cell can only be erased as a block and is written as a page. The size of each block and page is dictated by the design of the NAND Flash chip, but a block consists of many pages. If the information in a cell changes, it is written to a different cell and the old data marked as ‘ready for deletion’. Then, the ‘good’ data in a block is moved elsewhere and the whole block erased.

Impact:
- the reason why SLC has P/E cycle value of about 60,000 (the reason why SLC NAND is so expensive)
- and 3D TLC range is about 500 - 3,000 (what a heavy range between consumer and enterprise/industrial usage)
- more P/E cycle value more quality-grade of the NAND chip = higher cost of the device.

More stacked 3D TLC = more capacity/performance and OFC more P/E Cycles due to less cell to cell interface in this technology= more endurance.


Back to the mentioned SSD details:

WD Blue is based on 64-layer (P/E impact) 3D TLC NAND SanDisk and Marvel 88S1074 controller (WAF impact)
vs.
Syno SAT is based on 96-layer (seems to be Intel) 3D TLC NAND , Phison PS3112-S12DC controller and NA7AG64A0A DRAM chips

Overprovisioned Syno SSD - another one key to heavy endurance:

- allowing extra space for the flash controller to manage incoming data
- improves wear-leveling and random write performance, and reduces the write amplification factor (WAF), thereby improving the endurance of NAND-based SSDs.

When the Marvel controller is targeted to the consumer-grade usage. Then the Phison is a pure enterprise. Another reason for different WAF values.
The WAF simple understanding - when the actual amount of written physical data is more than the amount of logical data that is written by the host (NAS). Then the WAF is just a pure mathematical expression of this situation.
WAF formula = Bytes written to NAND / Bytes written (delivered) from host

Btw you can measure these dynamical data based on SMART (smartctl) attributes, where:
SMART 247 = host program page count
SMART 248 = background program page count

WAF = 1 + ( SMART248 / SMART247 )

SMART SSD Attributes implementation for Micron:

https://www.micron.com/-/media/client/global/documents/products/technical-note/solid-state-storage/tnfd34_5100_ssd_smart_implementation.pdf

An ideal value of the WAF is = 1.0 .... no amplification is needed, e.g. when all writes are performed in sequence with large files. But it is too far from the consumer-grade NAS reality.
In the case of the WAF, you need to count also with a "self" increasing of the WAF due to host storage operation as scrub/dedupe/filesystem operations, even RAID parity, .... all of them generate additional I/O, which makes an impact on the WAF. And more WAF = less TBW = less lifetime.

Finally, when you take into consideration the simple math for the TBW (TB written) mentioned in my post:

'SSD capacity' x 'PE NAND Cycles' / 'WAF'

Some calc examples (GB to TB transformation by 1024)

and, near to reality:

as you can see the differences between these two models - from the endurance point of view - no marketing is there. Just pure physics dependencies.

When you need to go deep dive, you can read this doc:

Understanding Life Expectancy of Flash Storage for LabVIEW Real-Time Systems

This document discusses how several factors impact the Drive Endurance formula when used to calculate how much data can be written to a drive before it becomes unreliable. Selecting the right flash-based storage device for your application often demands a tradeoff between cost, capacity, and...

www.ni.com

Take it as a short consideration impulse to this discussion.


OFC:
there is still main question - a purpose of the SSD usage?
You can purchase every 4y new SSD with more capacity, better technology, performance and also endurance for same or less cost.
No doubt. And it is valid point.
But the TBW will remain as still easily checked value. Because the pure physics is behind. No marketing. For some.

 

[Robbie]

Whilst physics was my thing I didn't make PhD level as I decided to go and fly military aircraft as a career, so it all went a bit rusty. When I hit flight test many years later we did some physics to PhD level (in a narrower field) and I did find it quite hard again. But I can still follow a tune, so to speak.

I still think marketing guff has dropped from the sky and treated like a demonstrable truth. I am not poking holes but from your text above:

- The 600 TBW figure for the WD Blue just drops from the sky (of marketing?)
- The 1000 P/E cycles awarded to the WD NAND is unsupported (without reference to the TBW)
- The 3000 P/E cycles awarded to the Intel NAND is unsupported (without reference to the TBW)
- The assertion that the controller in the WD SSD adversely impacts WD NAND life is unsourced
- The WAF figure calculated above, as you state, relies upon the same marketed TBW

I probably should mention the use of SLC in TLC architectures but as that is common to both drives it could be ignored for simplicity.

I do agree with you that the actual WAF can be calculated by dividing NAND writes by host writes, if the drive is established enough to produce a valid averaged (and workload dependant) number. As a worked example from one of my older installed WD Blue 4TB SSDs:

Power_On_Hours: 17058 (around 2 years)
NAND_GB_Written_TLC: 49306
Total_Host_GB_Written: 38567

Actual WAF: 1.278

So I just loop back to my general 'not believing' stance with the marketed TBW. The same NAND used in the WD Blue attracts a much higher TBW when fitted in different SSDs and, unsurprisingly, the make and model of controller makes little difference to WAF or by extension, NAND life. This is why consumer SSDs have achieved petabytes of writes under long-term testing - they are better than the artificial marketing segmentation suggests. No competitive NAND manufacturer sets out to produce low-quality NAND and NAND in SATA SSDs has a very easy life behind that interface bottleneck when compared to the same NAND serving a PCIe SSD.

The marketed TBW does have some meaning when it comes to warranties though. If the manufacturer sets this as a high number it will attract those with particularly abusive workloads (and deeper pockets) at the cost of more warranty returns. The cost of these returns is averaged out and does add a small premium to the price - the rest is pure profit. For this to be relevant you have to calculate your usage over time as the warranty dies at 5 years, no matter how little NAND writes you have actually used.

I baulk at the idea that the average Synology NAS user has to even consider 'Enterprise' SSDs and the idea that Synology mandates their use in newer products is driven by greed, not need.

All just an opinion, of course, whilst I sip my tea.

️

 

[jeyare]

maybe we can create an additional physics threat . I have pre-reviewed some evaluation of the entire processing of the Silicon dioxide crystalline compound to the final chip production with a significant impact on CO2 emissions.

back to the story

request:

The 600 TBW figure for the WD Blue just drops from the sky (of marketing?)

source:

https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-blue-ssd/product-brief-wd-blue-3d-nand-sata-ssd.pdf

+ from the product manual:

The endurance (longevity) of the WD Blue 3D NAND SATA SSD is calculated using JEDEC client workload (JESD219). Endurance is a direct function of user workload and access pattern. Defined in terms of terabytes written (TBW), endurance
represents the amount of data that can be written to the WD Blue 3D NAND SATA SSD during the WD Blue 3D NAND SATA SSD lifetime and varies by product capacity.

and here is (ugly) but the useful presentation of the JEDEC/JESD219 standards for the endurance evaluating:

https://www.flashmemorysummit.com/English/Collaterals/Proceedings/2011/20110810_T1B_Cox.pdf

and upgrade of the presentation about more complex info:

https://www.jedec.org/sites/default/files/Alvin_Cox%20%5BCompatibility%20Mode%5D_0.pdf

and attached is PDF from Viking technology (Sanmina Corporation) about this - Viking is one of the leaders in really enterprise SSD class (mainly SLC), e.g. with 430,000 IOPS of sustained random R/W
--------------------------

request:

- The 1000 P/E cycles awarded to the WD NAND is unsupported (without reference to the TBW)
- The 3000 P/E cycles awarded to the Intel NAND is unsupported (without reference to the TBW)

answer:

P/E cycles value (fixed, for each NAND different) is one of the main influencers of the TBW, based on the math. The physical damage of the NAND cell is defined by design:

NAND flash memory has a finite number of Program/Erase (P/E) cycles for two reasons:
- Electrons that are trapped (i.e. trap-up) in the thin oxide layer that insulates the floating gate;
- Break down of the oxide structure, due to hot carrier injection.
Once the damage to the oxide layer is large enough, it becomes increasingly difficult for electrons to travel between the P-substrate and the floating gate. The Erase Block encompassing the oxide layer cannot be erased properly with the standard threshold voltages thus needs to be retired and added to the pool of Bad Blocks.
The number of Program /Erase (P/E) cycles that NAND flash manufacturers specify for their flash devices is an indication of the expected wear out of the oxide layer. With each process shrink, flash manufacturers are facing challenges to maintain the same number of write/erase cycles and increase the ECC requirements for the flash. Flash management techniques, such as Wear Leveling, Error Correction, and Bad Block management are required to overcome and manage the flash wear out limitation.

bad message for the precise definition of the exact NAD used in the exact SSD - sourced from the JEDEC standard:
"Measurement of TBW requires access to information about NAND program/erase cycles which is generally not available
to third parties."

but I found a document from Micron about the 3d TLC P/E cycles, which is about 1500 with LDPC and 500 with BCH:

https://www.micron.com/-/media/client/global/documents/products/product-flyer/3d_nand_flyer.pdf

and Intel QLC NAND has 3000 P/E Cycles:

https://www.intel.com/content/dam/www/central-libraries/us/en/documents/qlc-nand-ready-for-data-center-white-paper.pdf

then by the design, their 3D TLC must get the same and more P/E cycles.
--------------------------

request:

- The assertion that the controller in the WD SSD adversely impacts WD NAND life is unsourced

source:
to save time with the long description here I rather use a source with a really simple and good description:

Write amplification - Wikipedia

en.wikipedia.org

this is one of the accurate descriptions of the WAF complexity and impact of the controller setting. I was surprised that the wiki also contained something like that.

--------------------------

request:

- The WAF figure calculated above, as you state, relies upon the same marketed TBW

explanation:
as you can see in my picture notes: it was a simple derivation of the values (TBW or WAF) from fixed sources

and this is one heavy doc for the complexity understanding (related to the endurance topic):

https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/over-provisioning-nand-based-ssds-better-endurance-whitepaper.pdf

 

[jeyare]

We can complain that whatever the manufacturer says is just an act of well-influenced marketing. I understand this attitude. Therefore, I choose measurable indicators only. Also, the derivation from well-known values is a helpful method.

However, I agree that long-term measurement is much more accurate than a fixed indicator. I emphasize - measurement.
Because I often come across only emotional simplifications, which tend to sound like an excuse to choose.

Each storage user has a unique environment for storing and operating data, which significantly impacts the final TBW value = then the only efficient way to increase the TBW is by reducing the WAF value (possible, defined) when the features are available in the controller (another cost impact). Not for an average NAS user. Sure.

However, be aware that the more we (user or host or filesystem or SSD controller) pay attention to data protection, the more we shorten the life of the SSD itself = final TBW value is affected. And that was the essential message.


Last:

Power_On_Hours: 17058 (around 2 years)
NAND_GB_Written_TLC: 49306
Total_Host_GB_Written: 38567

49306 GB or 48.15TB x 5y product warranty = 241TB what is necessary to compare with the vendor maxed TBW, in this case: 600TB. You will get an indicator of the expected vendor TBW range:

600TBW; 5y = 600/5/365 = 336.66 GBW daily (be careful this is not same value as DWPD)

then

within 711days (17058 hours) it is expected value of about: 711 x 336.66 GBW = 239,279.3 GBW by the vendor
follow your measure it is just 49,306 GBW

which is just 21% of the expected written capacity range from the vendor. Then your SSD is underutilized from the vendor expected value within 5y.
Then, logically, your SSD can last longer as the vendor expects in the (marketed) value.

Note:
Disk usability increases over time, and this number cannot be evaluated only linearly w/o an understanding of the data architecture used. However, this gives one of the indications - if the disk pool capacity isn't overestimated or vice-versa.

Finally, it is still up to the user decision. No doubt.