User article Panic around the hard drive. Or just well thought out marketing (from/for whom)? Part No. 2

Currently reading
User article Panic around the hard drive. Or just well thought out marketing (from/for whom)? Part No. 2

Part No. 2
..... previous Part No. 1 is available here
------------------------------

Nowadays technologies of recording - PMR:

The most widespread technology standard for data recording on magnetic media since 2005 is Perpendicular recording (or perpendicular magnetic recording, PMR). Reason = optimized recording layer and W/R heads brought more Areal density = better utilization of the 3.5/2.5“ HDD form factor. Toshiba as first vendor prepared 1.8“ HDD with 133Gb/in2 for PMR based HDD, shortly thereafter Seagate in 2006 created the first commercially available 2.5/3.5“ HDD with PMR. ... the race for more and more Areal density had picked up speed.

Picture of PMR principle, sourced from Wikipedia:
1588001923991.png



In 2013, Seagate introduced shingled magnetic recording (SMR) as the next generation of PMR based data recording. Since then Conventional magnetic recording (CMR) was used for preliminary PMR technology and SMR for second generation of PMR. Then Both CMR & SMR are PMR (based on the orientation of magnets in platter). When you see “PMR only” in the description of an HDD, you can’t distinguish whether the disk has CMR or SMR. And vendors hide such information. Why, if SMR is better? Better for whom?

“With SMR technology, Seagate is on track to improve areal density by up to 25 percent or 1.25TB per disk, delivering hard drives with the lowest cost per gigabyte and reaching capacities of 5TB and beyond.” said Mark Re, Seagate’s chief technology officer. And there was also announced that – „Importantly, SMR can improve reliability by allowing Seagate to use fewer heads and disks to achieve new capacity points. SMR also provides a better value by increasing storage capacity while utilizing the same disk and heads as drive configurations shipping today thereby, providing a more cost-effective approach to increasing aerial density.“

Just remember the relation of the vendor costs and the number of platters in HDD. See that point in SMR? Who is the winner?

Where is the problem then?

Some simplification is needed, for better understanding, so I will use an example based on highway traffic (next three pictures sourced from xin.at):

1. CMR technology is based on enough width (guard space) of highway lanes (disk capacity), then cars can
1588002124060.png
securely drive with defined speed and direction, without additional complicated regulations. Reason why CMR can’t provide more highway lanes in the closed environment of a 3.5” HDD form factor = they need more patterns, but they can’t change the dimensions of the HDD box. There are just three possible solutions to it:
  • Keep the maximum possible used platters, then the maximum available disk capacity is defined by max. tracks for the write process. No way to infinitely upgrade. Seems to be we’ve reached the physical possibilities of CMR.
  • Change the HDD form factor to bigger than 3.5“, but can you imagine how this would be done? Who will be first? As you read above it was changed many times in HDD history. Achievable, but really costly for everyone.
  • Upgrade the track technology = more tracks (SMR) or new, totally redefined HDD technology (HAMR, ...).
2. SMR technology is based on really short (tiny) highway lane-width, without the guaranteed space for a car
1588002069783.png
within a single lane. That is risky, but achievable for really low traffic (low workload). However in regular traffic (when all the lanes are almost full) you need to regulate all cars separately in these lanes to achieve movement for all cars = reason for the existence of DMSMR (device managed SMR = HDD firmware-controlled). The append-only highway lanes (platter‘s zones) are very slow for random writing, so writes are first sent to a cache, and the disk moves this data to SMR parts when idle. And the term „when idle“ is a problem for all who use the HDD with a workload of >180TB/yearly (explained below) . Finally, you have achieved a model of more highway lanes (SMR), but a really slow movement of cars vs the normal highway (CMR). This model has additional advantage for vendors = less patterns. And less platters = less cost for vendors.

Back to disk technology terms - SMR must overwrite adjacent magnetic tracks and then rewrite them. That’s the reason why they’re so slow and why they need a bigger cache for such operations.

1588002145847.png
An additional problem for NAS users is native Zoned device suppor (ZBC/ZAC commands layer for the SMR) for etx4 file system = doesn’t exist for the DMSMR, as far as I know. Thusly the SMR disk can‘t beat CMR speed with etx4. Even using an SSD cache in your NAS can‘t help you with such a basic issue. Some real tests have confirmed such problems during rsync from the SMR drive, especially if the filesystem wasn‘t mounted read-only with the noatime option. Such test confirms that an SMR drive needs to write a timestamp for each file rsync read, resulting in a significant performance degradation from around 80 MB/s down to 3-5 MB/s plus annoying head wear / clicking noise. Different story is for BTRFS, which nativelly supports a Zoned device (SMR), but you have to purchase an HDD with HMSMR (host managed SMR). If you’re operating a system with BTRFS, you should specifically consider purchasing such HDDs. That way you can get better performance. If you’ve already purchased a disk with DMSMR, you might be sorely dissapointed.

From all the mentioned SMR features I would summarize these points:

  • when your Storage pool is about frequent write and read, then SMR isn’t for you. Because your disk buffer will frequently be almost full (no idle time as basic principle of the SMR). Then the HDD firmware will need to locate a single empty track, or rewrite existing ones to be ready to write long sequential data from the buffer. Which will definitely kill your disk performance.
  • when your Storage pool is about frequent read and casual write, then SMR is OK for you (up to first degradated disk incident, because then you will write a lot of data very, very slowly). E.g. media files for Plex server.
  • Never mix both kinds of PMR (CMR or SMR) into a single storage pool (RAID), you will kill your performance, even more
But vendors have prepared a „better“ recommendation for us, regarding when these SMR HDDs are useful for us and when not. Their recommendation is based on 180TB/yearly workloads in the HDD. First, it seems like magic, because every single vendor uses the same threshold = 180TB/yearly. Or is it a kind of mutual recommendation from all the vendors to us? I feel some kind of strange precedents here, when each independent vendor has the exact same operation conditions.

So let’s understand how the independence of HDD R&D functions:
  • In 2011/12, three mergers (Seagate/Samsung, Western Digital (WD)/Hitachi, and Toshiba/Hitachi) reduced the number of HDD manufacturers from 5 to 3 companies.
  • Follow-up Research report named: Southeast Asia beyond Crises and Traps (2017), stated that Thailand is one of the world’s largest HDD manufacturing bases. In 2012 it held approx. 40% of the HDD global share. In 2018 it was 82%. The sector is dominated by two global leaders, Seagate and WD with more than 50% of production there. We need to speak about dominant R&D association:
  • International Disk Drive Equipment and Materials Association (IDEMA), have HDD R&D in Thailand
  • ASTC consortium exclusively covers members from IDEMA only
  • WD, Seagate and Toshiba – use a collaborative joint R&D in ASTC
  • A really interesting read is this 2019 research: Mergers and innovation: Evidence from the Hard Disk Drive market.
Now’s the time to understand, what the 180TB/year workload threshold means

Simple check using a standard situation:
We will utilize an HDD for just 2 hours/daily with backup between two NASes with the average speed of data throughput 70MB/sec (which isn’t perfect speed on a Gigabit network).

Then we can calculate the total amount of data transported between the NASes (disk groups):
70MB/sec x 2hours = 0,6TB/day x 365 days = 221 TB/year .... which is +23% over the 180TB/year threshold for the SMR disk. Then, following this simple check, the operation model is out for the SMR disk.

We still haven’t mentioned RAID operations and RAID factor penalties for the write operation, when you need to define your yearly workload based on a simple IOPS measurement. Then 180TB/year workload is out of any NAS enthusiast’s group operation model.



Conclusion:

SMR with DMSMR is great technology for all NAS owner, when they use such HDD(s) or groups isolated for a specific purpose, e.g. media center source.
In this case heavy workload for written operation isn’t expected. When the read throughput for TV is about 60Mbps = 7.5MB/s. Or for a photo library. With such operation model you will hardly reach 180TB/y and you will be happy with the price per TB.

SMR DMSMR isn’t useful for others who need to utilize their disk more than 2hour daily with the average write speed of about 70MB/sec, because you might experience significant speed decrease. There aren’t enough reviews or tests for BTRFS and SMR disks with HMSMR, which would confirm they are better in such conditions. Obviously, you need CMR disks for all virtualization platforms, cloud platforms or heavy backup operations in your NASes, what was an enabler of NAS purchases in last few years. Never use a mixed RAID disk group with CMR and SMR disks. Ever.

When you’re looking for a new HDD, try to compare the number of platters in the watched disks, then if one has SMR technology described and another has the same disk platters, even same capacity, there is a 99,9999% chance, that both of them use the same technology. Yes, it’s complicated, but useful until vendors start clearly disclosing that information.

I really don’t know if the 3 vendors have any mutual agreements about the “SMR disclosure” in their Disk data sheets, I don’t really care, but it seems to be, that their R&D doesn’t care about the NAS development, especially when they create NAS HDDs. And this is a strong message for the customer’s wallet. But I think, they need to take responsibility for such a managed chaos. I have some recommendations:
  • When you purchase an HDD and you’re looking for CMR and not an SMR disk, but the seller can’t confirm the technology level, put this request into your order (e-shop).
  • If you get different technology as was requested, you can claim this order as wrong. In the EU we have strong regulations for it. It is exactly the same example as with a car purchase with a petrol engine and your dealer delivers a diesel engine. Different technology for different driving purpose. And hidden during the ordering process!
  • If vendors don’t care about customers, they will care about claims from their distribution channels, when people massively return their goods, which were different from order specifications.
Protect your purchasing rights!
 

jeyare

Subscriber
2,486
833
NAS
Synology, TrueNAS
Operating system
  1. Linux
  2. Windows
problem of the usage of the SMR in complicated RAID structures as for RAID6 or RAID10 in Syno NASes, that there isn’t pure Btrfs RAID, but a hybrid split between LVM and Btrfs and Device managed SMR at the disk side.
What is in opposite side with a Host managed SMR - far away from the Syno setup. Then there is a big chance of stuck of data within - what isn’t primary problem of the SMR technology, but used architecture workflow.
 

jeyare

Subscriber
2,486
833
NAS
Synology, TrueNAS
Operating system
  1. Linux
  2. Windows
this is another possible reason for an existence of the Syno branded disk drives - hiding of Syno NAS RAID Achilles tendon. Makes a sense.
They don’t need solve the reason, just cover it by nice story.
 
5
0
NAS
DS1618+
Operating system
  1. Linux
Last edited:
how to easy check how many platters do you have in your current HDD:
Code:
hdparm -I /dev/sda   .... sdb or sdc ... up to number of your HDD

Result = 16 heads
View attachment 1385

Naturally in PMR technology, a platter has 2 sides and thus 2 surfaces on which data can be manipulated
Then usually there are 2 heads per platter.
Follow above "hdparm" result. You have 16/2= 8 platters
The you can see what is the RAW size of your HDD, from same hdparm command:
View attachment 1387

Now you need to calculate, what is RAW value in MB in single platter for a comparison with another HDD (CMR or SMR).
- Device RAW size = 953 869 MB
- Platter RAW size = 953 869 / (8 platters) = 119 233.625 MB/platter = 116.44GB/Platter

So what about make a table such this (columns):
- Vendor
- Disk model (e.g. NAS Red)
- Form factor (3.5"/2.5")
- Model number, achivable from the hdparm results
- Firmware revision, dtto
- Device size (RAW), dtto
- Cylinders, dtto
- Heads, dtto
- Platters based on Heads/2 formula
- RAW size per platter based on mentioned formula (check)

Then we can easily compare when the HDD have less platter for same RAW size, include the last check parameter. Hm?

I need speak with @SynoMan, how we can build comparison table based on your measurement. It will help each others to check HDD CMR/SMR valiadtion: 1,953,525,168 LBA49 sectors * 512 Bytes per Sector = 1,000,204 MBytes = 1 TB (Decimal calculation, not binary!)

Hi, Jeyare

Your drive in the screenshots is a Seagate Constellation 1TB. This drive has 1 platter and 2 heads, physically. Its Areal Density is 625 Gbit per square-inch in average. It has 4096 Bytes per sector physically but uses 512 Byte Emulation.
Seagate provides the so called Product Manuals as public download where all technical details are included. Best use Google to find the PDF and search criteria "Seagate ST1000NC001 product manual" for this specific drive.

The output of HDPARM regarding heads, cylinders and sectors/track is the setting that must be provided to any BIOS for drive recognition and calculation of the drive capacity. Be careful here as these outputs show totally outdated CHS Cylinder-Head-Sector geometry. The usage of CHS is misleading, because all (actual) drives are using LBA Logical Block Addressing. CHS was set obsolete in 2002 with the ATA-6 specification.

CHS calculation: Cylinders * Heads * sectors per track * Bytes per Sector = 7.87 Gigbytes for your above drive = wrong
Correct calculation is amount of LBA48 1953525168 * Bytes per Sector = 1,000,204 MBytes = 1 TB (decimal, not binary!)

Hope this helps to explain the mismatches.

If you want to identify if the drive is PMR Perpenticular Magnetic Recording or SMR Shingled Magnetic Recording, take a look at the Areal Density. An Areal Density higher than 1,200 Gbit per square Inch is mostly an SMR drive.
This Areal Density Statement is valid for drives in the past. Future drives with any kind of HAMR/MAMR etc. will have a much higher density due to the nature of X-Assisted-Magnetic-Recording Technology.

The term CMR means Conventional Magnetic Recording. Either a drive is PMR Perpenticular or SMR Shingled. I assume that "Conventional" was introduced as replacement of the hard-to-speak "Perpenticular"; CMR and PMR are the same.
-- post merged: --

SMR is OK for their single drive (non-RAID) NASes - I keep forgetting them.
But they didn't say "do not use any SMR HDD for a RAID", which implies that they work for some kind of RAID...?

"which implies that they work for some kind of RAID...?"

No, other way round. SMR works fine in any kind of RAID until something happens!
A RAID with SMR drives will not "survive" any RAID Rebuild due to the fact that the individual drives will get out-of-sync pretty fast.

Keep in mind that for example shifting data from one LBA to another causes five steps in SMR drives instead of one step in CMR/PMR drives. That's why for example overwriting data on SMR breaks down the drive's performance to 20% of the original performance; up 5 steps needed for overwriting a SMR sector instead of one with CMR/PMR.
 

jeyare

Subscriber
2,486
833
NAS
Synology, TrueNAS
Operating system
  1. Linux
  2. Windows
e careful here as these outputs show totally outdated CHS Cylinder-Head-Sector geometry. The usage of CHS is misleading, because all (actual) drives are using LBA Logical Block Addressing. CHS was set obsolete in 2002 with the ATA-6 specification.
- thx for your care, as you can see no one is calculating the CHS value there

Correct calculation is amount of LBA48 1953525168 * Bytes per Sector = 1,000,204 MBytes = 1 TB (decimal, not binary!)
- decimal and binary values are clearly recognisable from the hdparam output
but I don't care about the decimal "marketing" values, because just the binary value counts in the binary environment.

Hope this helps to explain the mismatches.
what exactly mismatches?


The term CMR means Conventional Magnetic Recording. Either a drive is PMR Perpenticular or SMR Shingled. I assume that "Conventional" was introduced as replacement of the hard-to-speak "Perpenticular"; CMR and PMR are the same.
as you can read from the initial article content related to this discussion:
The most widespread technology standard for data recording on magnetic media since 2005 is Perpendicular recording (or perpendicular magnetic recording, PMR). In 2013, Seagate introduced shingled magnetic recording (SMR) as the next generation of PMR based data recording. Since then Conventional magnetic recording (CMR) was used for preliminary PMR technology and SMR for the second generation of PMR.

If you want to identify if the drive is PMR Perpenticular Magnetic Recording or SMR Shingled Magnetic Recording, take a look at the Areal Density. An Areal Density higher than 1,200 Gbit per square Inch is mostly an SMR drive.
This Areal Density Statement is valid for drives in the past. Future drives with any kind of HAMR/MAMR etc. will have a much higher density due to the nature of X-Assisted-Magnetic-Recording Technology.
as you stated - the Areal Density - isn't a guarantee of the SMR identification. I prefer more easy way:
- definition by the vendor must be clear (1.5 y ago it was a wild west in such identification)
 
5
0
NAS
DS1618+
Operating system
  1. Linux
- thx for your care, as you can see no one is calculating the CHS value there


- decimal and binary values are clearly recognisable from the hdparam output
but I don't care about the decimal "marketing" values, because just the binary value counts in the binary environment.


what exactly mismatches?



as you can read from the initial article content related to this discussion:
The most widespread technology standard for data recording on magnetic media since 2005 is Perpendicular recording (or perpendicular magnetic recording, PMR). In 2013, Seagate introduced shingled magnetic recording (SMR) as the next generation of PMR based data recording. Since then Conventional magnetic recording (CMR) was used for preliminary PMR technology and SMR for the second generation of PMR.


as you stated - the Areal Density - isn't a guarantee of the SMR identification. I prefer more easy way:
- definition by the vendor must be clear (1.5 y ago it was a wild west in such identification)

The 16 Heads of the drive are related to the CHS output, which is not correct; nothing else.

but I don't care about the decimal "marketing" values, because just the binary value counts in the binary environment.
As of End of 1998, also the HDD industry had to move to the SI units. Since then, every HDD capacity has to be named in decimal units, not binary units. See this simply as explanation why HDDs are showing 1,000,000 MBytes, for example.

as you stated - the Areal Density - isn't a guarantee of the SMR identification. I prefer more easy way:
- definition by the vendor must be clear (1.5 y ago it was a wild west in such identification)

For all drives before xAMR, it's pretty much the best identification. Even 1.5 years ago, it was the best identification. But this information was - and probably still is - not easy to find.

BTW, I found this thread accidentially as I was searching for HM-SMR drives and BtrFS formating options.
 

jeyare

Subscriber
2,486
833
NAS
Synology, TrueNAS
Operating system
  1. Linux
  2. Windows
The 16 Heads of the drive are related to the CHS output, which is not correct; nothing else.
So,
when you need to get suitable Areal density information you need to calculate:
length of the tracks around the platter multiplied by the number of tracks per inch within the same platter

then the number of platters is important (and has an impact on the disk capacity) in the first touch (intent of the initial post) because it is impossible to get Areal density information from the drive firmware (yes product manual contains it). So you need just compare platter numbers with similar drives.

In this specific case: Seagate ST1000NC001 (1TB), there is used platter with 1TB capacity
same as for:
6TB BarraCuda Pro ... 6 x platters per 1TB (ST6000DM004)
3TB Enterprise Capacity ... 3 x platters per 1TB (ST3000NC002)
2TB IronWolfPro ... 3 x platters per 1TB (ST2000NE0025)
... etc
I have access to heavy drives db source about the drives, platters, ...

so CHS output has nothing to do with the Areal density and I don't care about the CHS value.

As of End of 1998, also the HDD industry had to move to the SI units. Since then, every HDD capacity has to be named in decimal units, not binary units. See this simply as explanation why HDDs are showing 1,000,000 MBytes, for example.

The use of SI units was introduced by disc manufacturers to prove to people that they supply more (capacity) for the same costs. Nothing more. It is one of many unnecessary benefits for practical use. That's why I don't bother me and I don't deal with it.

For all drives before xAMR, it's pretty much the best identification. Even 1.5 years ago, it was the best identification. But this information was - and probably still is - not easy to find.
As you say

BTW, I found this thread accidentially as I was searching for HM-SMR drives and BtrFS formating options.
1. HM-SMR needs the Host Managed feature.
Host Managed drives expose the underlying zone layout via SCSI or ATA commands. The host is required to access the
zones according to the rules described by the zone layout. Any commands that violate the rules will be returned with
an error.
2. Synology NASes (or DSM based on BSD) don't support this feature. Just Drive Managed = DM-SMR is a supporter option.
3. Then running BTRFS on HM-SMR drive/s in Syno NASes is out of possible operation
 
5
0
NAS
DS1618+
Operating system
  1. Linux
6TB BarraCuda Pro ... 6 x platters per 1TB (ST6000DM004)
6 disks, 12 Heads, Areal Density 732 Gbit/in² avg

3TB Enterprise Capacity ... 3 x platters per 1TB (ST3000NC002)
3 disks, 6 heads, Areal Density 625 Gbit/in² avg

2TB IronWolfPro ... 3 x platters per 1TB (ST2000NE0025)
2 disks, 4 heads, Areal Density 651 Gbit/in²

BarraCuda ST8000DM004, 3.5" 8TB SMR
4 disks, 8 heads, Areal Density 1203 Gbit/in²

BarraCuda ST5000LM000, 2.5" SMR
5 disks, 10 heads, Areal Density 1307 Gbit/in²

I have access to heavy drives db source about the drives, platters, ...
I'm using offical public data sheets and technical manuals

The use of SI units was introduced by disc manufacturers to prove to people that they supply more (capacity) for the same costs. Nothing more. It is one of many unnecessary benefits for practical use. That's why I don't bother me and I don't deal with it.
Urban Legend, international trade laws and regulations caused this mandatory move to SI units.
Even if you ignore this, a lot of users (still today) have issues with understanding this and complain about differences in TeraByte and TebiByte capacity.

1. HM-SMR needs the Host Managed feature

HM-SMR means Host-Managed SMR, the host has to manage all logical details, e. g. every Linux Kernel since 4.7 can do this with hazelfree BtrFs or even Ext4 with some manual work. HM-SMR drives are used in data centers and clouds where the guys know what to do. DM-SMR means Device-Managed SMR, the drive itself handles everything internally.

3. Then running BTRFS on HM-SMR drive/s in Syno NASes is out of possible operation

Correct, this btrfs implementation does not support the needed features for Zoning
BtrFS in general can of course support HM-SMR drives.
 

ed.j

Subscriber
180
51
NAS
DS920+, DS416slim
Operating system
  1. Windows
Mobile operating system
  1. Android
Stupid question maybe... but how do you measure the disk workload? Is there a way of doing this within DSM?
 

jeyare

Subscriber
2,486
833
NAS
Synology, TrueNAS
Operating system
  1. Linux
  2. Windows
Seagate:
Annualized workload rate is the sum of lifetime reads and writes, multiplied by 8760 (the number of hours in a year) over total power- on hours. The result is expressed in TBs (terabyte) per year.

More here:
https://www.seagate.com/www-content/surveillance-center/files/Understanding-Reliability-Metrics.pdf

or

Annualized Workload Rate = (Lifetime Writes + Lifetime Reads) * (8,760 / Lifetime Power On Hours)

all of them you can get by CLI - smartctl
except some vendors mess in the SMART standards
 

Create an account or login to comment

You must be a member in order to leave a comment

Create account

Create an account on our community. It's easy!

Log in

Already have an account? Log in here.

Similar threads

  • Article
I spent a time with googling around Internet and people are crazy. There is a solution, but need to read...
Replies
3
Views
882

Welcome to SynoForum.com!

SynoForum.com is an unofficial Synology forum for NAS owners and enthusiasts.

Registration is free, easy and fast!

Top