4k Native HDDs vs Synology

[Robbie]

Probably one of those really dumb questions but...

All my HDDs are native 4k drives (ie 4096 byte sectors) and, like most modern HDDs, they can emulate 512 byte sectors for logical addressing in order to expand compatibility. However, DSM fails to recognise this and treats the drives as having 512 byte native sectors.

DSM does highlight that 4k native drives should be run and managed '...separately from hard drives in other disk sector formats when creating, repairing, expanding and migrating any volume, disk group, RAID group and LUN'. Again, this makes sense from a performance perspective, especially if you want the best out of 4k native drives.

So how can you make DSM actually recognise and run 4k native drives as 4k native drives?

 

[jeyare]

I will send you a proof when I will receive new ST6000NM022A and ST4000NM001A set of HDDs for the new project with RS1221+ for my friend

One point is DSM Frontend state and second point is what you will get from CLI:
- while is the HDD pulled out and connected to another Linux host

fdisk -l

- while is the HDD inside the NAS

cat /sys/block/sda/queue/hw_sector_size

then

cat /sys/block/sdc/queue/physical_block_size

here you will see two diff results 512 and 4096

 

[Robbie]

Yes, I know. The trick here is getting an 512e drive to run at the native 4k.

 

[EAZ1964]

Maybe this post helps:

How to use/format Native 4Kn drives in Synology or NAS

A few years ago I started using 4Kn drives in my storage (Open-E and Synology NAS). You had to decide for a 4Kn (native) drive or a 512-byte drive when buying it.Now, a few years later, companies like Western Digital (HGST) and Seagate come with 'Advanced Format' drives, it's one drive which you...

www.roelbroersma.nl

Real world benefits are probably not noticeable, unless your disk has to write an overload of small files. So questionable if it makes sense.

 

[Robbie]

@EAZ1964 - Thanks - I had not stumbled on that link but I have seen references to Hugo and Seachest before.

Using 4k-based OS with a 4k file system on a 4k drive makes perfect sense to me. The only real reason that 512 emulation came in as 'AF' was to maintain backwards compatibility with Windows XP as, at the time, there were many consumers unwilling to move away from that OS.

Everything runs at 4k or a variation of that these days so the requirement for emulation is obsolete. As for performance (in throughput terms) the HDD's ASCICs and DRAM effectively mask any real observable performance difference. You could probably find one if you specifically benchmarked for it (as the drive has to work harder) but yes, noticeable it is not.

For me the performance advantage has to be data reliability as the 4k ECC algorithm is so much better.

I feel it should be an easy option for a dedicated NAS OS such as DSM. If they gave the consumer a choice of 2 options at initialise where one was 'Format the drive for a slightly slower, slightly smaller and less reliable experience (required for XP)' and a second button for 'Native Format' I think the end users would express a similar preference.

I'll give 'Hugo' a go. I'm not a Windows user but there is an 'emergency' unit in my rack that probably needs the exercise.

️

 

[Robbie]

Gave it a quick go but frustrated by Windows and an issue with my main Syno NAS.

Hugo seems simple enough but the syntax in the help files does not match the support paperwork and varies considerably between versions too. I got a command syntax together that would run but didn't get further than that:

️

 

[jeyare]

Robbie,

follow the totally unclear WD support (as usual) you can use:

hugo s --device        # to show the devices list
hugo format -g \\.\PhysicalDrive<n> -b 4096 --fastformat        # for the new format

there is the error in your command:

hugo format -m <partnumber> --fastformat -b 4096 -n max

your "\\.\PhysicalDrive<n>" variable is = \\.\PhysicalDrive1

but it is better to use PowerShell command:

Get-Disk | Format-List

Number: ...is the PhysicalDrive<n>


I hope for a better result!
Chers

-- post merged: 7. Mar 2022 --

there is maybe the error in the 'm' (model) variable, then the 'g' (target, device path) variable is more suitable

-- post merged: 7. Mar 2022 --

btw, you can use HUGO utility also with Lnx

https://archive.org/download/wdc-hugo/HUGO_Users_Guide-7.2.6.pdf

 

[Robbie]

follow the totally unclear WD support (as usual) you can use:

hugo s --device        # to show the devices list
hugo format -g \\.\PhysicalDrive<n> -b 4096 --fastformat        # for the new format

there is the error in your command:

your "\\.\PhysicalDrive<n>" variable is = \\.\PhysicalDrive1

The -g option is not included with either v7.2.6 (latest) or v7.0.3 (recommended) of Hugo. Only -m or -s exist; if there is an error in this part of the syntax you do not get to the 'Format Y/N' dialogue. For an unknown reason it is failing after that point.

-- post merged: 7. Mar 2022 --

For clarity, neither the -m (model) or -s (serial) flags work for me, even though both are recognised. A little frustrating:

   HUGO CLI

(hugo) format -m WD80EZAZ-11TDBA0 --fastformat -b 4096 -n max

        ******************************** WARNING **********************************

           The Format command will result in loss of data on the specified device.

           This device might be your boot device and it won't be protected.

        ******************************** WARNING **********************************

                Are you sure you want to Format this device? (Y/N) Y

Format device on 1 Device(s)...
Failed: Format command Failed on device 7SKM84UW.
        Device will not be formatted.
(hugo) format -s 7SKM84UW --fastformat -b 4096 -n max

        ******************************** WARNING **********************************

           The Format command will result in loss of data on the specified device.

           This device might be your boot device and it won't be protected.

        ******************************** WARNING **********************************

                Are you sure you want to Format this device? (Y/N) Y

Format device on 1 Device(s)...
Failed: Format command Failed on device 7SKM84UW.
        Device will not be formatted.
(hugo)

 

[jeyare]

the idea about 'g' variable was based on Hugo User Guide 7.2.6 ...... not surprised, that WD must kidding users. Maybe they went to a joint customer care course together with Syno support staff.

what about:

hugo format -c - m <model>

Set media compatibility check

-- post merged: 7. Mar 2022 --

-- post merged: 7. Mar 2022 --

check this one>

GitHub - pig1800/WD4kConverter

Contribute to pig1800/WD4kConverter development by creating an account on GitHub.

github.com

 

[Robbie]

No luck.

(hugo) format -c -m WD80EZAZ-11TDBA0
--media-compatibility-check is only valid when --fastformat is also specified.
(hugo) format -c -m WD80EZAZ-11TDBA0 --fastformat

        ******************************** WARNING **********************************

           The Format command will result in loss of data on the specified device.

           This device might be your boot device and it won't be protected.

        ******************************** WARNING **********************************

                Are you sure you want to Format this device? (Y/N) y

Format device on 1 Device(s)...
Failed: Format command Failed on device 7SKM84UW.
        Device will not be formatted.
(hugo)

I am accessing the drive by a USB toaster and whilst others have used them perhaps a direct SATA connection may overcome the issue.

I only have 1 (fragile) internal sata port and it only has power for a 2.5" drive... not an ideal disk test bench.

Now if I could install Hugo on DSM...?

 

[jeyare]

Now if I could install Hugo on DSM...?

my idea after the HUGO user manual reading:

-- post merged: 7. Mar 2022 --

@Robbie did you try the GitHub link?

 

[Robbie]

The GitHub details are rather sparse but I could not get it to run anyway as it is missing a brace of dll dependancies.

 

[HDD-Joergl]

Probably one of those really dumb questions but...

All my HDDs are native 4k drives (ie 4096 byte sectors) and, like most modern HDDs, they can emulate 512 byte sectors for logical addressing in order to expand compatibility. However, DSM fails to recognise this and treats the drives as having 512 byte native sectors.

DSM does highlight that 4k native drives should be run and managed '...separately from hard drives in other disk sector formats when creating, repairing, expanding and migrating any volume, disk group, RAID group and LUN'. Again, this makes sense from a performance perspective, especially if you want the best out of 4k native drives.

So how can you make DSM actually recognise and run 4k native drives as 4k native drives?

4096 Bytes are the physical sector size whereas 512 bytes is the logical sector size; 512e means that 8 logical sectors are stored within one physical sector, that's why 512e is called "emulated". Tools like hdparm show both the logical and the physcial sector sizes. True 4Kn drives have both physical and logical size set to 4096.

Normally all drives that support 4096 physical can be reformatted from 512e to 4Kn, 4096 native. The HDD manufacturers offer tools to do this reformat. It might be a low level format or a fast format, depends on the drive itself and its capabilities.
What kind of format is supported by the drive, needs to be investigated. Not all drives support FastFormat. Use the tool's options to identify what format method is supported by the drive.

But please note that reformatting (in general) wipes all data on the drive. Reformatting with preserving data lasts ages to finish.
Never mix 512n, 512e or 4Kn drives within the same disk group or RAID. This will not work long-term.

 

[Robbie]

Isn't that what I said?

 

[jeyare]

@Robbie
there are three groups of posts.

The first is created after a deep re-study of the whole thread.

The other arises from the immediate need to write something, without the need for a deeper examination of what has been written so far, resp. needs to consider whether it will help the OP to eliminate the problem that is the root of the thread.

The last ones are the funny ones, the essence of which is to make the day more beautiful.

Regarding your issue - do you have output from the Hugo - DSM test?

 

[HDD-Joergl]

Isn't that what I said?

No, Robbie, sorry to correct. It's not what was written in this line:

All my HDDs are native 4k drives (ie 4096 byte sectors) and, like most modern HDDs, they can emulate 512 byte sectors for logical addressing in order to expand compatibility. However, DSM fails to recognise this and treats the drives as having 512 byte native sectors.

4Kn: 4K native drives have 4096 bytes per physical sector and 4096 bytes per logical sector.
512n: 512n native drives have 512 bytes per physical and per logical sector.
The Term "native" means that these sector sizes cannot be changed from 512 to 4096 or the other way.

Variations are allowed like 520 or 528 instead of 512. Similar to this, 4112, 4160 or 4224 sectors can be set at 4K drives.
But, again, no "big switch" allowed from 512 to 4k.

512e: means that 8 times 512 bytes are emulated as eight logical sectors within a 4096 bytes physical sector.
This drive has 512 bytes per logical sector. These drives support the so called Advanced Format, introduced 2010 as a new Standard.

DSM does not fail to recognize this drive properly because this drive is not a 4Kn drive, it is a 512e drive.
The drive shows 512 bytes per sector to the DSM, not 4096. Therefore, DSM correctly reports this drive as 512 bytes per sector.

Reformatting the drive changes the size and the amount of the logical sectors.

Try to reformat your drive using normal format option, not fastformat. FastFormat is supported only on drives which support the Advanced Format Standard.

hugo format -b 4096 should do the formatting.
Afterwards, this drive then reports 4096 bytes per logical sector and will be treated corrected as a 4K drive.

 

[jeyare]

@Robbie

I found this site with an interesting conclusion in the same "Format command Failed" case:

"Even though it fails to format the disk, it does write the sector size / partition table to it and when you insert the drive in you Synology now, it will show you 4096 byte sector size and you can now create your 4Kn volume!"

How to use/format Native 4Kn drives in Synology or NAS

A few years ago I started using 4Kn drives in my storage (Open-E and Synology NAS). You had to decide for a 4Kn (native) drive or a 512-byte drive when buying it.Now, a few years later, companies like Western Digital (HGST) and Seagate come with 'Advanced Format' drives, it's one drive which you...

www.roelbroersma.nl

Check the disk for this mysterious miracle.
Fingers crossed!

 

[Robbie]

It has not worked for me yet and my observation on that thread is the use of v7.0.3, which is one of the two versions I have been testing with. The copy of v7.0.3 that I have does not contain all of the options used in the command shown - there is no -g or path option.

I download the v7.0.3. the poster is hosting (very good of them to do that - top work) to see if it had been tweaked in any way but it still does not have the path option, including any reference to it in its help file.

Still, not given up yet and I will to rig a way of getting a 3.5" drive powered and connected onto what is effectively a laptop SATA connector. Beyond that this may need someone who runs a full-up Windoze machine.

️

 

[jeyare]

It's literally a shame for WD not to be able to provide a simple tool for their users.

I am waiting for the delivery of my native 4kN Exos (Seagate).
In the case of Native 4k drives - labelled as "4kN", it is not necessary to create such "Hugo" entertainment. One of the reasons for my last project:

Ultrastar HS3xx or Exos 7E8 for specific data architecture

new project for my friend RS1221+ equipped with 2 pools: 3x6TB in RAID5 + 1x spare (HDD) 3x4TB in RAID5 + 1x spare (HDD) 4kn (native) 4096 Bytes per logical/physical sector is necessary for better performance (purpose definition) Seagate 7E8 (6TB) ST6000NM022A & (4TB) ST4000NM001A are ok with...

www.synoforum.com

For others:
- be careful 4k doesn't mean the same as 4kN

 

[Robbie]

No, Robbie, sorry to correct. It's not what was written in this line:


4Kn: 4K native drives have 4096 bytes per physical sector and 4096 bytes per logical sector.

You win the award for splitting hairs! I describe a drive as having 'native 4k (ie 4096 byte sectors)' with 512 byte emulation and you bound that all up in your '4Kn' label before pulling it apart again as it isn't true 4k on 4k.

Dude, it really is a drive with a 4096 byte sectors running a 512 byte emulation within those 4096 byte sectors. If/when you remove the emulation for the drive you are left with a drive running on pure, unadulterated, native, unsullied, untainted and impeccable 4096 byte sectors.

I don't think anyone has any doubt now that a drive such as the one described is running 4096 byte sectors with a selectable 512 byte emulation layer within them.