Bit rot, a myth?

[EAZ1964]

My view on the bit rot topic is that it is a "popular" word for degradation issues we are all aware off.
It is used mainly by the marketing departments by soft and hardware manufacturers to scare us and sell us additional tools and additional hardware.

My point: things get old, for sure things like electromagnetic storage degrades, as part of regular wear/tear and also if not used, over time.
No news and nothing to worry if you just use your common sense and replace stuff as you used to do.

 

[Telos]

I took 70 TB of data (about 300,000 files) and stored two copies on separate sets of hard drives. One set went into cold storage, the other set was used actively and moved/copied around between different drives and filesystems. The set in cold storage was spun up once a year to keep the bearing fluid from settling. The active set was moved/copied around in its entirety maybe 5 times in total (say, 350 TB of transfers). All RAM was non-ECC. Drives were consumer models across a mix of all brands. The active set spent one year in ZFS under FreeBSD (when I was feeling particularly paranoid about bitrot and wanted file CRCs), the rest of the time in NTFS under Windows 7.

After 7 years, I ran an MD5 check on both sets of data.

Read on...

 

[EAZ1964]

Exact my point, just electromagnetic degradation on 7 year old, partly cold stored unused disks. 12 files on 300.000 0.004%. Nothing to worry about, certainly not if you have double backups or change your disks before they reach EOL.
No panic, nothing new.

I would like to add (set in perspective) that a risk for full (100%) data loss in a 3 disk RAID 5 in 10 years time is around 0.005%, which means a full data loss risk is way higher than those few in the quoted "bit rot" issue.

RAID Reliability Calculator | Simple MTTDL Model | ServeTheHome

ServeTheHome's RAID reliability calculator based on a simple MTTDL model based on a Poisson distribution and various user inputs about their environment

www.servethehome.com

 

[Deleted member 5784]

I tend to agree. I have yet to knowingly encounter true 'bit rot' in any of my computing activities, and being a fairly old fart I have data archives going back to 1990 at least. Then again, I've not gone explicitly looking for it either.
I use ZFS for my main storage nowadays, but this is primarily for its ease of use / centralisation of LVM / fs / RAID layers rather than its checksumming.

I understand why you'd want this level of security if you are a financial institution or landing spacecraft on the moon with the data - the cost of 'just in case' protection is worth it vs the disastrous consequences of an even very low chance of bit-flips - but I think it is an issue that has gained currency via internet forums more than from real life incidents.

Corporations do all sorts of stuff in the name of compliance, which aren't necessarily reflected in real life situations.

 

[zombiephysicist]

I just had a bunch checksum error on my synology raid with 4.5year old (crap) iron wolf drives. Not a myth.

 

[Deleted member 5784]

I just had a bunch checksum error on my synology raid with 4.5year old (crap) iron wolf drives. Not a myth.

How do you know it was a bit flip? The fact that it was 'a bunch' suggests something else, like a dodgy cable or that one of your 'crap' drives was dying. By their nature, bitflips have to be single, and random.

Drives die all the time. We're talking specifically about 'bitflips', ie when a cosmic ray or similar causes a single, random bit to change its value without any prosaic hardware / dead disk stuff being the cause.

 

[zombiephysicist]

How do you know it was a bit flip? The fact that it was 'a bunch' suggests something else, like a dodgy cable or that one of your 'crap' drives was dying. By their nature, bitflips have to be single, and random.

Drives die all the time. We're talking specifically about 'bitflips' here, ie when a cosmic ray or similar causes a single, random bit to change its value without any hardware / dead disk stuff being the cause.

Bit rot refers to things other than a bit flip IMO. Particularly the "rot" part, of decay, which likely is from a drive starting to go, or a weak part of the drive, over the years succumbing.

Bit flips, are bit flips. Neutrinos hitting ram, ARE, a thing, and now happen with a frequency of once a day or more, if you have enough ram.

Computer Device Makers Ignore Bit-Flip Errors and Data Corruption

John Kheit wants to know why all modern device makers haven't moved to ECC memory considering the problem of bit-flip errors.

www.macobserver.com

I'm not sure how neutrinos interplay with hard drives. I guess it could flip a bit there too, but never studied the topic.

 

[Deleted member 5784]

Bit rot refers to things other than a bit flip IMO. Particularly the "rot" part, of decay, which likely is from a drive starting to go, or a weak part of the drive, over the years succumbing.

Ok you have redefined 'bit rot' to mean any sort of failure, mechanical or otherwise, of a drive.
That's your prerogative, but it then becomes a discussion without meaning because, afaia, nobody is disputing that drives can fail, over time, for mechanical reasons.
Believe me, myself and the posters above had a more specific definition of 'bit rot' in mind when they posted

 

[zombiephysicist]

Ok you have redefined 'bit rot' to mean any sort of failure, mechanical or otherwise, of a drive.
That's your prerogative, but it then becomes a discussion without meaning because, afaia, nobody is disputing that drives can fail, over time, for mechanical reasons.
Believe me, myself and the posters above had a more specific definition of 'bit rot' in mind when they posted

Yea, because you know the "true" meaning and one else can. Because the "rot" part of a brand new drive being hit by a neutrino sounds so apropos of 'bitROT'. And it's not any type, because I'm saying specifically not a bit flip.

If youre using the term incorrectly, don't be surprised if discussions get off track:

bit rot — definition, examples, related words and more at Wordnik

All the words

www.wordnik.com

Definition of BIT ROT

the tendency for digital information to degrade or become unusable over time… See the full definition

www.merriam-webster.com

Definition of bit rot

What does bit rot actually mean? Find out inside PCMag's comprehensive tech and computer-related encyclopedia.

www.pcmag.com

Bit Rot

This definition explains the meaning of Bit Rot and why it matters.

www.techopedia.com

 

[Deleted member 5784]

I dont think a dictionary is going to solve our communications impasse...I'll repectifully bow out of this one and leave the floor to you, sir.

Wishing you a great NYE!

 

[zombiephysicist]

I dont think a dictionary is going to solve our communications impasse...I'll repectifully bow out of this one and leave the floor to you, sir.

Wishing you a great NYE!

Happy New Year to you too!

 

[EAZ1964]

Nice discussion!
The issue is that every error, even checksum errors are now called bitrot, while it can be caused by many hard disk failures. The term is useless in my world...

 

[Telos]

We can quibble over the definition of bit rot,, but this began with a discussion on data scrubbing, which is not specific to bit flips.

 

[Jan Janowski]

Saw a program on how radiation from space can change a bit of ram. Or cause a blue screen. Program indicated data change would be in a power of two, if it was in data calculated, as it was one bit that was toggled.

 

[RonV42]

When I ran Windows Home Server I used to run a couple of utilites to validate the integrity of the data. One was intelligent re-write. I would keep track of sectors that have data and if they weren't accessed over a period of time it would do random reads against them and also a re-write and checksum analysis. And it never detected any "rot" but did lead to discovering bad sectors on two WD drives.

Many of these utilites dont get to see the real ECC rate of the data as stored since the drives proprietary firmware manages the physical layer of the drive.

 

[Coop777]

bad sectors would translate to possible file corruption......