2y ago

Most efficient and economic way to store about 20 TB of data

Hey gais, pretty much the title. So far I was buying HDDs every few years always having a backup, had some drives fail tho. Today I was visiting a local data centre and they are using these cool but expensive high TBW enterprise TLC SSDs, (Samsung, Micron, Kioxia).

I know shiz about data preservation, if I buy one of those, do you think they are going to last longer without failing? If I lets say give them a power up once a while?

But it's probably still way cheaper to just swap bad sector HDDs.

18 comments

Depends what you want/need. You could just buy a 20TB disk (or two 10TB disks is probably cheaper). If you want redundancy, get (3) 10+ TB disks in a RAID5 array or raidz1 pool. If you want more redundancy, step up to RAID6 or raidz2.
Your first though, to buy an SSD may head into problems if they are unpowered for a long time in your shelf. They need power to keep the data
TBW is a measure of performance/resiliency for drives being written to very frequently.
You use the word "backup" which implies infrequent writing. "Redundancy" is what we call duplication of data that is copied very frequently. Very different application. So how frequent are you making these copies?
In the 20TB range, a few offline HDD are by FAR the most economical backup solution, even in triplicate. As far as reliability, the MTBF of an offline HDD stored properly is vastly greater than the upgrade cycle. Even less of a concern with triplicated backups.
The most economic and efficient way of storing around 20 TB of data is to buy an HDD and store it on that. Your best options are probably either a Seagate Exos X20 or a Western Digital Elements. Enterprise SSDs are possible options,but far more expensive than an HDD. You especially do not want to buy an Enterprise SSD, keep it offline and power it on occasionally.
Instead of high quality expensive drives, consider more of the medium quality drives with more copies. And HDDs are much cheaper than SSDs at high capacities.
Those data centers need drives that are accessed 24/7 by many users simultaneously. They have perfect operating conditions such as temperature, don’t care as much about noise, etc. That’s not your case.
Consumers need consumer NAS drives, not enterprise drives.
Two 20 GB drives in RAID-1 with a 3rd 20 GB drive as off-site backup will give you a pretty good level of data security.
It’s not 3-2-1, but it’s a lot better than just a single 20 GB drive. :)
- That moment when you realise they said "TV" and you're thinking in GB. You became old today.
  
  Lol, I became old a looong time ago. Back in my day, drives were measured in MB, and we liked it! shakes cane
- Where do I find 20GB drives? 🤔
  
  2004
  
  From very old computers, lol!
Right now, two cheap 20 TB HDDs in a mirror. Use ZFS for the mirror and you’ll have ultimate data immortality. As long as both drives persist.
(RAID isn’t a backup.)
SSD is not archival storage. The NAND needs Power or it will lose its charge over time. Maybe not in one to 2 Years but maybe in 5-10.
If its active yes hdds are still cheaper unless you need less then 10tb
For 20 TB of computer measured storage you need a 22 TB hdd. ( Due to the way hard drive manufactures market vs how computers read disk size, they use different base systems)
I recently got 20 TB Seagate EXOs for $260 on eBay sold new by Newegg .(20 TB EXOs has 18 TB usable as read by Windows and Linux )
- This is TiB vs TB. Base 2 vs base 10.
  https://en.m.wikipedia.org/wiki/Byte#Multiple-byte_units
In my experience, HDDs fail more often than SSDs, but when HDDs fail they usually do so gracefully, losing just a few sectors at first. When SSDs fail, it is usually catastrophically.
Additionally, my guess is that HDDs are probably less likely to fail if sitting on a shelf, unpowered, for a long time, than if they're powered up and running. But SSDs might be more likely to fail if sitting on a shelf, unpowered, for a long time.
If you don't need to access it much and just want to archive it, tape is probably a contender.
How did your drives fail? Did you ever just try to save them? Or just toss em?

18 comments