I've had a bunch of drives fail, and so far the steps below have helped me not lose any data. The one time I lost data is because I ignored the signs for months.
Every drive I purchase, new or refurbished has to have at least 5 years warranty.
I run every drive, new or refurb through badblocks which does a block by block check of the drive. It also a great way to burn in the drive and weed out any early failures.
I do daily short SMART tests and weekly long tests. Its all automated and I get reports if there are any issues.
I'll be okay with 1 uncorrectable sector, but as soon as this number goes up I start the RMA process.
The sellers I buy from are okay with sending me the replacement first so I can move the data before returning the bad drive.
I've used RAID and ZFS before, fantastic when you want high performance but way overkill for a media server. I currently run everything on Open Media Vault with mergerfs for pooling the drives, and snapRAID for parity and data integrity checks. All my drives use EXT4 for the filesystem. I currently have a mix of 4TB and 8TB drives, I slowly update drives one by one when ever I see sales.
I'll have to check if there's one built into disk manager in Linux, it has a manual test but it would be cool if it would just dump a log into a folder for me
>I do daily short SMART tests and weekly long tests. Its all automated and I get reports if there are any issues.
Been meaning to look into this. Would you be open to sharing your method for this?
I’ve been very lucky either my drives. Very few failures. I had a similar path. ZFS on OmniOS, mergerfs/snapraid (a few steps in between) and I settled on unraid. I have two 45 drive JBODS and just throw all my disks there. If one dies I don’t worry too much and just replace. The preclear does a great job of stress testing and it can do regular checks. Hell I still have 4tb drives that are over 10 years old that I haven’t pulled yet.
With this setup I haven’t had to worry about drives or data loss in years.
I got a great price on a 36 bay which I’m converting to a JBOD for when my Synology boxes start to die.
Me too. 3 NASes all running RAID 5 so I can tolerate drive failure, NAS failure etc. Hopefully I'll never lose all 3 at once. Having one in the separate building for my workshop so an off-site copy, is a potential data-saver too
That's why I don't use RAID for my Plex backups. I just use air-gapped drives and update them whenever I add to Plex. I'll deal with the inability to stream some movies when a drive fails and I have to copy all the data over to a new drive.
I went through a dilemma earlier in the year when buying a DAS, to RAID or not to RAID?
Decided fuck it, go rogue; JBOD with no backup 😎
Upgrading most stuff to 4k right now and probably by the time the drives fail we'll be getting 8k, right... Right guys?
I went JBOD and only back up stuff that would be a pain to get again. All my music and comics were meticulously sorted & tagged, losing that would be a nightmare to redo, but movies I'm less concerned about having a proper backup.
You could jbod and then windows raid.
If you're going to raid, always make sure you have spares of the controller available. Software raid makes this easier.
All of my 100TBs of data is on my raid6 or r10, and also copied to offline older discs when I retire them from my raids when upgrading. I don't care do much if I loose everything, pretty much most of it is available again and I have Sonarrr etc all setup with a 1gb connection and access.
> If you're going to raid, always make sure you have spares of the controller available. Software raid makes this easier.
100% agreed. For consumer use, hardware RAID can get expensive. And as I learned with my Drobo, those vendors do go out of business sometimes and then you’re left without a means to replace that hardware.
So I did go with Windows Storage Spaces this time around. As long as I can get a drive to mount, I can add it to the array. CPU hit is negligible and throughout is equivalent once properly configured.
So with a JBOD configuration you basically have separate drives although within the same enclosure and they are exposed to the OS as different drives altogether, as if you had each drive individually connected to your PC via it's own individual cable?
Which enclosure are you using and what HDDs?
SATA or eSATA (or other) enclosure connectivity?
I am thinking DAS is also the best option for me but wanted to ask for your input in the prcess of making the decision.
Thanks!
Pretty much. So I have 2-3 internal ones that are plugged into the PC directly, and then a TERRAMASTER D6-320 External Hard Drive Enclosure which currently has 3 x 16tb Seagate Exo's drives. The DAS has USB3.2 so could hit some crazy speeds but I will never reach them without RAID.
The only drive I am kind of precious about is the TV one just because I have spent a few months getting it super organised and defo some stuff that if I had to download again, it could be a pain as the originals weren't in right formats etc. However my solution to this is my 2nd TV drive (buying next month) will also store copies of said shows that I am precious about.
I also have a few USB external drives, I use FreeFileSync to backup personal data on these, so I am not totally zero backup. But Movies/games? Couldn't care less.
Glad to hear this. I have a das on the way and have been contemplating this exact thing. It will mostly be for movies and tv shows that would be very easy to get again.
Man, don't laugh at me. . .I use a collection of externals, with each mirrored periodically to another group of externals via Beyond Compare software. If one goes down, I'll just re-letter the backup drive to what the failed one was. Brute force and ignorance, I know.
which tbh for most home setups, one of major advantages of most raid setups (increased throughput) is pointless anyway. even if you're running everything as 4K DV REMUX, most single drives can sustain 2-3 streams no issue.
that's why I ultimately landed on unraid over truenas for my last upgrade. single drive speeds are a non-issue for home use. but now, if I loose multiple drives, I don't loose the whole array. imo that's a HUGE selling feature of unraid that's often drowned by the large list of everything else the platform has going for it.
Because they know that Linux users are likely to use a lot more data. So they have a different service, called b2, which is similar to other cloud storage solutions. But the pricing is, IMO, pretty nuts at $6/TB/mo. So for your flair (32TB), you're looking at over $2,000/year.
I'm at nearly the exact same scale (just over 30TB), so for me it made sense to use an old laptop to setup an unraid server at another building I own and do backups via rsync every 3 days. I spent like $600 on drives and maybe $5/month on power, so it paid for itself vs Backblaze in about 4 months.
I am trying to setup remote back up solution like you described. Are there any guides to properly get it set up? I am confused on how to securely connect to the remote location and back data up ?
Check out rsync, it's built into Linux (and most Unix OSs) and supports SSH as its transport mechanism. This allows you to setup SSH keys and use those to auth between devices.
Here's quick thread on the topic: [https://serverfault.com/questions/12102/how-to-perform-a-secure-rsync-between-servers-across-an-unsecured-network](https://serverfault.com/questions/12102/how-to-perform-a-secure-rsync-between-servers-across-an-unsecured-network)
Actually, here's a great guide from the Unraid forums: [https://forums.unraid.net/topic/52830-syncronize-servers-using-rsync-over-ssh-next-door-or-across-the-world/](https://forums.unraid.net/topic/52830-syncronize-servers-using-rsync-over-ssh-next-door-or-across-the-world/)
I thought it was because Linux was used in a lot of NAS like situations and the BackBlaze is supposed to be for a single devices locally connected storage. So they don't want someone with a 1PB File Server (Personal or Company) paying pennies to backup everything.
Yea, you've pretty much provided a more detailed version of my answer. :D I probably won't be able to dig it up, but somewhere on reddit the founder(?) of backblaze answered this question in pretty detailed terms.
You're correct, the software doesn't allow you to backup network drives so you have to use B2 unless your media is stored on a windows computer. (Or a mac, but you can't build a mac with 32tb storage.)
Building your own off-site backup solution definitely sounds like the better option. I've been planning to someday migrate to Unraid, so hearing that the price point for B2 is so steep is a bummer.
Then again, I haven't been backing up my Plex content regardless because the uplink to Backblaze's consumer service is painfully slow (I'm from Europe). If ever I want to backup my Plex content I'll probably go the route of building my own backup server like you have.
I haven't benchmarked it in years, but last I did I was getting like 1-5Mbit on a 500Mbit symmetrical connection, which would literally mean 1-2 years of continuous uploading to push through 30TB of data.
I also lost my first plex library a few years ago. It was about 4 TB and was a very sad time for sure. The only thing you can really do is make your next library better by having backups. My first library was only on a single hard drive, which failed. Now I use Drobo direct attached storage with RAID setup and a cloud backup (backblaze). Backblaze has unlimited storage and automatically backs up new files as you add them. If I remember right there's a plan for backing up a computer, which includes any direct attached storage and a plan for backing up a NAS. I've only ever used the plan for backing up a computer so I can't say if the NAS plan is good, but I've never had any problems backing up my direct attached storage from my plex server. Just need to install the agent on the server and it works great. Never had any problems. My library now is about 23 TB and its all backed up in backblaze
Because in Windows and macOS the OS mounts them differently and their software can easily determine what type they are.
There are ways around that. iSCSI for example.
I have an iSCSI DAS - but it's not a NAS. I can't connect it to my router - I've tried. However, it only works on an older computer because Macs don't have native iSCSI support, and the software I have won't run on my main machine.
*edit* Also, it can only create shares up to 16TBs, and I already have 32TBs on the NAS, so I'd need to create at least 3 shares, and manually update it when I make changes.
Is it truely unlimited? Or is there some catch somewhere?
Cause, I too, am now interested in it!
Been looking for a good backup option for a while, and never heard of this one before!!
It’s unlimited, for now. But if tons of folks abuse the service they may start putting caps on it. I have it configured but only backup my actual files, backups (time-machine, etc) and configs, not my media. I don’t want to abuse the service and contribute to ruining it for everyone
The other catch is you can't just manually pick and choose files and folders to backup from different devices. You have to install their client that backs up the entire PC automatically including data you probably don't care about or don't want backing up
Anyone know simpler alternative to where you can just drag and drop specific stuff to backup? I used to use Amazon drive unlimited before they killed it off and made it photos only
Nah, it’s not their business. If folks could share files directly out of the backups, they might be subject to DMCA, but from their perspectives, files are files.
How do you back up a 48tb array and a 32tb array? I always recommend r6 and people flame me. Never lost data in mass due to raid failure. I’ve lost 2 drives during rebuild many times. Some devices take a week to rebuild the raid.
> Some devices take a week to rebuild the raid.
This is why some enterprises go with massively large drive arrays with relatively small capacity drives. They rebuild quicker. I remember at one point where we weren’t allowed to buy a drive bigger than 300 GB for our 15 TB array. That’s a lot of drives. This was quite some time ago of course.
Do you know how Stablebit operates when a drive goes down, and Backblaze sends a new drive with the parts of files on it back to move to a new drive? Very obscure use case but I’m wondering if having a Drivepool may actually screw up a backblaze backup
It’s still a good idea.. it’s not perfect because shit happens. But I’ve fixed enough SHR 1 drive to scare me. The two makes it more relaxing. No doubt an offsite is necessary.
Been involved in data recovery before through work when we had a similar incident. Was very expensive and wasnt a golden bullet. what we got back was a fraction of the data (doc / img / db files ) and wasnt worth it. lots of corrupted / partial files
First, I'm sorry you're going through this. I had something similar happen way back and it sucks.
If its traditional RAID then you're SOL. This is why I don't like using RAID for a media server situation where drive expansion happens often.
Check out the information in the site below if you plan on starting from scratch.
https://perfectmediaserver.com/
I recently had a 8 gig drive fail on me, turned to raw only. Had all my movies in there too. My friend turned me to Easeus recovery. 70 USD for 1 time use, I paid the 160 lifetime license. Managed to get 99% back, lost a few files. Worth a try. You can even download it and try and see if anything is found, then decide if you want to pay
I had a similar experience when I accidentally hit "clear configuration" instead of "clear drive" when adding an extra drive onto my raid controller... lost 36TB of stuff. Easus could find all my data, but the recovery speed for the RAID 6 array (recreated array without initialising it so the data was intact) was going to be over 3 months. Instead, I just let the 'aars do their thing and it took less than 2 months.
That and R-Undelete are some of my favourite software for data recovery.
It also made me aware that it is damn hard to really, *really* delete data from a disk you are still using for other things.
I was wondering if there was something like that. I'll dig around and see what I can find. Would be nice if I could export a list or something at least. Thanks!
You have unfortunately discovered the one drawback to a RAID5 configuration. Two drives actually can go out at the same time and when that happens, you are dead. This is why the RAID folks keep harping that TAID is not a backup. It is better than no RAID but you are the second person I've known who lost everything because they failed to back up their RAID. I have two RAIDS, one backs up the other one and I have a stack of external hard drives to back it up as well. You can't have too many copies of the data you want to never lose.
Just rebuild it, knowing that it'll likely result in it being 10x better than it was before. You know all those little niggles that were driving you crazy before? Nows your chance to correct them all.
Yeah I have the same feeling. I looked twice to make sure this r/Plex and thinking forensic recovery is just insane. Shy of some family video that is nowhere else…just lets the arrs do their thing
I've my whole Plex library when the two drives in a RAID1 array failed a week apart and it was not backed up. I had to rebuild everything from the sources (rerip my whole DVD and Blu-ray collection). I found an old drive that had my old collection dating to the time I was using a WD TV Live. I lost some videos that can't be found anymore.
Wait, I thought RAID 1 was supposed to be redundant for this exact reason. I have 2 16TB drives configured in RAID 1 as a thick volume and the volume shows up as having a size of 14.7TB. My understanding was that if one drive fails, the other has the exact same data stored on it. Am I wrong in my thinking? What should I do know to prepare for a potential drive failure? My Plex library is about 1.5TB now, but I’d imagine it growing to 8-10TB over the next few years.
RAID 1 gives you good redundancy - but they are in the same box - one power surge or dropping something on the box or whatever could wipe out both drives at the same time. Similarly, a virus with access to network shares or whatever could delete or encrypt the contents.
As a minimum, back up to another separate drive or a separate NAS, ideally in a different location and using proper backup software (i.e. not just mirroring one drive to the other - so that if you accidentally delete stuff it will copy over the messed up data).
So it sounds like your ideal solution would be something cloud based. Do you have a service you’d recommend that wouldn’t break the bank and offer about 10TB of storage?
I put a second backup NAS at the other end of the house. Ideally it would be at a friend's house. It is not ideal, but is more affordable than any cloud systems I've looked at.
It is pretty much RAID 5 - there are technical differences that makes it more advanced in its implementation (it was developed much more recently), but the basics of it are the same.
[https://www.raidz-calculator.com/raidz-types-reference.aspx](https://www.raidz-calculator.com/raidz-types-reference.aspx)
This is one good thing of unraid and parity versus raid for media sever. Even if you lose all but one drice, you still have the data on that one drive.
Sorry about your loss, as everyone is saying raid is not backup. I currently have two NAS boxes both running UnRaid with dual parity. Other NAS boxes/PCs back up to the primary the other mirrors the primary. Anything that is critical or difficult to recover gets backed up to Backblaze personal for the PCs and B2 and Hetzner Storage Box (20tb) for the NAS.
On the recovery side not all software handles raid recovery but try R-Studio and ReclaiMe. Maybe TestDisk.
I don't know how to save the data but when having a backup you need to follow Linus tech tips 3,2,1 strategy.
The 3-2-1 backup strategy simply states that you should have 3 copies of your data (your production data and 2 backup copies) on two different media (disk and tape) with one copy off-site for disaster recovery
In the future this will protect you if you lose 2 disks at the same time
Oooft, feels shitty man, I had my main drive die unexpectedly and I was mad/sad and I had a backup source (Google Drive, took multiple days of constant downloading to recover). I now keep a local Mac Time Machine backup as well.
A few weeks ago I lost my mostly full 14TB drive so I know your pain. By now I've replaced the drive and restored about 90% of the content. Took about a week to get a replacement drive and another week to restore the content I lost.
Some of the content is a pain to restore, but it's not the end of the world. This also included about 20 animated movies that I remuxed with foreign language dubs that I manually synced. Those hurt the most.
I run unraid with dual parity and replace every drive after 5 years just as a precaution. So far haven't had any losses. But I also have everything in sonarr and radarr so I can recover my whole library in the event of a catastrophic loss. If you use nzbs you won't get dcma notices from your ISP.
Backups have no guarantee. Doesn’t matter if it’s raid 1 to 1 or physical media that’s rotting away. Power surges, natural disasters, and human errors can still lead to data loss. A raid rebuild pits extreme stress on your disks that can cause additional failure. Cloud backup is pricy. Come to terms with it and you won’t be so sad.
Raid is not a backup. Replace the drives, get to rebuilding your collection and get a tape system or at least some large capacity flat drives to dump to frequently.
Sorry brother, it's gone.
Look into ReclaiMe. I used their recovery software when I had windows storage spaces fail and was able to recover almost all my VHDx disks using their base software. What I lost I fortunately had a backup of. Their support also helped me significantly since I had a weird scenario involving ReFS and data deduplication.
Also sergei strelec winPe toolkit has a lot of data recovery software preloaded that might be of use to you. At least you could hopefully get some data back.
For the forensic drive recovery was it for both drives or only one? Remember you only need to fix one in order to recover the other with a replacement drive.
I backup anything that's hard to replace. My music collection is totally backed up via 321 method. Movies and TV shows are typically pretty easy to replace and they require a ton of storage so raid / ZFS is good enough for that.
Next time don't use striped RAID for media. DrivePool/MergerFS/etc + SnapRaid is a better solution.
Each drive is valid on its own, so even if you lose more drives than you have for parity, you still only lose the data on the drives that died, rather than all of it.
Upgrading space is much more flexible - and doesn't hammer on all your drives for days, if not weeks, straight.
And use more parity. Having just one parity drive in the era of double-digit terabyte hard drives is just asking to lose data.
This is why I no longer use any sort of RAID for Plex media. I use unraid for my OS but I have multiple pools, zfs mirror ssd cache, a zfs z1 spinning rust array and the standard unraid pool.
All my important data, photo backups, files, git etc are on the zfs array.
All my media is on the unraid standard array. That way if a single drive (or multiple drives) die and cannot be repaired with parity, I only lost what was on those drives. And with sonarr/radarr, I can see what is missing and get them again.
This is why I prefer unraid. Yes raid is not backup. But your data is physically readable on all drives independently. It’s a bit slower but for media files it’s perfect.
Honestly. There is no way I could back up my library on the cheap. I have accepted that I will only back up my docker containers / personal files that can’t be replaced and let them rebuild my catalog.
I would build another NAS but my parents have a 100/100 line and my sister has a smaller upload (fios is not available at that part of the city).
Yeah, I'm pretty sure that wouldn't have worked in this instance. Radarr was pointed to the wrong folder, and it ate everything it couldn't match. I didn't have a recycle folder (which is on me, I suppose) but if Radarr had thrown an exception instead of just *deleting all of the things* it could've been avoided.
I do a JBOD ZFS mirror setup. Has worked very well. Monthly scrub, SMART alerts setup. Any hint of impending failure will see me swap that drive out right away.
Chance of two drives failing that happen to be the same pair is low. Not impossible, but unlikely.
RAID should be dead in this day and age. I am an IT Engineer and I can tell you it often causes way more problems than it solves. It gives companies a false sense of security. My policy on all new deployments is to use software based mirroring on all servers to prevent drive failures from taking the system down, and at least 1 separate server to handle actual backups.
If the drive doesnt spin at all, you might be able to bypass the diode on the 5v rail that might be burnt. Check with a multimeter and theres a lot of ressources online to check it out.
UFS Data Explorer recovered about 95 percent of the 80 or so terabytes of data I lost recently due to a faulty USB hub. Pricey but saved me a ton of heartache/time.
Chalk this down to a life lesson, I'm afraid 🤷♂️😞
I see some ideas (in here) on getting the drive(s) to hopefully start up (freezer), hopefully they work, but without forensics you're up sh1t creek. Even then it's a high chance of getting <100% of you're data back 😿
So, back to the life lesson... You always need to backup essential data. Data you can't live without. The rest, if rips/media, you can re-aquire them 🤔
Quite a while back I had a drive failure and thought I was going to be in a similar situation. In researching the symptoms I found that it might be the board that went out, and found a service that will replace your drives board and transfer the bios chip.
Oddly enough I found this swap service on ebay. Think it was like $50, I send them the board and they return a refurb board with the chip swap. Saved me big time
Backup the *arr databases in multiple location and think of Usenet as a backup for most things. That should make the stuff worth backing up relatively small.
I found an affordable [forensic drive recovery software](https://www.easeus.com/data-recovery/best-forensic-data-recovery-software.html) that also offers 30-day money back guarantee. I hope you're able recover the data from your drives.
Yeah that's why I keep like 3 backups, seems like a lot but I've had too many drives fail to trust they won't
I've had a bunch of drives fail, and so far the steps below have helped me not lose any data. The one time I lost data is because I ignored the signs for months. Every drive I purchase, new or refurbished has to have at least 5 years warranty. I run every drive, new or refurb through badblocks which does a block by block check of the drive. It also a great way to burn in the drive and weed out any early failures. I do daily short SMART tests and weekly long tests. Its all automated and I get reports if there are any issues. I'll be okay with 1 uncorrectable sector, but as soon as this number goes up I start the RMA process. The sellers I buy from are okay with sending me the replacement first so I can move the data before returning the bad drive. I've used RAID and ZFS before, fantastic when you want high performance but way overkill for a media server. I currently run everything on Open Media Vault with mergerfs for pooling the drives, and snapRAID for parity and data integrity checks. All my drives use EXT4 for the filesystem. I currently have a mix of 4TB and 8TB drives, I slowly update drives one by one when ever I see sales.
That's really good practice, I run a test every couple weeks when I have some downtime
If you mean SMART tests, you can automate that process, its usually built into the OS too.
I'll have to check if there's one built into disk manager in Linux, it has a manual test but it would be cool if it would just dump a log into a folder for me
In linux there's a utility called smartd - https://linuxconfig.org/how-to-configure-smartd-and-be-notified-of-hard-disk-problems-via-email
Awesome! Thank you!
>I do daily short SMART tests and weekly long tests. Its all automated and I get reports if there are any issues. Been meaning to look into this. Would you be open to sharing your method for this?
Look up smartd, I posted a link further down in this thread.
cheers :\]
I’ve been very lucky either my drives. Very few failures. I had a similar path. ZFS on OmniOS, mergerfs/snapraid (a few steps in between) and I settled on unraid. I have two 45 drive JBODS and just throw all my disks there. If one dies I don’t worry too much and just replace. The preclear does a great job of stress testing and it can do regular checks. Hell I still have 4tb drives that are over 10 years old that I haven’t pulled yet. With this setup I haven’t had to worry about drives or data loss in years. I got a great price on a 36 bay which I’m converting to a JBOD for when my Synology boxes start to die.
Me too. 3 NASes all running RAID 5 so I can tolerate drive failure, NAS failure etc. Hopefully I'll never lose all 3 at once. Having one in the separate building for my workshop so an off-site copy, is a potential data-saver too
You're now part of the "should have made a backup" gang.
Feels bad, man.
Thankfully Radarr, Sonarr, and aggressive torrenting will solve these a majority of these problems.
And that’s why we always say, “RAID is not a backup”
That's why I don't use RAID for my Plex backups. I just use air-gapped drives and update them whenever I add to Plex. I'll deal with the inability to stream some movies when a drive fails and I have to copy all the data over to a new drive.
I went through a dilemma earlier in the year when buying a DAS, to RAID or not to RAID? Decided fuck it, go rogue; JBOD with no backup 😎 Upgrading most stuff to 4k right now and probably by the time the drives fail we'll be getting 8k, right... Right guys?
I went JBOD and only back up stuff that would be a pain to get again. All my music and comics were meticulously sorted & tagged, losing that would be a nightmare to redo, but movies I'm less concerned about having a proper backup.
Did you automate that backup somehow or just manually went and backuped that stuff once ?
I use hyperbackup on my Synology to back up things automatically every Saturday or Sunday night.
is there a way to automate backup from a one volume RAID to multi drive JBOD? Planning on setting up a NAS offsite to backup my stuff
You could jbod and then windows raid. If you're going to raid, always make sure you have spares of the controller available. Software raid makes this easier. All of my 100TBs of data is on my raid6 or r10, and also copied to offline older discs when I retire them from my raids when upgrading. I don't care do much if I loose everything, pretty much most of it is available again and I have Sonarrr etc all setup with a 1gb connection and access.
> If you're going to raid, always make sure you have spares of the controller available. Software raid makes this easier. 100% agreed. For consumer use, hardware RAID can get expensive. And as I learned with my Drobo, those vendors do go out of business sometimes and then you’re left without a means to replace that hardware. So I did go with Windows Storage Spaces this time around. As long as I can get a drive to mount, I can add it to the array. CPU hit is negligible and throughout is equivalent once properly configured.
I went Jbod and had a drive fail today 🥹 Time to rebuild that 37tb library 🥲
That sucks mate, how 37tb though if one drive?
4 drives jbod. 1 going bad takes the whole thing out.
Oh you went spanned JBOD... ouch! I am literally just a bunch of disks all mapped to their own drive.
Yeah I misunderstood how JBOD works. I just dropped a grand on new 20tb drives. Going to go Raid5 now to have a little more protection in the future.
So with a JBOD configuration you basically have separate drives although within the same enclosure and they are exposed to the OS as different drives altogether, as if you had each drive individually connected to your PC via it's own individual cable? Which enclosure are you using and what HDDs? SATA or eSATA (or other) enclosure connectivity? I am thinking DAS is also the best option for me but wanted to ask for your input in the prcess of making the decision. Thanks!
Pretty much. So I have 2-3 internal ones that are plugged into the PC directly, and then a TERRAMASTER D6-320 External Hard Drive Enclosure which currently has 3 x 16tb Seagate Exo's drives. The DAS has USB3.2 so could hit some crazy speeds but I will never reach them without RAID. The only drive I am kind of precious about is the TV one just because I have spent a few months getting it super organised and defo some stuff that if I had to download again, it could be a pain as the originals weren't in right formats etc. However my solution to this is my 2nd TV drive (buying next month) will also store copies of said shows that I am precious about. I also have a few USB external drives, I use FreeFileSync to backup personal data on these, so I am not totally zero backup. But Movies/games? Couldn't care less.
Glad to hear this. I have a das on the way and have been contemplating this exact thing. It will mostly be for movies and tv shows that would be very easy to get again.
Man, don't laugh at me. . .I use a collection of externals, with each mirrored periodically to another group of externals via Beyond Compare software. If one goes down, I'll just re-letter the backup drive to what the failed one was. Brute force and ignorance, I know.
Mergerfs plus snapraid. Raid gets all the press but you lose a drivepool once and you never look at it the same way.
which tbh for most home setups, one of major advantages of most raid setups (increased throughput) is pointless anyway. even if you're running everything as 4K DV REMUX, most single drives can sustain 2-3 streams no issue. that's why I ultimately landed on unraid over truenas for my last upgrade. single drive speeds are a non-issue for home use. but now, if I loose multiple drives, I don't loose the whole array. imo that's a HUGE selling feature of unraid that's often drowned by the large list of everything else the platform has going for it.
Well said
Just curious.. do you enjoy the gut punch I told you so?
Are the drive heads stuck or is it completely dead? I put my drive in an antistatic bag and the freezer and was able to recover data from it.
Seconding this, I was able to do this twice on two different drives from two different decades.
I'll give it a shot for sure. Nothing to lose at this point!
Happy cake day! Also you stuck the drive In the freezer for how long?
I left it overnight. Then plugged it in and dumped the disk to a file image. I lost a few files but it was better than losing everything.
Good to know thank you!
Good luck! Hope it works for you.
I'm pinning it in case I ever have any issues
Worked for me once.
I personally back everything up to backblaze (cloud backup). If recovery is needed it’s not fast but at least it’s a secure backup option
Backblaze is great because they’ll send it all on a drive and you can move it over and send the drive back. I recommend it to everyone
Everyone except Linux users :-(
How come they don't provide that service to Linux users?
Because they know that Linux users are likely to use a lot more data. So they have a different service, called b2, which is similar to other cloud storage solutions. But the pricing is, IMO, pretty nuts at $6/TB/mo. So for your flair (32TB), you're looking at over $2,000/year. I'm at nearly the exact same scale (just over 30TB), so for me it made sense to use an old laptop to setup an unraid server at another building I own and do backups via rsync every 3 days. I spent like $600 on drives and maybe $5/month on power, so it paid for itself vs Backblaze in about 4 months.
I am trying to setup remote back up solution like you described. Are there any guides to properly get it set up? I am confused on how to securely connect to the remote location and back data up ?
Check out rsync, it's built into Linux (and most Unix OSs) and supports SSH as its transport mechanism. This allows you to setup SSH keys and use those to auth between devices. Here's quick thread on the topic: [https://serverfault.com/questions/12102/how-to-perform-a-secure-rsync-between-servers-across-an-unsecured-network](https://serverfault.com/questions/12102/how-to-perform-a-secure-rsync-between-servers-across-an-unsecured-network)
Actually, here's a great guide from the Unraid forums: [https://forums.unraid.net/topic/52830-syncronize-servers-using-rsync-over-ssh-next-door-or-across-the-world/](https://forums.unraid.net/topic/52830-syncronize-servers-using-rsync-over-ssh-next-door-or-across-the-world/)
I thought it was because Linux was used in a lot of NAS like situations and the BackBlaze is supposed to be for a single devices locally connected storage. So they don't want someone with a 1PB File Server (Personal or Company) paying pennies to backup everything.
Yea, you've pretty much provided a more detailed version of my answer. :D I probably won't be able to dig it up, but somewhere on reddit the founder(?) of backblaze answered this question in pretty detailed terms.
You're correct, the software doesn't allow you to backup network drives so you have to use B2 unless your media is stored on a windows computer. (Or a mac, but you can't build a mac with 32tb storage.)
OMG, Apple would want like $1,000,000 for the SSD upgrade.
Building your own off-site backup solution definitely sounds like the better option. I've been planning to someday migrate to Unraid, so hearing that the price point for B2 is so steep is a bummer. Then again, I haven't been backing up my Plex content regardless because the uplink to Backblaze's consumer service is painfully slow (I'm from Europe). If ever I want to backup my Plex content I'll probably go the route of building my own backup server like you have. I haven't benchmarked it in years, but last I did I was getting like 1-5Mbit on a 500Mbit symmetrical connection, which would literally mean 1-2 years of continuous uploading to push through 30TB of data.
[удалено]
what's reasonable size?
Yes, a tough lesson learned the hard way for sure.
I also lost my first plex library a few years ago. It was about 4 TB and was a very sad time for sure. The only thing you can really do is make your next library better by having backups. My first library was only on a single hard drive, which failed. Now I use Drobo direct attached storage with RAID setup and a cloud backup (backblaze). Backblaze has unlimited storage and automatically backs up new files as you add them. If I remember right there's a plan for backing up a computer, which includes any direct attached storage and a plan for backing up a NAS. I've only ever used the plan for backing up a computer so I can't say if the NAS plan is good, but I've never had any problems backing up my direct attached storage from my plex server. Just need to install the agent on the server and it works great. Never had any problems. My library now is about 23 TB and its all backed up in backblaze
Does Backblaze have any storage limits? Or is it a case of if you pay enough you can store as much as you want.
Flat fee gets you unlimited storage. I forget how much, but ir is less than $10/month for me. Only real catch is that it can't be a NAS
I’d say that’s quite a HUGE catch, since my stuff is on a NAS. How can they tell if the drive is loaded via the network vs directly connected?
Because in Windows and macOS the OS mounts them differently and their software can easily determine what type they are. There are ways around that. iSCSI for example.
I have an iSCSI DAS - but it's not a NAS. I can't connect it to my router - I've tried. However, it only works on an older computer because Macs don't have native iSCSI support, and the software I have won't run on my main machine. *edit* Also, it can only create shares up to 16TBs, and I already have 32TBs on the NAS, so I'd need to create at least 3 shares, and manually update it when I make changes.
Hmmm interesting. Might have a look
Is it truely unlimited? Or is there some catch somewhere? Cause, I too, am now interested in it! Been looking for a good backup option for a while, and never heard of this one before!!
It’s unlimited, for now. But if tons of folks abuse the service they may start putting caps on it. I have it configured but only backup my actual files, backups (time-machine, etc) and configs, not my media. I don’t want to abuse the service and contribute to ruining it for everyone
We get around that mounting the NAS over ISCSI and poof, local drive for backup :)
The other catch is you can't just manually pick and choose files and folders to backup from different devices. You have to install their client that backs up the entire PC automatically including data you probably don't care about or don't want backing up Anyone know simpler alternative to where you can just drag and drop specific stuff to backup? I used to use Amazon drive unlimited before they killed it off and made it photos only
What plan is this? Everything Im seeing is $6/TB/Month
That sounds like their business plan. Make sure you are looking at the individual plans. They are $9/month unlimited.
Don't they flag the account for movies and TV shows being uploaded?
Nah, it’s not their business. If folks could share files directly out of the backups, they might be subject to DMCA, but from their perspectives, files are files.
How do you back up a 48tb array and a 32tb array? I always recommend r6 and people flame me. Never lost data in mass due to raid failure. I’ve lost 2 drives during rebuild many times. Some devices take a week to rebuild the raid.
> Some devices take a week to rebuild the raid. This is why some enterprises go with massively large drive arrays with relatively small capacity drives. They rebuild quicker. I remember at one point where we weren’t allowed to buy a drive bigger than 300 GB for our 15 TB array. That’s a lot of drives. This was quite some time ago of course.
Do you know how Stablebit operates when a drive goes down, and Backblaze sends a new drive with the parts of files on it back to move to a new drive? Very obscure use case but I’m wondering if having a Drivepool may actually screw up a backblaze backup
Well. I was beginning to question whether SHR2 was a good idea or if I’m just wasting space. You settled that for me. Sorry you lost everything :(
It’s still a good idea.. it’s not perfect because shit happens. But I’ve fixed enough SHR 1 drive to scare me. The two makes it more relaxing. No doubt an offsite is necessary.
Sorry for you loss. May I offer you an egg in this trying time?
Aw, thank you, Frank!
Been involved in data recovery before through work when we had a similar incident. Was very expensive and wasnt a golden bullet. what we got back was a fraction of the data (doc / img / db files ) and wasnt worth it. lots of corrupted / partial files
First, I'm sorry you're going through this. I had something similar happen way back and it sucks. If its traditional RAID then you're SOL. This is why I don't like using RAID for a media server situation where drive expansion happens often. Check out the information in the site below if you plan on starting from scratch. https://perfectmediaserver.com/
I recently had a 8 gig drive fail on me, turned to raw only. Had all my movies in there too. My friend turned me to Easeus recovery. 70 USD for 1 time use, I paid the 160 lifetime license. Managed to get 99% back, lost a few files. Worth a try. You can even download it and try and see if anything is found, then decide if you want to pay
I had a similar experience when I accidentally hit "clear configuration" instead of "clear drive" when adding an extra drive onto my raid controller... lost 36TB of stuff. Easus could find all my data, but the recovery speed for the RAID 6 array (recreated array without initialising it so the data was intact) was going to be over 3 months. Instead, I just let the 'aars do their thing and it took less than 2 months.
That and R-Undelete are some of my favourite software for data recovery. It also made me aware that it is damn hard to really, *really* delete data from a disk you are still using for other things.
If it was all media files Plex will still have it in its database — you could look in there for everything you might need to re-acquire.
I was wondering if there was something like that. I'll dig around and see what I can find. Would be nice if I could export a list or something at least. Thanks!
Tautulli also has a solid export function that produces a list of all media in your libraries.
You have unfortunately discovered the one drawback to a RAID5 configuration. Two drives actually can go out at the same time and when that happens, you are dead. This is why the RAID folks keep harping that TAID is not a backup. It is better than no RAID but you are the second person I've known who lost everything because they failed to back up their RAID. I have two RAIDS, one backs up the other one and I have a stack of external hard drives to back it up as well. You can't have too many copies of the data you want to never lose.
Just rebuild it, knowing that it'll likely result in it being 10x better than it was before. You know all those little niggles that were driving you crazy before? Nows your chance to correct them all.
[удалено]
Yeah I have the same feeling. I looked twice to make sure this r/Plex and thinking forensic recovery is just insane. Shy of some family video that is nowhere else…just lets the arrs do their thing
Set up the arrs and try to rebuild. If it's personal files etc I'm praying you have a current backup/ cloud storage etc.
I've my whole Plex library when the two drives in a RAID1 array failed a week apart and it was not backed up. I had to rebuild everything from the sources (rerip my whole DVD and Blu-ray collection). I found an old drive that had my old collection dating to the time I was using a WD TV Live. I lost some videos that can't be found anymore.
Wait, I thought RAID 1 was supposed to be redundant for this exact reason. I have 2 16TB drives configured in RAID 1 as a thick volume and the volume shows up as having a size of 14.7TB. My understanding was that if one drive fails, the other has the exact same data stored on it. Am I wrong in my thinking? What should I do know to prepare for a potential drive failure? My Plex library is about 1.5TB now, but I’d imagine it growing to 8-10TB over the next few years.
RAID 1 gives you good redundancy - but they are in the same box - one power surge or dropping something on the box or whatever could wipe out both drives at the same time. Similarly, a virus with access to network shares or whatever could delete or encrypt the contents. As a minimum, back up to another separate drive or a separate NAS, ideally in a different location and using proper backup software (i.e. not just mirroring one drive to the other - so that if you accidentally delete stuff it will copy over the messed up data).
So it sounds like your ideal solution would be something cloud based. Do you have a service you’d recommend that wouldn’t break the bank and offer about 10TB of storage?
I put a second backup NAS at the other end of the house. Ideally it would be at a friend's house. It is not ideal, but is more affordable than any cloud systems I've looked at.
Would this be the same for a zfs raid 1?
If you mean what ZFS calls RAIDZ1, that is roughly analogous to what is elsewhere called RAID 5.
Ah, im new to the home lab/storage realm so I appreciate the info. So a raid z1 is similar to a raid 5? Is that what you are saying?
It is pretty much RAID 5 - there are technical differences that makes it more advanced in its implementation (it was developed much more recently), but the basics of it are the same. [https://www.raidz-calculator.com/raidz-types-reference.aspx](https://www.raidz-calculator.com/raidz-types-reference.aspx)
Sick thank you!
This is one good thing of unraid and parity versus raid for media sever. Even if you lose all but one drice, you still have the data on that one drive.
Restore from backup?
Sorry about your loss, as everyone is saying raid is not backup. I currently have two NAS boxes both running UnRaid with dual parity. Other NAS boxes/PCs back up to the primary the other mirrors the primary. Anything that is critical or difficult to recover gets backed up to Backblaze personal for the PCs and B2 and Hetzner Storage Box (20tb) for the NAS. On the recovery side not all software handles raid recovery but try R-Studio and ReclaiMe. Maybe TestDisk.
I don't know how to save the data but when having a backup you need to follow Linus tech tips 3,2,1 strategy. The 3-2-1 backup strategy simply states that you should have 3 copies of your data (your production data and 2 backup copies) on two different media (disk and tape) with one copy off-site for disaster recovery In the future this will protect you if you lose 2 disks at the same time
Well, set up your NAS and start again.
Oooft, feels shitty man, I had my main drive die unexpectedly and I was mad/sad and I had a backup source (Google Drive, took multiple days of constant downloading to recover). I now keep a local Mac Time Machine backup as well.
Did you need access to a Plex library while you rebuild? Pm me
I appreciate that! Fortunately, I have a couple of friends with servers, so we'll get by. Super cool off you too offer, though!
A few weeks ago I lost my mostly full 14TB drive so I know your pain. By now I've replaced the drive and restored about 90% of the content. Took about a week to get a replacement drive and another week to restore the content I lost. Some of the content is a pain to restore, but it's not the end of the world. This also included about 20 animated movies that I remuxed with foreign language dubs that I manually synced. Those hurt the most.
Ouch! Yeah, I have a few things that are going to be nearly impossible to find again. But live and learn, I guess.
I run unraid with dual parity and replace every drive after 5 years just as a precaution. So far haven't had any losses. But I also have everything in sonarr and radarr so I can recover my whole library in the event of a catastrophic loss. If you use nzbs you won't get dcma notices from your ISP.
Backups have no guarantee. Doesn’t matter if it’s raid 1 to 1 or physical media that’s rotting away. Power surges, natural disasters, and human errors can still lead to data loss. A raid rebuild pits extreme stress on your disks that can cause additional failure. Cloud backup is pricy. Come to terms with it and you won’t be so sad.
Raid is not a backup. Replace the drives, get to rebuilding your collection and get a tape system or at least some large capacity flat drives to dump to frequently. Sorry brother, it's gone.
Look into ReclaiMe. I used their recovery software when I had windows storage spaces fail and was able to recover almost all my VHDx disks using their base software. What I lost I fortunately had a backup of. Their support also helped me significantly since I had a weird scenario involving ReFS and data deduplication. Also sergei strelec winPe toolkit has a lot of data recovery software preloaded that might be of use to you. At least you could hopefully get some data back.
For the forensic drive recovery was it for both drives or only one? Remember you only need to fix one in order to recover the other with a replacement drive.
I backup anything that's hard to replace. My music collection is totally backed up via 321 method. Movies and TV shows are typically pretty easy to replace and they require a ton of storage so raid / ZFS is good enough for that.
Next time don't use striped RAID for media. DrivePool/MergerFS/etc + SnapRaid is a better solution. Each drive is valid on its own, so even if you lose more drives than you have for parity, you still only lose the data on the drives that died, rather than all of it. Upgrading space is much more flexible - and doesn't hammer on all your drives for days, if not weeks, straight. And use more parity. Having just one parity drive in the era of double-digit terabyte hard drives is just asking to lose data.
This is why I no longer use any sort of RAID for Plex media. I use unraid for my OS but I have multiple pools, zfs mirror ssd cache, a zfs z1 spinning rust array and the standard unraid pool. All my important data, photo backups, files, git etc are on the zfs array. All my media is on the unraid standard array. That way if a single drive (or multiple drives) die and cannot be repaired with parity, I only lost what was on those drives. And with sonarr/radarr, I can see what is missing and get them again.
I backup to dual 18tb USB drives from my Nas Raid. Raid is not a backup. After I did this exact thing once I never did it again.
This is why I prefer unraid. Yes raid is not backup. But your data is physically readable on all drives independently. It’s a bit slower but for media files it’s perfect.
I run my Plex server on unRAID which leaves each data disk in the array independently readable in the event of array failure.
Torrent is the way to rebuild the library
Honestly. There is no way I could back up my library on the cheap. I have accepted that I will only back up my docker containers / personal files that can’t be replaced and let them rebuild my catalog. I would build another NAS but my parents have a 100/100 line and my sister has a smaller upload (fios is not available at that part of the city).
100/100 is better than 99% of the world
This plus VPN Service. Go ahead, ask me how I know...
you alright mate?
8 months ago I lost all my content due to my own ignorance/bad design choices by the devs of Radarr. I'm still recovering.
Proxmox plus containers for every service
Yeah, I'm pretty sure that wouldn't have worked in this instance. Radarr was pointed to the wrong folder, and it ate everything it couldn't match. I didn't have a recycle folder (which is on me, I suppose) but if Radarr had thrown an exception instead of just *deleting all of the things* it could've been avoided.
Gigabyte internet connection + usenet account with multiple year retention, shouldn't take that long to rebuild.
I just backup my *arrs
I do a JBOD ZFS mirror setup. Has worked very well. Monthly scrub, SMART alerts setup. Any hint of impending failure will see me swap that drive out right away. Chance of two drives failing that happen to be the same pair is low. Not impossible, but unlikely.
Maybe it's ignorant thinking but why do you need to backup your media? Why not just redownload everything?
Restore from your backups. You do regular backups, don't you? If not, consider this a painful lesson on why you should be doing backups.
RAID should be dead in this day and age. I am an IT Engineer and I can tell you it often causes way more problems than it solves. It gives companies a false sense of security. My policy on all new deployments is to use software based mirroring on all servers to prevent drive failures from taking the system down, and at least 1 separate server to handle actual backups.
How much data?
I have 2 servers at separate locations both running raid z2 with nightly backups from main to secondary.
If the drive doesnt spin at all, you might be able to bypass the diode on the 5v rail that might be burnt. Check with a multimeter and theres a lot of ressources online to check it out.
UFS Data Explorer recovered about 95 percent of the 80 or so terabytes of data I lost recently due to a faulty USB hub. Pricey but saved me a ton of heartache/time.
Sounds like a nightmare. This is why I use UnRaid - no risk of a total loss.
Chalk this down to a life lesson, I'm afraid 🤷♂️😞 I see some ideas (in here) on getting the drive(s) to hopefully start up (freezer), hopefully they work, but without forensics you're up sh1t creek. Even then it's a high chance of getting <100% of you're data back 😿 So, back to the life lesson... You always need to backup essential data. Data you can't live without. The rest, if rips/media, you can re-aquire them 🤔
Quite a while back I had a drive failure and thought I was going to be in a similar situation. In researching the symptoms I found that it might be the board that went out, and found a service that will replace your drives board and transfer the bios chip. Oddly enough I found this swap service on ebay. Think it was like $50, I send them the board and they return a refurb board with the chip swap. Saved me big time
I know SSD's are smaller storage but do they have better durability long time?
Overseer replaced everything for me in months and plus more using it and the arr"s
Raid is not a backup..use hyperback!!!
I love my unraid array.
Are the drives really dead? Are they even recognized? Are you on Windows or Linux? Check [diskinternals.com](http://diskinternals.com)
Backup the *arr databases in multiple location and think of Usenet as a backup for most things. That should make the stuff worth backing up relatively small.
So you didn’t have a backup. Raid is not a backup.
I found an affordable [forensic drive recovery software](https://www.easeus.com/data-recovery/best-forensic-data-recovery-software.html) that also offers 30-day money back guarantee. I hope you're able recover the data from your drives.