If you keep up with the news, work in IT or even on a computer, it does not take much imagination to think of a scenario in which one might loose access (temporarily or permanently) to your records and files. These days, stories about targeted events like hackers, home invaders or arsonists seem almost as common as mass events such as malware/ransomware, the heightened solar flare activity reported by NASA and the European Space Agency or the threat of nuclear war, including the use of nondestructive EMP devices.
With this in mind and while it may seem counter to many of the reader’s risk models, I’ve recently conducted a fair amount of research into technologies for long-term data archival of non-sensitive information. Files such as personal documents which may be publicly available, copies of some legal documents which my family may need to access after my passing and (of equal importance) family photo’s and video footage, are all examples of this. They’re not sensitive, not under strict non-disclosures and aren’t likely to be used as evidence in a court case if seized, but they are of immense personal value and as such, should not be trusted to third-party service providers such as cloud services; or written to memory which can be modified or erased magnetically.
In this post, I will share a brief overview of some of the advantages of optical storage formats over other forms of long-term data storage, share my personal use case and setup as well as physical security considerations for setting up your own archive. I will also directly address natural and man made events, from solar flares to EMP and mass cyber events….or “Cyber Pandemic” as the WEF calls them.
Use Case
Like many people, I have Terabytes of photographs, video footage, documents, books, manuals, movies – you name it, all stored on external hard drives, flash drives, my home-built Network Attached Storage (NAS) and some in encrypted cloud accounts. However, unlike many people that I know, I have a calendar event set up in my phone to transfer these files once every 30 – 45 days, from their storage medium to Blu-ray disk for long-term storage.
I do this in 50 GB increments due to the time duration and nature of this size disk, though I’m planning to consolidate to 100 GB or 125 GB disks in the future if we start to run out of space in the binder for disks. But why do this – why not just keep my data on clouds and hardware storage devices like a ‘normal’ person.
Well, as you run a basic risk analysis comparing the value (monetary, personal and sentimental) against the chance of loss, averaged across just the events listed in the introduction, you may find that you have a greater than 1% chance of loosing everything. This was far to great for me, especially given that the total cost of the equipment to archive mine and my family’s data in the most durable, compact, light weight format available, was far less than 1% of the of the value of the data stored. For me, this includes everything from family photos, video footage of my kids, deeds and legal documents to family journals and records dating back to the mid-1600’s – a tradition in my family since.
The Blu-ray Advantage
If any of the scenarios discussed thus far have crossed your mind in the recent past (or any time) or your use case is similar to mine, then Blu-ray may be a good candidate for archival data storage, as it has several distinct advantages over other memory formats. Some of these include HDD, SSD, USB Flash memory and even DVD with this application:
- Stable optical storage format, with a 30 – 200 year lifespan.
- Highly availability of readers and system inter-operability.
- Highly resistant to changes in humidity or temperature.
- Optical data storage means zero impact by natural (solar) or man made EMF/EMP events.
- Blu-ray can store up 25 GB to 125 GB compared to DVD’s 4.7 GB to 17 GB.
- Can be scratched, but not corrupted, modified or ransomed.
- Standalone – does not require internet for the owner’s access.
- Low cost, at $17 per 500 GB of storage in 50 GB dual layer increments.
- Sole physical access to your own data – unlike cloud storage providers.
While all of the above made Blu-ray a clear winner in my book, I wanted to touch briefly on the last point: In my mind, and not just because I’m a nerd and read the user agreements, storing your personal data on some corporation’s computer is a major issue – not only for yours and your family’s privacy but also within the context of archival. When you push your data someone else’s computer and something happens to It through any type of event, you are now stuck calling customer support lines and submitting help desk tickets with the hope that who ever receives them shares your concerns for your data.
This recently happened to a friend of the family: She was trying to access a folder of older (about 5 years) documents through her cloud provider’s interface, and she kept receiving a message that the data was unavailable and to try back later. After a few days, she called customer support and was told that there was nothing that could be done, and they sent a gift card in the mail.
Re-writing/Editing & Storage Term
As with CDs and DVDs, you will need to purchase a disk with special re-writable blanks to reuse or re-write over a burned disk. As far as I can find, only single layer (25 GB) Blu-rays are available with this feature, but the 50 GB dual layer, 100 GB – 125 GB BDXL and four layer BD are not, which is likely because it is too complex to have more than one layer that can be rewritten over the top of each other.
Re-writable discs have the shortest lifespan as well, expected to last just 5–10 years while write-once discs can easily last 30–100 years and factory published discs are good for 200+ years if cared for properly. Keep this in mind and be sure to mark your disks or inventory with timestamps. All archives have some form of maintenance, but yours will be just once every 30 years or more.
Similar Formats
In 2015, a joint venture between Sony and Panasonic released a new format with the specific goal of archival storage. These utilize the same encoding as Blu-ray for data but with a different disc metadata format, and can store 300 GB to 1 TB of data per disk. At the time of this writing (October 2022), the hardware and disks needed for these were less common, thus making them a less economical personal (home) data storage solution, but I’m sure that many of us will be switching to the archival disks in the near future. The white paper to the Sony/Panasonic Archival Disk can be found here.
Something to watch out for are Blu-ray disks titled as Archival Disks on sites like Amazon, Best Buy, New Egg or where ever your disks are sold. I do not believe that this is nefarious, but it is obviously pure marketing. If you see a disk titled “Archival Disk” remember that the true Archival Disk format stores 300 GB on the low end, whereas most of the disks that I’ve seen carrying this title are just 25 GB single layer Blu-rays. This is too small for mine and many other archival applications that I’ve encountered.
Resistance to Natural & Unnatural EMP Events
Electromagnetic pulse (EMP) is a rapid discharge of electromagnetic energy from a natural or unnatural (man-made) source. As such, it follows that EMP can disrupt or permanently damage electronic equipment, posing a very real risk to your computers, flash drives external drives and even entire data centers. According to the UpTime Institute: “Many data center operators have overlooked EMP in their risk assessments and have not taken protective measures against EMP.”
To put ourselves in the corporate data center shoes, this makes good business sense because shielding and other design considerations are quite expensive for an event which is not all that likely to happen. Even if the event were likely, is would still be more cost effective (on a quarterly or any other basis) to add just one line in the user agreements, under something like a ‘Liability & Limitations’ section.
But, what about the drives and detachable memory that you store locally in your home? I’ve read a lot of internet experts claim that small devices which are disconnected from the power grid during an event will likely survive. I’m not going to pretend to be an EMP expert and enter a debate about this, but I will say that without any way test or verify this independently and with few others having access to the resources to do so; the now ‘known-unknown’ risk is simply too high for my model. It might not be for yours or maybe you know more about EMP and trust your Faraday.
Regardless, due to their optical format, most Blu-ray disks are immune to an EMP event. In his post on EMP Protection, Jerry Emanuelson brings up another important point that I had not previously considered:
It is important to have all of the computer data that is important to you backed up onto optical media, like CD or DVD. Paper printouts are fine, but after an EMP attack, most of the data on paper printouts will simply never get typed back into computers, so those paper printouts will just become your personal mementos.
Jerry Emanuelson, B.S.E.E.
My Setup
With my workflow, I utilize external drives such as USB Flash and SSD for my day-to-day file transfers and references, then transfer these files at a per-determined increment which is marked with a calendar event in my phone. This calendar app does not export data to the internet or sync with any online service. Also, when say ‘transfer’, what I really mean is “copy” because I may need to access that pdf document for the first time in 3 years, or would like for it to be referenced during a grep query, so many of these will stay on their drive in some form.
Now, keeping in mind that I was planning to write to the double layer 50 GB disk (and the 100 GB – 125 GB BDXL’s in the in future) from files stored on other drives, this put most of the sub-$100 burners were off of the table for me. LG makes a writer for $90 but it has a 8x write speed with USB 2.0, which equates to a lot more time spent during our archival activity type.
Additionally, I was planning to perform most of my archiving from my home office, not on the road or from the middle seat on a flight. It’s on the medium/higher end in terms of cost (at $120 at the time of this writing) and is even physically larger than a lot of the other writers on the market, but I went with the ASUS Powerful Blu-Ray for the following reasons:
- USB 3.0 or USB 3.1 Gen1 – where most on the market at the time of my purchase were USB 2.0.
- Supports writing to ALL Blu-ray disks sizes currently on the market, from single layer 25 GB up to 125 GB BDXL.
- Optical storage write speed of 16x compared to the mentioned LG’s 8x.
- Universal compatibility: Linux/Mac/Windows 10 & 11
If these differences are not important to you or you are on a budget, the LG Electronics Ultra-Slim 6x Writer (USB 2.0) can write to dual layer (limit of 50 GB) and is about 35% cheaper. Put that savings into your training fund – I totally get it. The Amazon Choice LG product I saw had a 4.5 star review with 765 ratings, but I did not dive into it because it did not support the speed or expandability I was looking for. In other words: It’s likely a great mid-range product today, but you will forced to buy an upgrade pretty soon.
Disk Storage
Personally, I bought a CD/DVD soft binder and keep this in one of my fire/water proof gun safes for easy access. If you do not have a lot of safe storage and this is important to you, Sentry makes some small fire/water proof safes that are designed to store 8.5 x 11” file folders. This safe will normally run about $40 at Walmart and are a great option, though you should remember that they are mostly protection from fire and water, not as much theft. Aside from the fact that you can run with it on your shoulder (which may also double to some as an advantage) I have picked into these easily when my lock picks were closer than another trip up the stairs and down the hall for the keys.
Which ever solution or combination of solutions you decide on, remember that the biggest threats to a Blu-ray archive are scratching, contact with excessive heat, prolonged submersion in water. Depending on its composition, dust may also have long term effects if allowed to settle or is wiped off without special care.
Conclusion
Not all archives are created equal and that our data, records and recorded memories are within a threat category which exceeds that of many of our other possessions. For data archival, being information that you are not indexing very high frequency, I personally use and recommend the Blu-ray format.
As always, I hope that you took something away from this post and that you will feel free to reach out through the Contact form here or on my twitter if you have any questions or want to bounce a specific use-case off the wall.