Storage Migration Planning: What Slows It and What Loses Data
Moving data from one storage system to another is where estates lose data, and almost always for avoidable reasons. This covers planning a migration that finishes without incident.
Before anything moves: three facts you need
How much data, and how it is structured. A single large volume and millions of small files migrate at completely different speeds even at the same total size. Small-file migrations are dominated by file count, not capacity.
What can be offline, and for how long. This determines whether you need a live migration or can take an outage, and it is a business decision rather than a technical one.
What the data actually is. Migrations are the best opportunity to not move things that should have been deleted or archived years ago. Our storage space guide covers finding what is really there.
Back up first, and verify the backup
Not negotiable, and the verification is the part people skip.
A migration touches every byte you own. If something goes wrong mid-transfer, the backup is what you have. A backup you have not tested restoring from is an assumption, not a backup. Our backup guide covers the principle.
And keep the source system intact until the migration is verified complete. The temptation to reuse those drives immediately is strong and has cost people everything.
What limits migration speed
Four bottlenecks, and identifying which one you have saves days.
Network bandwidth, if the transfer crosses the network. 1GbE moves far less per hour than people estimate for large volumes. Our 10GbE guide covers upgrading it.
Source read speed, particularly on older mechanical arrays.
Destination write speed, and on SMR drives this collapses part-way through, which is one of several reasons SMR does not belong here. Our SMR guide covers why.
File count, where per-file overhead dominates. Millions of small files can take longer than a much larger volume of big ones.
Measure a sample transfer before committing to a window. A migration estimated at eight hours that takes three days is how outages become incidents.
Choosing the destination hardware
Five things to get right before the data arrives.
Capacity with headroom. Migrating into a destination that is nearly full leaves no room for growth and degrades performance. Our capacity planning guide covers sizing.
The right drive class. CMR for any array, and enterprise or NAS class rather than desktop. Our drive class guide covers this.
RAID level appropriate to the capacity. At large drive sizes, double parity and a hot spare. Our RAID guide covers the trade.
Controller support for the drive capacity and interface. Our controller guide covers checking.
Build and test the destination before migrating, including a sustained load test. Finding a faulty drive during the migration is much worse than finding it the week before. Our acceptance testing guide covers this.
Verify before you decommission
Three checks.
File and byte counts match between source and destination.
Permissions and metadata survived. This is the most commonly damaged thing in a migration, and it frequently is not noticed until someone cannot access a file weeks later.
Applications work against the new location. Test the actual workloads, not just that the files are present.
Only then decommission the source, and do it properly — our sanitisation guide covers wiping drives that held your data, and our selling guide covers recovering value from them.
The five mistakes that cause migration incidents
No verified backup. Decommissioning the source too early. Underestimating the time, so the window is missed and the migration runs into production hours. Not testing the destination hardware first. Not checking permissions afterwards.
Every one is avoidable, and four of the five cost nothing but time.
Frequently asked questions
How long does a storage migration take?
It depends on the bottleneck: network bandwidth, source read speed, destination write speed, or file count. Millions of small files can take longer than a much larger volume of big ones. Measure a sample transfer before committing to a window.
Do I need a backup before migrating?
Yes, and you need to verify you can restore from it. A migration touches every byte you own, and a backup you have not tested is an assumption rather than a backup.
When can I decommission the old storage?
After verifying file and byte counts match, permissions and metadata survived, and applications work against the new location. Reusing the source drives too early is how migrations become data loss.
What is most often damaged in a migration?
Permissions and metadata, and it frequently is not noticed until someone cannot access a file weeks later. Check them explicitly rather than assuming the files being present means the migration worked.
Should I test the new storage before migrating to it?
Yes, including a sustained load test. Finding a faulty drive during the migration is far worse than finding it the week before, when you can still replace it calmly.
Can I use SMR drives as a migration destination?
No. Write speed collapses part-way through a sustained transfer once the drive’s buffer fills, and SMR does not belong in an array regardless.
Tell us the data size, file count and the window you have, and we will size destination hardware that will not become the bottleneck.
