I experienced my first silent data corruption event that I witnessed and caught:
While copying a 900MB video file from one drive to another with Windows Explorer, one byte of the copied file changed value and was written to disk with a correct checksum. Neither Windows Explorer, the Adaptec RAID driver, or Windows Event log reported any error.
I only discovered it by chance because I had created a .md5 file of the source file, and checked that .md5 with the copied file. THEY DID NOT MATCH! DOS FC good bad /b confirmed a single bit in one byte had flipped:
10100111
vs.
00100111
I'm trying to fathom how it happened and was not reported. I could find no evidence that Adaptec is checksumming data it puts in its cache, so maybe the error happened in the RAID card's 512MB cache?
Windows Explorer has a fundamental design flaw in is that it is not verifying the copy against the original, and it is not doing a checksum of its own data buffer, which is where the error may have occurred. The box is very stable- no crashes ever. Maybe a cosmic ray flipped the bit while it was in the buffer?
I replaced the RAM in the box. I could rebuild the box with some new hardware and ECC RAM (if I had some spare time). I'm studying ZFS and BTRFS and may use one on a future file server.
Now I'm .md5ing my audio/video files. I wonder how many have been corrupted over the years, and I have not noticed due the forgiving nature of the data type.
Main question-
Does there exist an alternative to Windows Explorer that verifies the new file against the source file during moves and copies?
I've searched the web for several hours and found no alternates that verify and no way to easily enable the feature in Windows Explorer.
I could write my own Windows Explorer from scratch. I do know what's involved... the right click context menu is non-trivial, but the copy/move with verify (and checksummed buffer) would be relatively easy. Yes it will cut the speed in half. So what? Data integrity is far more important than speed. Amazing Microsoft doesn't know this.
BTW, the offending box is Windows 7 Pro 64bit, but my web searches told me no version of Windows Explorer on Win9x, XP, Win7, Win10, verifies a copy/move operation.
While copying a 900MB video file from one drive to another with Windows Explorer, one byte of the copied file changed value and was written to disk with a correct checksum. Neither Windows Explorer, the Adaptec RAID driver, or Windows Event log reported any error.
I only discovered it by chance because I had created a .md5 file of the source file, and checked that .md5 with the copied file. THEY DID NOT MATCH! DOS FC good bad /b confirmed a single bit in one byte had flipped:
10100111
vs.
00100111
I'm trying to fathom how it happened and was not reported. I could find no evidence that Adaptec is checksumming data it puts in its cache, so maybe the error happened in the RAID card's 512MB cache?
Windows Explorer has a fundamental design flaw in is that it is not verifying the copy against the original, and it is not doing a checksum of its own data buffer, which is where the error may have occurred. The box is very stable- no crashes ever. Maybe a cosmic ray flipped the bit while it was in the buffer?
I replaced the RAM in the box. I could rebuild the box with some new hardware and ECC RAM (if I had some spare time). I'm studying ZFS and BTRFS and may use one on a future file server.
Now I'm .md5ing my audio/video files. I wonder how many have been corrupted over the years, and I have not noticed due the forgiving nature of the data type.
Main question-
Does there exist an alternative to Windows Explorer that verifies the new file against the source file during moves and copies?
I've searched the web for several hours and found no alternates that verify and no way to easily enable the feature in Windows Explorer.
I could write my own Windows Explorer from scratch. I do know what's involved... the right click context menu is non-trivial, but the copy/move with verify (and checksummed buffer) would be relatively easy. Yes it will cut the speed in half. So what? Data integrity is far more important than speed. Amazing Microsoft doesn't know this.
BTW, the offending box is Windows 7 Pro 64bit, but my web searches told me no version of Windows Explorer on Win9x, XP, Win7, Win10, verifies a copy/move operation.
Last edited: