View data integrity, including recent posts Data loss prevented: IntegrityChecker saves my bacon by detecting corrupted files after a clone bit.
IntegrityChecker java runs on all platforms with a JVM. I am looking for Windows, Linux and NAS users to test IntegrityChecker java. Contact Lloyd.
A great update to diglloydTools IntegrityChecker is now available. Important things are listed below.
This finally candidate (“fc” 7 is ready for production, or so I think. Click through the license and download the page to download:
Download IntegrityChecker Java 2.0 fc1
Documentation is mostly in place, but may fall a little short in a few areas.
Buy diglloydTools (includes IntegrityChecker Java).
New feature: folder hash hierarchies
Folder hash registers record all hash values for an entire folder hierarchy or volume in a single “.icjh”
Hierarchical files can also be used in combination with traditional “-icj” files with each folder. This can be useful when reorganizing many folders, because the “.icj” file travels with each folder. However, for most backups and data verification purposes, the “.icj” files are not redundant.
For folders like my Apple Mail folder with the 113784 folder and growing, there is one (1) “.icjh” file instead of 113784 “.icj” files that eliminates the need to read / write 113784 “.icjh” files – which is also a bonus for minimizing backup activity. And there is absolutely no value in having the files in each folder, since the mail folder is never reorganized and always backed up in its entirety.
Hash data for 113784 folders containing 387296 files loaded in 12109 ms.
Hierarchical files also mean that you can use icj on “.git” hierarchies without git complaining about unwanted files in the “objects” and “package” folders. Do for any program that does not like unknown files in the folder hierarchy, library, etc.
A folder hash hierarchy also makes it possible to hash read-only media and store this hash file elsewhere. This can be implemented if you see interest in it.
New feature: on-site update, but with full compatibility
IntegrityChecker java maintains full compatibility with previous hash formats, including the original version “.ic” format (SHA1). Backward ‘verify’ compatibility is crucial, as many professionals burn their work for read-only media – read-only media cannot be updated or changed.
A revised hash format (linked SHA512) is introduced with the update command that automatically verifies existing hash and calculates the new and faster SHA512 hash. In this way, updating is seamless and results in the best possible speed when updated.
The performance is fantastic – shown below is the icj on a 2.5 GHz 28-core Mac Pro that does 12.5 GB / sec hashing (12500 MB / sec). It is about 446 MB / sec per real core running at a relatively slow 2.5 Ghz each.
Currently not implemented, but planned soon: file climbing, which will allow precise control over which files are hashed and which are not, folders to be included or excluded, etc.
56 virtual CPU cores in use (28 real cores) – diglloydTools IntegrityChecker makes SHA512 hash of 12.5 GB / sec