Compare directory structures and remove identical files?

Forums › Desktop computing › Compare directory structures and remove identical files?

richms

29098 posts

Uber Geek
+1 received by user: 10209

Trusted

Lifetime subscriber

#127265 4-Aug-2013 19:38

I have ended up with many copies of some old documents etc that have been well traveled between computers and HDDs etc. I know there is at least some files that have gone missing or become corrupted among the lot, but in large they are all identical.

Anyway, its taking massive amounts of space up, so is there something that will compare the directory trees, and any files that are in more than one, will compare the files, and if they are identical delete one of them? The idea being that I should end up with only the possibly corrupt ones exisiting in more than one place and I can look into which is the best copy out of them all.

Richard rich.ms

Ouranos

118 posts

Master Geek
+1 received by user: 41

#871745 4-Aug-2013 21:20

Assuming you're using Windows, have a look at http://winmerge.org/
I used it for a similar purpose recently, and it did the job.

Hammerer

2480 posts

Uber Geek
+1 received by user: 802

Lifetime subscriber

#871751 4-Aug-2013 21:31

Here's reviews for free duplicate file removers for Windows at a reputable site techsupportalert.com.

You're also likely to find a disk space analyser is useful.

1080p

1332 posts

Uber Geek
+1 received by user: 152
Inactive user

#871769 4-Aug-2013 22:26

TeraCopy should make this process painless. Obviously, nothing will be able to automatically check for corrupted fdata because the file system does not know they are corrupt already.

richms

29098 posts

Uber Geek
+1 received by user: 10209

Trusted

Lifetime subscriber

#871771 4-Aug-2013 22:28

Yeah, but if I am only left with the dupes in the origional file trees then its not hard to open them up and see which ones are broken. Im sure there are only a few, mainly they will be missing from some of the trees as files failed to copy off old laptop drives etc.

Richard rich.ms

surfisup1000

5288 posts

Uber Geek
+1 received by user: 2159

#871772 4-Aug-2013 22:37

Some useful tools :

To find corrupt files??? How do you determine this? Not sure you'll get much that will do this. Maybe if the file size or date is different you could determine that way? But, you need to check for patterns.

Anyhow, some good tools...

beyondcompare (pay)

Noclone (best duplicate finder I've used as it allows you to delete the dups in a certain directory branch, but, they make you pay)

Duplicate Cleaner (good duplicate finder, and free).

richms

29098 posts

Uber Geek
+1 received by user: 10209

Trusted

Lifetime subscriber

#871774 4-Aug-2013 23:19

Im going to let crashplan finish up on the files now they are all copied onto a single computer and then have a shot with the auslogics one linked to first, so that will be a couple of days backing up first before I let it loose on them.

Richard rich.ms

Lego

Shop now for Lego sets and other gifts (affiliate link).

PaulBags

809 posts

Ultimate Geek
+1 received by user: 184
Inactive user

#873494 7-Aug-2013 20:27

1080p: TeraCopy should make this process painless. Obviously, nothing will be able to automatically check for corrupted fdata because the file system does not know they are corrupt already.

When I tried to use teracopy to copy a large amount of files from one hard drive to another it crashed constantly, and when it did work it was slow as. I gave up and found fastcopy, which worked alot better. I know many people use it without problems, but to me teracopy seems like a great way to risk your files.