Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


55 posts

Master Geek
+1 received by user: 8


Topic # 126923 24-Jul-2013 01:02
Send private message

I am about to merge all my movie files on two hard drives onto one. The problem I know I will encounter is that some movie files have the same file names (everything has been renamed through Media Center Master to the conventional naming system) but different extensions so some folders will end up with two movie files with different extensions. What would be the easiest way to find out which folders will have two movie files without opening each one up and checking.

I want to delete the movie files of lesser quality and dont mind doing that manually but would like to automate finding the duplicates. All of the duplicate file finding programs I have tried dont just compare filenames minus the extentions, or they compare the content of the files and dont come up as duplicates.

Any advice?

Filter this topic showing only the reply marked as answer Create new topic
2703 posts

Uber Geek
+1 received by user: 95


  Reply # 864988 24-Jul-2013 12:26
Send private message

I started writing answer then realised it's not quite a complete solution because you have to remove the extensions before comparing the list and there is no easy way to do this without writing a script, so my answer is as follows

anyways

use the following cmds at cmd prompt (or in a .bat)
dir /b /s > c:\movies\*.mp4 >> list.txt
dir /b /s > c:\movies\*.avi >> list.txt
dir /b /s > c:\movies\*.mkv >> list.txt

move first half of collection to c:\movies\
run the commands
now rename c:\movies -> c:\movies1\
move second half of collection to c:\movies\
run the commands again


copy + paste list.txt into

http://www.textfixer.com/tools/alphabetize-text-words.php

It should be easy to scan for duplicates with your eye.

Explanation

/b = bare format (no filesize etc)
/s = include subdirectories
>> = add directory result into an existing text file




HTPC: Intel i3-2100 / 12GB RAM / AMD HD7750 / 480 GB SSD / 58TB Storage / MediaPortal / MadVR / Win10
AVR: Pioneer Elite SC-LX87 220W 9.2 Ch AVR
Speakers: Wharfedale Jade 7 Fronts / Jade 2c Center / Jade 5 Rears
Subs: iNuke 3000dsp 3000W proamp with 4x 15" JBL Sealed Subs
Display: Samsung 60" UA60H6400 LCD TV
Accessories: Gefen HDMI Detective with splitter


2695 posts

Uber Geek
+1 received by user: 529


  Reply # 865023 24-Jul-2013 13:34
Send private message

http://www.portablefreeware.com/?id=1334

*appears* to check based on filesize as well - so should find dups even with differing names...




Nothing is impossible for the man who doesn't have to do it himself - A. H. Weiler

2695 posts

Uber Geek
+1 received by user: 529


  Reply # 865029 24-Jul-2013 13:40
Send private message

Oh - I re-read your post - you want the names to be compared minus the extension then you want to work out which is the 'inferior'/smaller file and get rid of that.
Thats pretty specific - kiwijunglist is probably onto it - it will be tricky ....




Nothing is impossible for the man who doesn't have to do it himself - A. H. Weiler



55 posts

Master Geek
+1 received by user: 8


  Reply # 865041 24-Jul-2013 13:52
Send private message

Yeah pretty specific and annoying to do manually when there may be a hundred or so duplicate files. I'll give the batch file method a go tonight. Cheers.

3343 posts

Uber Geek
+1 received by user: 1089

Trusted
Vocus

  Reply # 865060 24-Jul-2013 14:18
Send private message

Tried Teracopy?  You can set it to fail on dupe, then you can deal with the exceptions after the bulk of copying has finished.



55 posts

Master Geek
+1 received by user: 8


  Reply # 865128 24-Jul-2013 15:09
Send private message

ubergeeknz: Tried Teracopy?  You can set it to fail on dupe, then you can deal with the exceptions after the bulk of copying has finished.


Would it ignore extensions and create a log of failed duplicates?

3343 posts

Uber Geek
+1 received by user: 1089

Trusted
Vocus

  Reply # 865157 24-Jul-2013 15:30
Send private message

OH - I overlooked the extension thing.  Nvm.  Script will be your only way forward then, really.

920 posts

Ultimate Geek
+1 received by user: 65

Trusted

  Reply # 865197 24-Jul-2013 16:29
Send private message

If you're already willing to accept you may need to delete the duplicates manually, why can't you just copy them all into the same folder do hit the name link to put them alphabetically (as you've said they're already been renamed the same so only difference is the file name), then look at your list you'll have two names the same next to eachother and then can decide which of the two you want to keep.

It may not be that simple and the answers here obvioulsly make me believe that it isn't but that's how I'd do it. If you wanted to run a script from there to pick which ones to delete surely one can be written to say keep the larger file size yadda yadda thing.

I am not super technical and have little experience with using scripts (tend to ask or research and copy what I find) but sometimes I think people especially us geeks (or in my case wanna be geek) we like to over complicate things. If this won't work sorry for wasting your time reading it lol. I will be checking back to see how you did it though so I know for myself as I know I have the same problem over maybe 2 - 4 hard drives worth of video content where a large chunk are exact duplicates as well as different file format types.

Good luck either way.


1332 posts

Uber Geek
+1 received by user: 152
Inactive user


  Reply # 865201 24-Jul-2013 16:36
Send private message

The above is the simplest way to do it.

Copy them all to one HDD/directory, arrange alphabetically and pick which of the duplicates you wish to keep.



55 posts

Master Geek
+1 received by user: 8


  Reply # 865318 24-Jul-2013 19:18
Send private message

jtbthatsme: If you're already willing to accept you may need to delete the duplicates manually, why can't you just copy them all into the same folder do hit the name link to put them alphabetically (as you've said they're already been renamed the same so only difference is the file name), then look at your list you'll have two names the same next to eachother and then can decide which of the two you want to keep.

It may not be that simple and the answers here obvioulsly make me believe that it isn't but that's how I'd do it. If you wanted to run a script from there to pick which ones to delete surely one can be written to say keep the larger file size yadda yadda thing.

I am not super technical and have little experience with using scripts (tend to ask or research and copy what I find) but sometimes I think people especially us geeks (or in my case wanna be geek) we like to over complicate things. If this won't work sorry for wasting your time reading it lol. I will be checking back to see how you did it though so I know for myself as I know I have the same problem over maybe 2 - 4 hard drives worth of video content where a large chunk are exact duplicates as well as different file format types.

Good luck either way.



I would be happy to do that, but all of the movie files are in separate organised folders with artwork, metadata etc. I would have to rip all the files out these folders and then get them back into the right place. I cant think of an easy way of doing that step either.

I will give the script a crack first, cant bugger up much that way.

920 posts

Ultimate Geek
+1 received by user: 65

Trusted

  Reply # 865349 24-Jul-2013 19:58
Send private message

Hmm well ok that does change it a little but still can be done similarly I think...

Do what was suggested above search your overall video folders for *.mkv, *.avi, *.whatever other file formats you have cut the name info from your search results into a Excell spreadsheet and then do the search. All files are in same location still and obviously using more detailed search parameters you could sort them from there (sounds good in theory but may just sound good and not actually work lol).

Good luck and remember to post back how you got it to work.



55 posts

Master Geek
+1 received by user: 8


  Reply # 865379 24-Jul-2013 21:25
Send private message

OK I'm done.

I ran the cmds but in the 2 different drives. I couldn't be bothered moving everything into and out of 1 drive.

So I ended up with a list in the .txt file with d:\movies at the start of half the lines and j:\movies i the other half. I then ran it through Notepad++ and used the column editor to remove d:\movies and j:\movies from the start of each line. I was left with just the pure filenames.

Then I ran it through the txt fixer and removed the duplicates and alphabetised the filenames. Viola! A bit of scrolling through later and I had picked out the files that I would have to manually delete after the merge.

Turned out there wasn't as many duplicates as I thought, I was only going to check the .mp4 and .avi files against .mkv for quality anyway and overwrite all .avi on the j:\ drive with those on the d:\. At least I didn't have to check every single folder.

Cheers for the help kiwijunglist. There may be a cleaner way of doing this but I am satisfied with this result.

2703 posts

Uber Geek
+1 received by user: 95


  Reply # 865384 24-Jul-2013 21:41
Send private message

Good idea,




HTPC: Intel i3-2100 / 12GB RAM / AMD HD7750 / 480 GB SSD / 58TB Storage / MediaPortal / MadVR / Win10
AVR: Pioneer Elite SC-LX87 220W 9.2 Ch AVR
Speakers: Wharfedale Jade 7 Fronts / Jade 2c Center / Jade 5 Rears
Subs: iNuke 3000dsp 3000W proamp with 4x 15" JBL Sealed Subs
Display: Samsung 60" UA60H6400 LCD TV
Accessories: Gefen HDMI Detective with splitter


3224 posts

Uber Geek
+1 received by user: 624

Trusted

  Reply # 866877 26-Jul-2013 22:45
Send private message

Personally I would go to www.ninite.com, download and install "Everything"

Everything is a quick search program that is way easier to use than windows search.

Dump all your movies onto the secondary hard drive, set Everything to search only that hard drive.

Search for the term *

Sort by Name

Go through the list and delete any duplicates you see.




Ray Taylor
Taylor Broadband (rural hawkes bay)
www.ruralkiwi.com

There is no place like localhost
For my general guide to extending your wireless network Click Here




Filter this topic showing only the reply marked as answer Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.