advice on custom .bat file for copying 3 years of my life

Forums › Microsoft Windows › advice on custom .bat file for copying 3 years of my life

nathanbarlow

48 posts

Geek

#195569 25-Apr-2016 02:09

Hi All,

I'm on Windows 7, and since me trusty HP TM2 is now getting a wee bit old, I'm looking at making sure all my files and important shizzle is backed up. Just in case. *touch wood*

Now, before I start with my question and plea for help, two very important considerations: 1) I am a postgraduate student and haz zero budget for specialised software to make my life happy and easy and 2) the aforementioned postgraduate student thing? Yeah my files from my first 12 months are everywhere. Thankfully my second year I had the brainwave of using a seperate USB drive and the cloud to enforce some order on my crazy :P

So what I'm looking to do is:

Copy files from

6x Librarys folders
Downloads... this has shamefully reached double digits in GB size
My Documents
Desktop
Google Drive folder (local)
Dropbox folder (local)
Wherever the heck Lightroom keeps its digital negative files (i still have never found this folder, and save a copy of photos to a library called "Photography")
Settings folder from various software, eg custom brushes from Photoshop
ANY .exe files for installations I have done (lots of specialised software used for my work)
ANY .pdf .doc .docx .xls .xlsx, etc files that might be hiding somewhere foolish, such as my C: folder
checking my D drive (HP's auto recovery partition) for any files that might be floating around there
Ripping everything from two USB drives which I'll have plugged into my TM2
probably gonna be a good idea to copy dat Recycle Bin and Temp folder too - just in case.

My best thought is a customised BAT file and an external harddrive/USB big enough. My crazy dream would be to have some sort of sorting rules written in my BAT cmds, so that I can dump files into some premade folders on the HDD/USB, rather than having to manually spend a few days of my life dragging and dropping. I can probably filter by file type combined with dates of creation.

My questions to GeekZoners are.... would this filter part work? How do I write this (i have experience in copying with DOS and the commands, so I know the copying would be possible) and more importantly, how do I not cock the filtering up. Finally, do I need to be anal about capital letters and underscores from my file naming system?

Ideally, I would like the copied files to retain their property data such as date created, date last edited/accessed, for photos EXIF data etc, so that the files remain searchable not only on the external HDD/USB but also my future computer if/when my TM2 gives up the ghost.

Hoping someone experienced might be able to help me out with some kind advice, as my programming and IT skills have taken a back bench for 3 years while doing postgrad (in neurosci)

Thanks
Nathan

1 | 2

2476 posts

Uber Geek

Lifetime subscriber

#1541411 25-Apr-2016 04:12

Use file synchronisation software which will do the filtering and copying better than batch files.

surfisup1000

5288 posts

Uber Geek

#1541442 25-Apr-2016 08:20

Robocopy is an excellent CMD tool built into most modern versions of windows.

The great thing about robocopy is the tremendous resilience and tons of options. Probably you can retain attributes and date stamps too, check out the documentation.

jnimmo

1097 posts

Uber Geek

#1541452 25-Apr-2016 08:41

For backup, I think you'll be much happier with a piece of software to assist

Long time ago used SyncToy but don't know if it exists anymore

Check out Veeam Backup https://www.veeam.com/endpoint-backup-free.html

Or, if you want the ability to add Cloud backup at some stage in the future, check out CrashPlan - it is free for using to backup to local disk, or to a friend's computer remotely.

timmmay

20579 posts

Uber Geek

Trusted

Lifetime subscriber

#1541476 25-Apr-2016 09:24

Take an image of the entire disk using Macrium Reflect Free, that way if you remember something later you can extract it. Before you do this, if it's an OS disk, uninstall all the programs that are easy to resinstall (Office, Photoshop, web browser), delete caches and temp files, run disk cleanup, etc. Make sure your image as as small as it can be without removing anything important.

Though Veem is also very reputable, and may be a good option. They use it at work, but there seem to be constant problems with backups - I don't think that's Veem's fault though.

Hiamie

354 posts

Ultimate Geek

#1541480 25-Apr-2016 09:51

Hammerer:Use file synchronisation software which will do the filtering and copying better than batch files.

I use FreeFileSync. (see link above)

Does the job, keeping folders in sync.

I recommend you have a peek.

And keep any backups off site, or they too could be stolen in a burglary etc!!

mdf

3513 posts

Uber Geek

Trusted

#1541524 25-Apr-2016 10:43

I still use MS SyncToy. Old, but it works really well for me.

In terms of the original query, I suspect you will spend more time creating an incredibly elaborate .bat file than you would just sorting/searching things manually then backing up/syncing.

I'm not sure what kind of specialist software you might use, but I've found it's really not worth backing up any of the obvious/mainstream installation files (browsers, Office, VLC etc.) since they get dated pretty quickly. You're better off just downloading the current version.

If you're really set on scripting, you might want to start with PowerShell rather than the command line/batch files. IIRC, it has a more powerful set of if/then logic. But it's a *looong* time since I've done anything like that and I may well be completely off piste.

richms

28172 posts

Uber Geek

Trusted

Lifetime subscriber

#1541547 25-Apr-2016 11:18

Crashplan is the easiest. Set it up once and then it will keep backing up to a friend/external drive or if you pay crashplan then to them.

The software is free, to backup to their cloud costs but it is not much. If you are not prepared to spend on something that cheap than it just shows how little you value your time you have put into creating the content.

Richard rich.ms

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Note that to use Quic Broadband you must be comfortable with configuring your own router.

linw

2849 posts

Uber Geek

#1541548 25-Apr-2016 11:20

Follow timmay's advice. No need to know where your scattered stuff is. Turn a nightmare into an easy and reliable solution.

The free version even does disk management and differentials.

toyonut

1508 posts

Uber Geek

#1541569 25-Apr-2016 11:56

DiskToVHD can also make a VHD file of your HDD and that VHD can be mounted by any version of windows natively from windows 8 upwards.

https://technet.microsoft.com/en-us/sysinternals/ee656415.aspx

Second the powershell script above over batch, but even that is overkill for a one off job like this. It will take longer to script something up than it will to just copy files manually. Although if you want to do it as a learning experience, go for it.

Try Vultr using this link and get us both some credit:

http://www.vultr.com/?ref=7033587-3B

nunz

1421 posts

Uber Geek
Inactive user

#1541622 25-Apr-2016 12:41

nathanbarlow:

Hi All,

I'm on Windows 7, and since me trusty HP TM2 is now getting a wee bit old, I'm looking at making sure all my files and important shizzle is backed up. Just in case. *touch wood*

Now, before I start with my question and plea for help, two very important considerations: 1) I am a postgraduate student and haz zero budget for specialised software to make my life happy and easy and 2) the aforementioned postgraduate student thing? Yeah my files from my first 12 months are everywhere. Thankfully my second year I had the brainwave of using a seperate USB drive and the cloud to enforce some order on my crazy :P

So what I'm looking to do is:

Copy files from

6x Librarys folders
Downloads... this has shamefully reached double digits in GB size
My Documents
Desktop
Google Drive folder (local)
Dropbox folder (local)
Wherever the heck Lightroom keeps its digital negative files (i still have never found this folder, and save a copy of photos to a library called "Photography")
Settings folder from various software, eg custom brushes from Photoshop
ANY .exe files for installations I have done (lots of specialised software used for my work)
ANY .pdf .doc .docx .xls .xlsx, etc files that might be hiding somewhere foolish, such as my C: folder
checking my D drive (HP's auto recovery partition) for any files that might be floating around there
Ripping everything from two USB drives which I'll have plugged into my TM2
probably gonna be a good idea to copy dat Recycle Bin and Temp folder too - just in case.
My best thought is a customised BAT file and an external harddrive/USB big enough. My crazy dream would be to have some sort of sorting rules written in my BAT cmds, so that I can dump files into some premade folders on the HDD/USB, rather than having to manually spend a few days of my life dragging and dropping. I can probably filter by file type combined with dates of creation.

My questions to GeekZoners are.... would this filter part work? How do I write this (i have experience in copying with DOS and the commands, so I know the copying would be possible) and more importantly, how do I not cock the filtering up. Finally, do I need to be anal about capital letters and underscores from my file naming system?

Ideally, I would like the copied files to retain their property data such as date created, date last edited/accessed, for photos EXIF data etc, so that the files remain searchable not only on the external HDD/USB but also my future computer if/when my TM2 gives up the ghost.

Hoping someone experienced might be able to help me out with some kind advice, as my programming and IT skills have taken a back bench for 3 years while doing postgrad (in neurosci)

Thanks
Nathan

Unstoppable Copier. By RoadKill - for three reasons.

1 - it handles bad files / disk issues.

2 - It can be stopped and restarted (if you use the only copy newer files options)

3 - If you use the batch functionality it creates a file called <whatevermy backup name I saved is called>.ucb open that in Notepad or equivalent. Inside you will see source and destination looking like:

c:\Users\MyName\documents|z:\Backups\MyName\Documents

P:\OnlineFileshare\ImportantGuff | z:\Backups\MyName\ImportantGuff

c:\Users\MycloudFolder\|z:\Backups\MyName\Mycloud

I usually edit that by hand putting in source and dest. Unstoppable Copier handles the creation of dest folders well.

It's robust, portable (ie one exe you can drop anywhere), small, and solid enough to be used in industrial situations via a batch file or soimilar.

The only copy new files options makes it quick.

It doesn't do delta copy but Syncrafy and Synaman do.

toyonut

1508 posts

Uber Geek

#1541857 25-Apr-2016 21:31

I'm sure this could be vastly improved but here is a quick knock-up in powershell:

#variables
$driveletters = "C:","D:","E:" #drives to search
$outputlocation = "F:" #drive we want everything to end up on
$musicexts = @("*.mp3","*.wma","*.m4a") #Extensions for music files
$docexts = @("*.pdf","*.doc","*.xls") #Extensions for document files
$ignoredexts = @("*.inf","*.exe","*.dll","*.ini") #Extensions to ignore as we don't care about them

#Copy all userprofile folders for the logged in user
$homedirs = Get-ChildItem $env:USERPROFILE
foreach ($homedir in $homedirs)
    {
        Copy-Item -Path $homedir -Destination "$outputlocation\Home" -force -Recurse -Verbose
    }

#recurse through drives and copy files over by specified filetype
foreach ($drive in $driveletters)
    {
        #Recurse through our directories from the root and get all files without an ignored extension
        $allitems = Get-ChildItem -Path "$drive\" -Recurse -File -Exclude $ignoredexts

        #Check for our output folder and create if it doesn't exist (Music)
        if (!(Test-Path -Path "$outputlocation\Music"))
            {
                New-Item -Path "$outputlocation\Music" -ItemType Directory
            }
        #Go through each item in our recursed list of files and copy any files with the correct extension into output directory
        foreach ($item in $allitems)
            {
            Copy-Item -Path $item.FullName -Destination "$outputlocation\Music" -Include $musicexts -Force
            }

        #Check for our output folder and create if it doesn't exist (Docs)
        if (!(Test-Path -Path "$outputlocation\Docs"))
            {
                New-Item -Path "$outputlocation\Docs" -ItemType Directory
            }
        #Go through each item in our recursed list of files and copy any files with the correct extension into output directory
        foreach ($item in $allitems)
            {
            Copy-Item -Path $item.FullName -Destination "$outputlocation\Docs" -Include $docexts -Force
            }

        #Repeat ad nauesum for your filetyes you want to sort. We could function this out as it is just repeating the same thing over and over, but meh
    }

It makes slightly more sense in the powershell ISE as the color coding makes it pretty, but hopefully it is pretty clear what is going on.

Try Vultr using this link and get us both some credit:

http://www.vultr.com/?ref=7033587-3B

mdf

3513 posts

Uber Geek

Trusted

#1541867 25-Apr-2016 22:07

Wow @toyonut, that is pretty gosh damn impressive!

jnimmo

1097 posts

Uber Geek

#1542018 26-Apr-2016 10:37

Still haven't got my head around Powershell :)

The advantage of Crashplan or Veeam is versioning, i.e they will have the base image and then changes will also be backed up so you can go back to previous versions etc too.

I guess the disadvantage is you would just have to install Veeam or Crashplan or similar to recover the files from backup, rather than just opening straight of the disk on a different PC like you can with a file copy

gzt

17118 posts

Uber Geek

Lifetime subscriber

#1542028 26-Apr-2016 10:46

Didn't Microsoft make it a lot harder to save user files in random places? ; ).

nathanbarlow

48 posts

Geek

#1542718 27-Apr-2016 12:27

Holy moley @toyonut. I'll check that out!

Thanks to all of you for the many suggestions made.

I might use toyonuts script to start with to get it all onto a USB for my daily use, and then follow up with Veem/Unstoppable Copier/Crashplan or Macrium Reflect Free as my computer backup for the TM2.

Thanks so much for the advice and time everyone has chucked into helping me out. This is why I come to GeekZone. The One Stop Shop for Advice.

Cheers
Nathan

1 | 2