Jump to content


 


Register a free account to unlock additional features at BleepingComputer.com
Welcome to BleepingComputer, a free community where people like yourself come together to discuss and learn how to use their computers. Using the site is easy and fun. As a guest, you can browse and view the various discussions in the forums, but can not create a new topic or reply to an existing one unless you are logged in. Other benefits of registering an account are subscribing to topics and forums, creating a blog, and having no ads shown anywhere on the site.


Click here to Register a free account now! or read our Welcome Guide to learn how to use this site.

Photo

help - corrupt JPGs on external hard drive


  • Please log in to reply
6 replies to this topic

#1 reggiereg

reggiereg

  • Members
  • 151 posts
  • OFFLINE
  •  
  • Gender:Male
  • Local time:06:09 AM

Posted 28 April 2012 - 06:10 AM

Hi all,

I recently got myself a new external hard drive, and proceed to move all my data to the new, bigger one.

During this process, I used a fantastic program called "Beyond Compare" to check the integrity of data between the old and new drive,
which resulted in me realising that I had lots of corrupt JPG files.

I have three 500gb external drives with identical data on each, thus managed to recover most of the files from each of the three drives.

I found a good, but relativity unstable JPG error checking program called "Badpeggy" and wondered if:

1) anyone knows a stable free program to scan through hundreds of GB of JPG, RAW and AVI files to check for errors?

2) why the files may have been damaged, and what is the best way to protect them in the future?

3) if its worth backing them up to DVD (350GB) for reliable archive?

I'd appreciate your expert advice, as I'm now very worried of loosing the pictures of my little boy as they're all digital!

Thanks in advance Reggie.

BC AdBot (Login to Remove)

 


#2 hamluis

hamluis

    Moderator


  • Moderator
  • 56,290 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Killeen, TX
  • Local time:12:09 AM

Posted 28 April 2012 - 01:42 PM

1) anyone knows a stable free program to scan through hundreds of GB of JPG, RAW and AVI files to check for errors?

Well...first thing is to somewhat understand some of the things which may result in file damage/corruption of data.

Causes of Data Corruption - http://ezinearticles.com/?Causes-of-Data-Corruption&id=817785

Data corruption and loss causes and avoidance - http://www.thexlab.com/faqs/datacorruption.html

As you can see...it's difficult to pin down an exact cause for every case of data corruption. I would say that the general tool used for attempting to overcome such...would be the chkdsk /r command within Windows, the general file system tool.

Back in the floppy disk era...files on floppies might possibly become corrupt at some time...the general fix was to run ScanDisk (precursor of chkdsk /r command) on the floppy and hope for the best.

2) why the files may have been damaged, and what is the best way to protect them in the future?
Covered above.

3) if its worth backing them up to DVD (350GB) for reliable archive?

"Reliable" is a word I seldom associate with any type of media...hard drive, CD, DVD, USB device...because there are too many unknowns. "Reasonable" would be the concept I apply to computer storage.

Every user has a preference...indulge in yours, being aware that nothing lasts forever or is guaranteed to be there when you want it. I prefer using hard drives to make clones of my valued data...to CD/DVDs or other media. No guarantee that a given drive will work when I want it to, but it's what I have confidence in.

If you do any research, you can see that CDs and DVDs cannot be expected to last forever, nor can hard drives. But each provides a reasonable attempt at storing valued files and that's all that I believe that I should expect. My take on data storage is that...the goal is to provide at least one additional copy of the things I consider valued...for the time when things go wrong with my system. I assume that things will go wrong, as opposed to those who do not believe such...and who do not backup their files in any routine manner.

Some Google Links that may help you determine how you want to approach backing up.

Louis

#3 reggiereg

reggiereg
  • Topic Starter

  • Members
  • 151 posts
  • OFFLINE
  •  
  • Gender:Male
  • Local time:06:09 AM

Posted 30 April 2012 - 04:27 AM

Thanks Louis for your eloquent reply.

I have read the links that you provides and checked the write cashing on my external drives (which were off), but am unsure as to what actually caused the errors in the first place. I take your point about it being "difficult to pin down"; so all I can do is duplicate my data and hope for the best?

Because I cannot find out what went wrong, this gives me little confidence in preventing it in the future.

therefore my stratigraphy is to:

1) duplicate my data on three separate external hard drives
2) each month synchronise them as my photo library grows
3) use tools to check (badpeggy for jpgs) entire content each month for errors
4) defrag and check for disc errors each month (chkdsk/r)
5) annually format, and re copy entire library back to each hard drive, to provide a fresh copy on the magnetic media.
6) hope for best!

Does this sound ok?

Questions:

10) I'm worried that if I have errors they will be copped across during the syncing process, is there a way to prevent this?
11) I need an automated tool to check raw and avi files automatically, does anyone know of such a tool/script?


Kind regards,

Reggie.

Edited by reggiereg, 30 April 2012 - 04:28 AM.


#4 rotor123

rotor123

  • Moderator
  • 8,093 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:New Jersey
  • Local time:01:09 AM

Posted 30 April 2012 - 11:59 AM

Hi all,

I recently got myself a new external hard drive, and proceed to move all my data to the new, bigger one.

During this process, I used a fantastic program called "Beyond Compare" to check the integrity of data between the old and new drive,
which resulted in me realising that I had lots of corrupt JPG files.

I have three 500gb external drives with identical data on each, thus managed to recover most of the files from each of the three drives.

I found a good, but relativity unstable JPG error checking program called "Badpeggy" and wondered if:

1) anyone knows a stable free program to scan through hundreds of GB of JPG, RAW and AVI files to check for errors?


Sorry I do not. I also do not really believe a program can do that reliably.
I use the try them and see if they work method for testing of all three types of file.

The other problem is I do not see how "Beyond Compare" could identify corrupt files instead of a file transfer that corrupted the files. If the source was corrupted I would expect the destination to be corrupt the same way and thus pass verification.

2) why the files may have been damaged, and what is the best way to protect them in the future?


Since they are on a USB external hookup. Do you stop them or shut down the computer before unplugging them? Failure to do so can cause corruption. The other possibility is that the USB ports are not reliable and are corrupting the data while transferring. Or a Windows problem.

I'm not sure why you are seeing such data corruption. I put video ranging from HD 1080i captures, AVI, MKV, MP4 etc. and MP3 and JPGs onto external drives to use with my Media player and I never see the problem you are describing.

3) if its worth backing them up to DVD (350GB) for reliable archive?


I would never back up that much data to DVDs. I would however buy three 50 pc spindles of Verbatim HTL (Not LTH) BluRay blank media and burn them in my Pioneers BluRay burner. Of the three brands of BD burner I have in terms of best to worst the Pioneer is first followed by the LiteOn and last the LG burner. I trust the Verbatim media to last better than the cheaper brands.

I have in fact been doing this to data I would hate to lose due to a hard drive death or virus.

Once they are burned you can do random sampling since many BluRay players can play JPGs and certain types of video.

I'd appreciate your expert advice, as I'm now very worried of loosing the pictures of my little boy as they're all digital!

Thanks in advance Reggie.


And of course none of this is 100% guaranteed.

The best course is multiple types of back up. Optical disk and hard drive and if affordable a online backup service as the third. The Online service such as Carbonite is to guard against natural disasters as well as things like break ins and fire.

You would of course have to have a decent upload speed to use the online backup service. And try to find one that will there in the long term.

Another option would be to take the digital images and print them on a color laser and in the future if your camera has removable memory store it away and do not use it just as if it were a film camera where you had to keep buying rolls of film.

And of course in my world view, things happen and life is to short to spend it worrying about things beyond my control.

I have lost files for various reasons. I shrugged my shoulders and said to myself Dang it (more or less) and moved on.

Good Luck
Roger

Edited by rotor123, 30 April 2012 - 12:15 PM.

Fortune Cookie says: Fortune not Found: Abort, Retry, Ignore?

Sent from my All-In-One Desktop. Perfect for Internet, Not for heavy usage or gaming however.

How Does a computer get Infected? http://www.bleepingcomputer.com/forums/t/2520/how-did-i-get-infected/
Forum Rules,    The BC Welcome Guide

167 @ June 2015


#5 reggiereg

reggiereg
  • Topic Starter

  • Members
  • 151 posts
  • OFFLINE
  •  
  • Gender:Male
  • Local time:06:09 AM

Posted 01 May 2012 - 07:19 AM

Hi Roror123,

I like the "try them and see if they work method"!

I don't think I explained myself clearly enough, upps. I used "Beyond Compare" to compare my master drive A to slave drives B and C.
It appearers that most of the corrupt files were in drive A, so I used beyond compare's "binary comparison" to identify differences between A&B and A&C.

This method only picked up on differences on what should have been identical data on identical drives; thus identified corrupt files, if they were only on one drive only.
Luckily I was able to find a good file from either B or C.

What worries me now is:

1) why this happened mainly to drive A (probably never know)?
2) could drive A be on its way out and how can I find out?
3) How to find other corrupt files, without looking at 80K pictures and videos.


I have always shut down or removed drives appropriately; I have them set up to disable write cashing (default in windows 7) but never looked into write cashing on my old windows XP. laptop. Most of the corrupt files are quite old, so I wonder if there was something wrong with my old system or the length of time the data has been stored on the drive magnetic media.

Probably will never know.

Looks like iv'e got a lot of pictures to flick through, so I better stop typing!

Thanks again,

Reggie.

#6 steve4444

steve4444

  • Members
  • 2 posts
  • OFFLINE
  •  
  • Local time:07:09 AM

Posted 07 August 2016 - 06:54 AM

 

A bit late.... but may be useful to someone.
I removed the hard drive from the usb case and plugged directly into the sata interface of the motherboard and all the files were undamaged!

The issue seems to be caused by some read error through the usb interface.



#7 steve4444

steve4444

  • Members
  • 2 posts
  • OFFLINE
  •  
  • Local time:07:09 AM

Posted 07 August 2016 - 06:55 AM

 

A bit late.... but may be useful to someone.
I removed the hard drive from the usb case and plugged directly into the sata interface of the motherboard and all the files were undamaged!

The issue seems to be caused by some read error through the usb interface.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users