Jump to content


 


Register a free account to unlock additional features at BleepingComputer.com
Welcome to BleepingComputer, a free community where people like yourself come together to discuss and learn how to use their computers. Using the site is easy and fun. As a guest, you can browse and view the various discussions in the forums, but can not create a new topic or reply to an existing one unless you are logged in. Other benefits of registering an account are subscribing to topics and forums, creating a blog, and having no ads shown anywhere on the site.


Click here to Register a free account now! or read our Welcome Guide to learn how to use this site.

Photo

How Did I Get All These Duplicate Files?


  • Please log in to reply
6 replies to this topic

#1 brillo

brillo

  • Members
  • 80 posts
  • OFFLINE
  •  
  • Location:Terrebonne, Oregon
  • Local time:01:23 AM

Posted 17 August 2006 - 07:42 PM

It seems to me that no matter how well managed (defragged, compacted, etc.) the HD is, duplicated files and folders drag down a computer. In addition to wasting HD space, of course. That's why I'm trying to get rid of at least some of the dupes in mine.

In this pursuit, I just dl'ed and ran a duplicate-file finder/cleaner program for the first time. It found 18305 duplicate groups (# of files duped 2X + # of files duped 3X, etc., totalled), 52300 duplicated files, 4.26 GB duplicate size. Holy #^&%! These are TRUE duplicate files (compared byte-to-byte). The dupe program has a "quick and dirty" button to instantly delete all but the first file in each of the dupe groups. That is certainly an easy solution, but it's also dangerous I think. The way I see it, I need to take a look at each of the files, (think about what they do, why they're where they are, if I want to keep them there or put them somewhere else), and then decide if I want to get rid of one or more of the duplicates.

This is complicated. Example: Let's say I have a folder of customized icons, and the folder happens to be duplicated in several places, and other files/folders use those icons, i.e., they get the icons via a path (thread, whatever you call it) to ONE of the duplicated icon source folders. Right? Right. So now what happens if I move or delete the wrong duplicate source folder? No custom icons is what happens. Because the path is a dead end. I know this because I just did it. I deleted the source icon folder -> no icons. Restored it -> icons. Changed the path to the other duped icon source folder -> icons.

Wouldn't this principle apply across the board? If I delete one of two duped files, any path to it will dead end, so I have to change the path to the remaining file. Some files are connected to hundreds of others. So, to delete dupes properly, it appears that I need to check every single path to/from all 52300 duplicated files/folders and reroute every last one if necessary?

Holy #^&%#%^(*$#!

So now that I vented (I feel better already), How DO I get rid of duplicate files responsibly, i.e., without using the "quick and dirty" button, AND do it in a reasonable length of time, in hours, not days or weeks. Is there some kind of tutorial or guru geek's web site explaining how to do this in a methodical, organized way? (I've looked and haven't found one) And most important, whatever the answer, it has to be understandable to an average-joe end-user like me.

And then... I could really use some way of preventing dupes in the future; not duplicating the problem someone might say. I wouldn't say that of course, but someone might. So if you have a handy-dandy method to prevent dupes, let me know that, too.

Thanks for any suggestions. They are greatly appreciated.

Rob

Edited by brillo, 18 August 2006 - 02:51 AM.


BC AdBot (Login to Remove)

 


#2 jgweed

jgweed

  • Members
  • 28,473 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Chicago, Il.
  • Local time:03:23 AM

Posted 17 August 2006 - 11:09 PM

It might help if we knew which application you downloaded to find the duplicates; it may be the output can be manipulated or sorted in such a way as to make deletion an easier process.
If you could give one or two examples of the duplicate files (including the path, of course), it might help us understand what KIND of files are being duplicated and this would suggest a cause of it in the first place.
Regards,
John
Whereof one cannot speak, thereof one should be silent.

#3 usasma

usasma

    Still visually handicapped (avatar is memory developed by my Dad


  • BSOD Kernel Dump Expert
  • 25,091 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Southeastern CT, USA
  • Local time:04:23 AM

Posted 18 August 2006 - 12:10 PM

Windows alone has many, many duplicate files scattered throughout your system. And deleting them may cause your system to malfunction.

Then, other programs do this also - so I can imagine the confusion that it causes.

The solution here is research - you'll have to research the filenames and see if there's any problem with deleting them. This will take a long, long time - but in the end you'll have a better handle on what's needed and what's not, and a cleaner system to go with it!
My browser caused a flood of traffic, sio my IP address was banned. Hope to fix it soon. Will get back to posting as soon as Im able.

- John  (my website: http://www.carrona.org/ )**If you need a more detailed explanation, please ask for it. I have the Knack. **  If I haven't replied in 48 hours, please send me a message. My eye problems have recently increased and I'm having difficult reading posts. (23 Nov 2017)FYI - I am completely blind in the right eye and ~30% blind in the left eye.<p>If the eye problems get worse suddenly, I may not be able to respond.If that's the case and help is needed, please PM a staff member for assistance.

#4 rubiconeye

rubiconeye

  • Members
  • 156 posts
  • OFFLINE
  •  
  • Local time:08:23 AM

Posted 18 August 2006 - 04:23 PM

hi, brillo, as usasma states there will be many dupe files on any system and in reality these are usually small in size so you are not going to retrieve much space for the effort of going thru the list file by file. saying that, 4.26 gig seems overly excessive for duplicate files. perhaps it would be better to maybe check the top 10 or 15 files by size, see if there is one particular app that is creating them, and if so, why?

as you said yourself, using these file dupe finder appz can cause a lot more problems if used indiscriminately. did you run any cleaner appz like 'ccleaner' before you ran this file dupe finder? maybe many of the dupe files are in folders that can be safely deleted by an app like 'ccleaner'

#5 brillo

brillo
  • Topic Starter

  • Members
  • 80 posts
  • OFFLINE
  •  
  • Location:Terrebonne, Oregon
  • Local time:01:23 AM

Posted 19 August 2006 - 02:37 AM

Thanks for responding. I'll try your suggestions.

But...

Is there a program, web site, or tutorial that tells how to responsibly delete duped files in a reasonable lenth of time, like hours, not days or weeks, AND explains it in a way that is comprehensible to an average-joe end-user like me? I assume from the responses, the answer is "no".

And since I don't want to be periodically deleting dupes ad nauseum.... Do you have/know of a program that notifies the user when dupes occur, and then provides a simple way to reroute the new path back to the original file so the dupe can be deleted, or, preferably, does all that automatically? Again, I assume from the responses that the answer is "no".

There should be such a program. It should have been one of the first programs ever written. An OS that requires duplicate files all over the place to run right, isn't right. Some redundency is understandable; sloppiness isn't.

That's how I see it.

Rob

#6 usasma

usasma

    Still visually handicapped (avatar is memory developed by my Dad


  • BSOD Kernel Dump Expert
  • 25,091 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Southeastern CT, USA
  • Local time:04:23 AM

Posted 20 August 2006 - 08:37 AM

There are such programs - a quick google search for "duplicate finder" or "duplicate file locator" will turn up many of them.

I agree - having duplicates isn't right, but that's the nature of the software industry (why create a new file when an old one will do).

Actually, deleting duplicates is very easy - but does require a bit of studying and experimentation.

As rubiconeye suggests, start with the biggest first. But don't just delete them - move them to another location. then work with the system a bit. If there's no problem, then delete them to the Recycle Bin (as an extra safety measure). Eventually the Recycle Bin will empty and they'll be gone.

Avoid deleting duplicates in the Windows folder - although there may be some that you can remove from there - removing the wrong one's can really hose your system. In particular, there's copies of some essential files that are kept there to replace one's that get corrupted. If you delete them, nothing will go wrong...until one of the originals needs replacing and the system won't be able to do it. And, this may result in the system not even booting into Windows!

Finally, I've done the "delete duplicates" thing on many occasions - and I find that the more time that I take, the less likely I am to hose my system. But, I'm prepared to reinstall my OS at any time, and have set things up so that I don't lose everything if Windows fails to boot.
My browser caused a flood of traffic, sio my IP address was banned. Hope to fix it soon. Will get back to posting as soon as Im able.

- John  (my website: http://www.carrona.org/ )**If you need a more detailed explanation, please ask for it. I have the Knack. **  If I haven't replied in 48 hours, please send me a message. My eye problems have recently increased and I'm having difficult reading posts. (23 Nov 2017)FYI - I am completely blind in the right eye and ~30% blind in the left eye.<p>If the eye problems get worse suddenly, I may not be able to respond.If that's the case and help is needed, please PM a staff member for assistance.

#7 covimom

covimom

  • Members
  • 2 posts
  • OFFLINE
  •  
  • Local time:03:23 AM

Posted 01 November 2006 - 11:16 PM

I have the same problem can easily find them if I some show up in the start / programs / then they are listed as a duplicate shortcuts (2) (3) ect - i guess no big problem just delete shortcut copy BUT A REAL PAIN AS THERE ARE SO MANY -- BIGGER ISSUE IS through windows explorer open up any one of many files and i find them for instance C:\program files \windows NT\accessories\mswrd6 again (2) again (2)(2) (3) yada yada yaday .... does anyone have an idea why ?????? how to stop

Edited by covimom, 01 November 2006 - 11:19 PM.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users