Wanted: Software to Find Duplicate Image Files on Hard Drive

On Sat, 8 Dec 2007 19:18:07 -0800 (PST), HKEK <coolmale.RemoveThis@hotmail.com>
wrote:

>I am looking for software that can eliminate dulplicate image files on
>my hard drive.
>Any recommendations? -- Thanks!

DeDuper is a small freeware program that finds exact duplicates, but
is limited to jpg and bmp. It's quite fast.

http://www.niftyprograms.com/deduper.htm

VisiPics is another freeware program that finds exact dups and near
dups, depending on settings used. It can compare jpg, bmp, tiff, png,
etc. Not quite as fast as DeDuper but more flexible.

http://www.download.com/VisiPics/3000-2193_4-10712877.html
jimbok
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

jimbok wrote:
> On Sat, 8 Dec 2007 19:18:07 -0800 (PST), HKEK <coolmale.DeleteThis@hotmail.com>
> wrote:
>
>> I am looking for software that can eliminate dulplicate image files
>> on my hard drive.
>> Any recommendations? -- Thanks!
>
> DeDuper is a small freeware program that finds exact duplicates, but
> is limited to jpg and bmp. It's quite fast.
>
> http://www.niftyprograms.com/deduper.htm
>
> VisiPics is another freeware program that finds exact dups and near
> dups, depending on settings used. It can compare jpg, bmp, tiff, png,
> etc. Not quite as fast as DeDuper but more flexible.
>
> http://www.download.com/VisiPics/3000-2193_4-10712877.html
> jimbok

There is a program called Windows which does that very well if you know the
names of the files which may be duplicated. You can get it from Microsoft.
Aren't they in Seattle or somewhere?

DP
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

This message is not archived
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

HKEK wrote:
> I am looking for software that can eliminate dulplicate image files on
> my hard drive. I would like this software to find all duplicates so I
> can eliminate them before backing up the image files to a DVD.
> Ideally, the software should be able to identify identical image files
> even if they have different file names [but the same image/pixels]. I
> would imagine that such software exists.
>
> Any recommendations? -- Thanks!
Go to Google and search for "DoubleKiller". I use the free version and
it works very well; you might need the Pro version, but I suggest that
you try the free first. I use it for other types of files which aren't
as big as many image files, but it might serve your purpose.
Allen
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

Dennis Pogson wrote:

> There is a program called Windows which does that very well if you
> know the names of the files which may be duplicated. You can get it
> from Microsoft. Aren't they in Seattle or somewhere?
>
> DP

No, they are in my dustbin. I use debian (much more reliable).

--
Neil
reverse ra and delete l
Linux user 335851
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

Try d'peg http://www.somewareonthe.net/index_dpeg.htm
I've used this to sort through thousands of pictures and it works like a
charm. It's extremely powerful and will find dups that have been
cropped, rotated or color corrected


HKEK wrote:
> I am looking for software that can eliminate dulplicate image files on
> my hard drive. I would like this software to find all duplicates so I
> can eliminate them before backing up the image files to a DVD.
> Ideally, the software should be able to identify identical image files
> even if they have different file names [but the same image/pixels]. I
> would imagine that such software exists.
>
> Any recommendations? -- Thanks!
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

On Sun, 09 Dec 2007 12:34:28 GMT, Ender
<ender1.TakeThisOut@hushmail.com> wrote:

>Try d'peg http://www.somewareonthe.net/index_dpeg.htm
>I've used this to sort through thousands of pictures and it works like a
>charm. It's extremely powerful and will find dups that have been
>cropped, rotated or color corrected

Surely they're not duplicates in that case?

--
John Bean
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

On Sun, 09 Dec 2007 04:17:22 GMT
"Jürgen Exner" <jurgenex.RemoveThis@hotmail.com> wrote:

> HKEK wrote:
> > I am looking for software that can eliminate dulplicate image files
> > on my hard drive. I would like this software to find all duplicates
> > so I can eliminate them before backing up the image files to a DVD.
> > Ideally, the software should be able to identify identical image
> > files even if they have different file names [but the same
> > image/pixels]. I would imagine that such software exists.
>
> I don't know of any pre-existing program but the core algorithm
> shouldn't take more than maybe 10 lines in any decend scripting
> language like e.g. Perl. Because literal comparison of file content
> is quite expensive I would compute and store some checksum, e.g. MD5,
> and do a binary comparision only if the checksums of 2 files match.
>
> Add another 20-30 lines for standard overhead and user interaction
> and you got a nice little script. I may actually do it if I feel
> bored this weekend.

Yup. Humans are all natural born puzzle solvers. What is it that
makes so many of us think we can't write simple software like this?

My first whack would be a hybrid of perl and bash. Stuff this in
a file called "finddups":

#!/usr/bin/perl
while (<>) {
chop;
@tokens = split / +/, $_;
if (exists($files{$tokens[0]})) {
print "$tokens[1] is a duplicate of $files{$tokens[0]}\n";
} else {
$files{$tokens[0]} = $tokens[1];
}
}

Then do something like this:

find / -name \*.jpg -exec md5sum {} \;| ./finddups

Adjust the find as needed if you also have .JPG, .jpeg, .JPEG, or
whatever. It'll take a long time because md5sum is expensive, but
it will identify all the dups on your disk.

Gee, that was fun! Now all it needs is to bring the find and the
MD5 stuff inside the perl script and add a nice Tk GUI. I could
sell it to people who think they can't program and get rich!

Paul Allen
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

John Bean wrote:

> On Sun, 09 Dec 2007 12:34:28 GMT, Ender
> <ender1 DeleteThis @hushmail.com> wrote:
>
>> Try d'peg http://www.somewareonthe.net/index_dpeg.htm
>> I've used this to sort through thousands of pictures and it works like a
>> charm. It's extremely powerful and will find dups that have been
>> cropped, rotated or color corrected
>
> Surely they're not duplicates in that case?


I've seen many politicians reverse themselves any number of times, yet
they remain exactly identical to their previous contents and quality.




mike
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

On Dec 9, 8:02 am, Jake Orsen <nocont....RemoveThis@noplace.org> wrote:
> On Sun, 09 Dec 2007 06:26:15 GMT, jimbok <jimkelREM....RemoveThis@frontiernet.net> wrote:
> >On Sat, 8 Dec 2007 19:18:07 -0800 (PST), HKEK <coolm....RemoveThis@hotmail.com>
> >wrote:
>
> >>I am looking for software that can eliminate dulplicate image files on
> >>my hard drive.
> >>Any recommendations? -- Thanks!
>
> >DeDuper is a small freeware program that finds exact duplicates, but
> >is limited to jpg and bmp. It's quite fast.
>
> >http://www.niftyprograms.com/deduper.htm
>
> >VisiPics is another freeware program that finds exact dups and near
> >dups, depending on settings used. It can compare jpg, bmp, tiff, png,
> >etc. Not quite as fast as DeDuper but more flexible.
>
> >http://www.download.com/VisiPics/3000-2193_4-10712877.html
> >jimbok
>
> Here's anotherhttp://www.bolidesoft.com/imagecomparer.html
>
> Here's an excellent freeware one that was originally designed for use with
> making tiled images from many smaller imageshttp://www.keronsoft.com/dupdetector.html
>
> There's hundreds of such utilities out there. The only ones that don't know
> about these are the resident-trolls in this newsgroup who have zero experience
> with real photography. They become more and more obvious the more that you know
> about real photography.- Hide quoted text -
>
> - Show quoted text -

That one is a bit scary. I don't want to eliminate "similar" shots.
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

"Jake Orsen" <nocontact.RemoveThis@noplace.org> wrote in message
news:vgpnl3dmsg5u0gauta39q0pk59v521641d@4ax.com...
> On Sun, 09 Dec 2007 06:26:15 GMT, jimbok <jimkelREMOVE.RemoveThis@frontiernet.net>
> wrote:
>
> Here's another http://www.bolidesoft.com/imagecomparer.html

That one is a good piece of software, but it's p*ss poor w.r.t. performance
on large image libraries. It's also not very stable, but as I understand it,
the programmer is working on stability...

Cheers,
ink
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive 

ink wrote:
> "Jake Orsen" <nocontact.DeleteThis@noplace.org> wrote in message
> news:vgpnl3dmsg5u0gauta39q0pk59v521641d@4ax.com...
>> On Sun, 09 Dec 2007 06:26:15 GMT, jimbok
>> <jimkelREMOVE.DeleteThis@frontiernet.net> wrote:
>>
>> Here's another http://www.bolidesoft.com/imagecomparer.html
>
> That one is a good piece of software, but it's p*ss poor w.r.t.
> performance on large image libraries. It's also not very stable, but
> as I understand it, the programmer is working on stability...
>
> Cheers,
> ink

A program I wrote for my own use is here:

http://www.david-taylor.myby.co.uk/software/disk.html#FindDuplicates

Free, and unsupported.

Cheers,
David
 >> Stay informed about: Wanted: Software to Find Duplicate Image Files on Hard Drive